Anda di halaman 1dari 623

page_ii

< previous page

page_ii

next page >


Page ii

Language, Speech, and Communication Statistical Language Learning, Eugene Charniak, 1994 The Development of Speech Perception, edited by Judith Goodman and Howard C. Nusbaum, 1994 Construal, Lyn Frazier and Charles Clifton, Jr., 1995 The Generative Lexicon, James Pustejovsky, 1996 The Origins of Grammar: Evidence from Early Language Comprehension, Kathy Hirsh-Pasek and Roberta Michnick Golinkoff, 1996 Language and Space, edited by Paul Bloom, Mary A. Peterson, Merrill F. Garrett, and Lynn Nadel, 1996 Corpus Processing for Lexical Acquisition, edited by Branimir Boguraev and James Pustejovsky, 1996 Methods for Assessing Children's Syntax, edited by Dana McDaniel, Cecile McKee, and Helen Smith Cairns, 1996 The Balancing Act: Combining Symbolic and Statistical Approaches to Language, edited by Judith Klavans and Philip Resnik, 1996 The Discovery of Spoken Language, Peter W. Jusczyk, 1996

< previous page

page_ii
If you like this book, buy it!

next page >

file:///D|/Junk/2_0262522667_Language_and_Space/page_ii.html[8/20/2009 12:17:21 AM]

page_iii

< previous page

page_iii

next page >


Page iii

Language and Space edited by Paul Bloom, Mary A. Peterson, Lynn Nadel, and Merrill F. Garrett A Bradford Book The MIT Press Cambridge, Massachusetts London, England

< previous page

page_iii
If you like this book, buy it!

next page >

file:///D|/Junk/2_0262522667_Language_and_Space/page_iii.html[8/20/2009 12:17:21 AM]

page_iv

< previous page

page_iv

next page >


Page iv

1996 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. This book was set in Times Roman by Asco Trade Typesetting Ltd., Hong Kong and was printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Language and space/ edited by Paul Bloom . . . [et al.]. p. cm.(Language, speech, and communication) Papers presented at a conference of the same name which was held Mar. 16-19, 1994, Tucson, Ariz. "A Bradford book." Includes bibliographical references and index. ISBN 0-262-02403-9 1. Space and time in languageCongresses. I. Bloom, Paul, 1963- . II. Series. p37.5.S65L36 1996 401'.9dc20 95-36427

CIP

< previous page

page_iv
If you like this book, buy it!

next page >

file:///D|/Junk/2_0262522667_Language_and_Space/page_iv.html[8/20/2009 12:17:21 AM]

page_v

< previous page

page_v

next page >


Page v

Contents Preface Participants Chapter 1 The Architecture of the Linguistic-Spatial Interface Ray Jackendoff Chapter 2 How Much Space Gets into Language? Manfred Bierwisch Chapter 3 Perspective Taking and Ellipsis in Spatial Descriptions Willem J. M. Levelt Chapter 4 Frames of Reference and Molyneux's Question: Crosslinguistic Evidence Stephen C. Levinson Chapter 5 The Confluence of Space and Language in Signed Languages Karen Emmorey Chapter 6 Fictive Motion in Language and "Ception" Leonard Talmy Chapter 7 The Spatial Prepositions in English, Vector Grammar, and the Cognitive Map Theory John O'Keefe 277 211 171 109 77 31 1 vii ix

< previous page

page_v
If you like this book, buy it!

next page >

file:///D|/Junk/2_0262522667_Language_and_Space/page_v.html[8/20/2009 12:17:22 AM]

page_vi

< previous page

page_vi

next page >


Page vi

Chapter 8 Multiple Geometric Representations of Objects in Languages and Language Learners Barbara Landau Chapter 9 Preverbal Representation and Language Jean M. Mandler Chapter 10 Learning How to Structure Space for Language: A Crosslinguistic Perspective Melissa Bowerman Chapter 11 Space to Think Philip N. Johnson-Laird Chapter 12 Spatial Perspective in Descriptions Barbara Tversky Chapter 13 A Computational Analysis of the Apprehension of Spatial Relations Gordon D. Logan and Daniel D. Sadler Chapter 14 The Language-to-Object Perception Interface: Evidence from Neuropsychology Tim Shallice Chapter 15 Space and Language Mary A. Peterson, Lynn Nadel, Paul Bloom, and Merrill F. Garrett Name Index Subject Index 553 579 585 531 493 463 437 385 365 317

< previous page

page_vi
If you like this book, buy it!

next page >

file:///D|/Junk/2_0262522667_Language_and_Space/page_vi.html[8/20/2009 12:17:22 AM]

Preface

The present volume consistsof chapters by participants in the Language and Space . In most casesthe chapters conferenceheld in Tucson, Arizona , 16- 19 March 1994 have beenwritten to reflect the numerous interactions at the conference , and for that reason we hope the book is more than just a compilation of isolated papers. The conferencewas truly interdisciplinary , including such domains as neurophysiology, , and linguistics. Neural , psychology, anthropology , cognitive science neuropsychology es mechanisms , and cultural factors were all grist for the , developmental process mill , as were semantics , syntax, and cognitive maps. The conferencehad its beginnings in a seemingly innocent conversation in 1990 betweentwo new colleaguesat the University of Arizona (Bloom and Peterson ), who MAP .) assumed of them confusions. One of left right wondered about the genesis ( that theseconfusions reflecteda languageproblem; the other (P. B.) was quite certain that they reflected a visual perceptual problem. Curiously, it was the perception researcherwho saw this issueas being mainly linguistic and the languageresearcher who saw it as mainly perceptual. In true academic form they decided that the best way to arrive at an answer would be to hold a seminar on the topic , which they did the very next year. Their seminar on languageand spacewas attended by graduate students , postdoctoral fellows, and many faculty membersfrom a variety of departments . Rather than answering the question that led to its inception, the seminar ? What aspectsof spacecan we raised other questions: How do we represent space ? And what role doesculture play in talk about? How do we learn to talk about space all thesematters? One seminar could not explore all of theseissuesin any depth; an enlarged group of interestedcolleagues(the four coeditors) felt that perhaps several workshops might . The Cognitive NeuroscienceProgram at the University of Arizona , in collaboration with the Cognitive ScienceProgram and the PsychologyDepartment, sponsored . Although two one-day workshops on the relations between space and language : other rise to still questions How does stimulating and helpful, the workshops gave

VIII

Preface

? How many kinds of spatial representations are there? the brain represent space ? Should What happensto spatial representationsafter various kinds of brain damage to closed restricted be and between relations of the tests language space experimental classlinguistic elementsor must the role of open-classelementsbe consideredas well? Given the scopeof thesequestion, we decidedto invite investigators from a variety . of disciplines to a major scientific conference , and Language and Spacetook shape . We do not imagine that the The conferencewas judged by all to be a great success chaptersin this book provide final answersto any of the questionswe first raised, but we are confident that they add much to the discussionand demonstrate the importance . We expectthat increasedattention of the relations betweenspaceand language will be given to this fascinating subject in the years ahead and hope that our conference , and this book , have made a significant contribution to its understanding. Meetings cannot be held without the efforts of a considerablenumber of people, . Our thanks to Pauline Smalley for all work and the support of many funding sources she did in organizing the conferenceand making sure participants got to the right place at the right time and to Wendy Wilkins , of Arizona State University, for her . We gratefully acknowledgethe gracious help both before and during the conference ' : McDonnell Pew Cognitive NeuroscienceProgram support of the conferences sponsors , the Flinn Foundation Cognitive Neuroscience Program, and the Cognitive ScienceProgram and Department of Psychology at the University of Arizona . We , which greatly thank the participants for their intellectual energy and enthusiasm ' of the MIT Pierce thank we . contributed to the conferences success Finally , Amy Pressfor her help with this volume. Editors Bloom and Petersontosseda coin one eveningover margaritas to determine whose name would go first.

page_ix

< previous page

page_ix

next page >


Page ix

Participants Manfred Bierwisch Structural Grammar Research Unit Humboldt University, Berlin. Paul Bloom Departments of Psychology and Linguistics University of Arizona Melissa Bowerman Max Planck Institute for Psycholinguistics Nijmegen Karen Emmorey Salk Institute for Biological Studies San Diego Merrill F. Garrett Department of Psychology and Cognitive Science Program University of Arizona Ray Jackendoff Linguistics and Cognitive Science Program Brandeis University Philip N. Johnson-Laird Department of Psychology Princeton University Barbara Landau Department of Cognitive Science University of California, Irvine Willem J. M. Levelt Max Planck Institute for Psycholinguistics Nijmegen Stephen C. Levinson Max Planck Institute for Psycholinguistics Nijmegen Gordon Logan Department of Psychology University of Illinois Jean M. Mandler Department of Cognitive Science University of California, San Diego

file:///D|/Junk/2_0262522667_Language_and_Space/page_ix.html[8/20/2009 12:21:56 AM]

page_ix

Lynn Nadel Department of Psychology and Cognitive Science Program University of Arizona John O'Keefe Department of Anatomy and Developmental Biology University College, London

< previous page

page_ix
If you like this book, buy it!

next page >

file:///D|/Junk/2_0262522667_Language_and_Space/page_ix.html[8/20/2009 12:21:56 AM]

page_x

< previous page

page_x

next page >


Page x

Mary A. Peterson Department of Psychology and Cognitive Science Program University of Arizona Daniel D. Sadler Department of Psychology Indiana University of Pennsylvania Tim Shallice Medical Research Council, U.K. Len Talmy Department of Linguistics and Center for Cognitive Science State University of New York, Buffalo Barbara Tversky Department of Psychology Stanford University

< previous page

page_x
If you like this book, buy it!

next page >

file:///D|/Junk/2_0262522667_Language_and_Space/page_x.html[8/20/2009 12:21:55 AM]

Chapter of the Linguistic-Spatial Interface The Architecture ~ Ray Jackendoff

1.1

Introduction

? More specifically How do we talk about what we see , how does the mind / brain encodespatial information (visual or otherwise), how does it encodelinguistic information , and how does it communicate betweenthe two? This chapter lays out some of the boundary conditions for a satisfactory answerto thesequestionsand illustrates the approach with somesampleproblems. The skeleton of an answer appears in figure 1.1. At the language end, speech perception converts auditory information into linguistic information , and speech production converts linguistic information into motor instructions to the vocal tract. Linguistic information includes at least somesort of phonetic/phonological encoding es of visual perception convert retinal information . ! At the visual end, the process of speech . into visual information , which includes at least some sort of retinotopic mapping. The connection betweenlanguageand vision is symbolized by the central it is clear there cannot be a direct relation double-headedarrow in figure 1.1. Because betweena retinotopic map and a phonological encoding, the solution to our problem lies in elaborating the structure of this double-headedarrow.

1.2 Representational Modularity


The overall hypothesisunder which I will elaborate figure 1.1 might be termed Representational , chapter I ) . , chapter 12; Jackendoff 1992 Modularity (Jackendoff 1987 formats distinct in information encodes mind brain that the The generalidea is many / " " for or languagesof the mind. There is a module of mind/ brain responsible each of these formats. For example, phonological structure and syntactic structure are distinct levels of encoding, with distinct and only partly commensurateprimitives and principles of combination. RepresentationalModularity therefore posits that the architecture . Each of the mind / brain devotesseparatemodules to thesetwo encodings

Ray Jackendoff

auditory signals ---.........

...- eye 4 ~ visual information information linguistic ~ motor signals ~ - - - -_ J C \ - - - "' -- Y - - - ---- I ~ ~ - -yVISION LANGUAGE

Figure 1.1 Coarse sketch of the relation betweenlanguageand vision.

of thesemodules is domain-specific (phonology and syntax, respectively ); and (with " in Fodor ' s " certain caveatsto follow shortly) each is informationally encapsulated . Representational modules differ from Fodorian modules in that they ) sense ( 1983 are individuated by the representationsthey processrather than by their function as faculties for input or output ; that is, they are at the scale of individual levels of , rather than being entire faculties such as languageperception. representation A conceptual difficulty with Fodorian Modularity is that it leavesunansweredhow ; modules communicate with each other and how they communicate with Fodor s ' central, nonmodular cognitive core. In particular , Fodor s languageperception module ' " derives " shallow representations - some form of syntactic structure; Fodor s " " " central faculty of " belief fixation operatesin terms of the languageof thought , a " " nonlinguistic encoding. But Fodor doesnot tell us how shallow representations are " " converted to the languageof thought, as they must be if linguistic communication is to affect belief fixation . In effect, the language module is so domain-specific and informationally encapsulatedthat nothing can get out of it to serve cognitive purposes .2 And without a theory of intermodular communication, it is impossible to approach the problem we are dealing with here, namely, how the languageand vision modules manageto interact with each other. es this difficulty by positing, in The theory of RepresentationalModularity address . modulesproposed above, a systemof interfacemodules addition to the representation An interface module communicatesbetweentwo levels of encoding, say Ll and L2 , by carrying a partial translation of information in Ll form into information in L2 : the phonologyform. An interfacemodule, like a Fodorian module, is domain-specific to-syntax interface module, for instance , knows only about phonology and syntax, -purpose audition . Such a module is also in not about visual perception or general : the phonology-to -syntax module dumbly takes whatever formationally encapsulated phonological inputs are available in the phonology representationmodule, translates the appropriate parts of them into (partial) syntactic structures, and delivers them to the syntax representation module, with no help or interference from , say, beliefs about the social context. In short, the communication among languagesof the mind es as well.3 is mediated by modular process

-Spatial The Architecture of the Linguistic Interface


g-p

auditory ............ ........- phonology ~ .. motor


eye ~ retinotopic
. ~

~ syntax

imagistic

/ ,haptic *,,action localization .... auditory


.. ~

audition ,smell ,emotion ,... / , * structure / :..~ conceptual tresentation spatial rep ;

1.2 . Figure less sketch of coarse Slightly

the relation between language and vision .

The levelsof representationI will be working with here, and the interfaces among them, are sketchedin figure 1.2. Each label in figure 1.2 standsfor a level of representation served by a representation module. The arrows stand for interface modules. Double-headedarrows can be thought of either as interface modules that processbi directionally or as pairs of complementary unidirectional modules (the correct choice is an empirical question) . For instance , the phonology-syntax interface functions from left to right in speechperception and from right to left in speechproduction . " " Figure 1.2 expands the linguistic representation of figure 1.1 into three levels involved with language : the familiar levelsof phonology and syntax, plus conceptual structure, a central level of representation that interfaces with many other faculties. " " Similarly, visual representation in figure 1.1 is expandedinto levelsof retinotopic, ' imagistic, and spatial representation , corresponding roughly to Marr s ( 1982 ) primal sketch, 21 0 sketch, and 3 D model, respectively ; the last of theseagain is a central representationthat interfaceswith other faculties. In this picture, the effect of Fodor ian faculty -sized modules emergesthrough the linkup of a seriesof representation and interface modules; communication among Fodorian faculties is accomplishedby interface modules of exactly the same general character as the interface modules within faculties. The crucial interface for our purposeshere is that betweenthe most central levels of the linguistic and visual faculties, conceptual structure and spatial representation . Beforeexamining this interface, we have to discusstwo things: ( I ) the generalcharacter of interfaces betweenrepresentations(section 1.3); and (2) the general character of conceptual structure and spatial representationthemselves (sections 1.4 and 1.5) .

1.3 Character of Interface Mappings


To say that an interface module " translates " between two representations is , strictly speaking , inaccurate . In order to be more precise, let us focus for a moment on the

Ray Jackendotr

interface between phonology and syntax, the two best-understood levels of mental . representation It is obvious that there cannot be a complete translation betweenphonology and syntax. Many details of phonology, most notably the segmentalcontent of words, , many details of syntax, for instance the play no role at all in syntax. Conversely elaborate layering of specifiersand of arguments and adjuncts, are not reflected in phonology. In fact, a complete, information -preserving translation betweenthe two representationswould be pointless; it would in effect make them notational variants - which they clearly are not. The relation between phonology and syntax is actually something more like a partial homomorphism. The two representationsshare the notion of word (and perhaps .4 But ), and they share the linear order of words and morphemes morpheme segmentaland stressinformation in phonology has no direct counterpart in syntax; and syntactic category (N , V , PP, etc.) and case , number, gender , and person features 5 have no direct phonological counterparts. Moreover, syntactic and phonological constituent structures often fail to match. A classicexampleis given in ( I ) . ( I ) Phonological: [ Thisis the cat] [that ate the rat] [that ate the cheese ] Syntactic: [ Thisis [the cat [that ate [the rat [that ate [the cheese ]]]]]] The phonological bracketing, a flat tripartite structure, contrasts with the relentless , English articles cliticize phoright -embeddedsyntactic structure. At a smaller scale nologically to the following word , resulting in bracketing mismatches such as (2) . (2) Phonological: [the [ big]] [ house ] Syntactic: [the [ big [ house ]] Thus, in general, the phonology-syntax interface module createsonly partial correbetweenthesetwo levels . spondences A similar situation obtains with the interface between auditory information and phonological structure. The complex mappingbetweenwaveforms and phonetic segmentation in a sense the relative order of information : a particular auditory preserves cue may provide evidencefor a number of adjacent phonetic segments , and a particular be a number of phonetic segmentmay signaledby , but the adjacent auditory cues " bands" of the in an stream overlapping correspondenceprogress through speech orderly linear fashion. On the other hand, boundaries betweenwords, omnipresentin phonological structure, are not reliably detectable in the auditory signal; contrari -

The Architecture of the Linguistic - Spatial Interface

wise, the auditory signal contains information about the formant frequenciesof the ' speakers voice that are invisible to phonology. So again the interface module takes only certain information from each representation into account in establishing a betweenthem. correspondence These examples show that each level of representation has its own proprietary information , and that an interface module communicates only certain aspects of this information to the next level up- or downstream. Representational modules, : precisely to the extent that they then, are not entirely informationally encapsulated receiveinformation through interface modules, they are influenced by other parts of the mind.6 In addition to general principles of mapping, such as order preservation, an interface module can also make use of specialized learned mappings. The clearest instances of suchmappings are lexical items. For instance , the lexical item cat stipulates that the phonological structure / kret/ can be mapped simultaneously into a syntactic ' noun and into a conceptual structure that encodesthe word s meaning. In other words, the theory of Representational Modularity leads us to regard the lexicon as a learned component of the interface modules within the language faculty (see Jackendoff forthcoming) .

Structure 1.4 Conceptual


Let us now turn to the crucial modules for the connection of language and spatial cognition : conceptual structure (CS) and spatial representation (SR) . The idea that these two levels share the work of cognition is in a sensea more abstract version . To use the terms of Mandler (chapter 9, of Paivio' s ( 1971 ) dual coding hypothesis this volume), Tversky (chapter 12, this volume), and Johnson- Laird (chapter II , this " " volume), CS encodes" propositional representations , and SR is the locus of image " " " . schema or mental model representations in , 1990 ) is an encoding of Conceptual structure, as developed Jackendoff ( 1983 linguistic meaning that is independent of the particular languagewhose meaning it . It is an " algebraic" representation encodes , in the sensethat conceptual structures are built up out of discrete primitive features and functions. Although CS supports " " formal rules of inference , in , it is not propositional in the standard logical sense that ( I ) propositional truth and falsity are not the only issueit is designedto address , and (2) unlike propositions of standard truth -conditional logic, its expressionsrefer not to the real world or to possibleworlds, but rather to the world as we conceptualize it . Conceptual structure is also not entirely digital , in that some conceptual features and some interactions among features have continuous (i.e., analog) characteristics effectsto be formulated. that permit stereotypeand family resemblance

Ray Jackendoff

The theory of conceptualstructure differs from most approach es to model-theoretic ' " " semanticsas well as from Fodor s ( 1975 ) Languageof Thought , in that it takes for " " grant~ that lexical items have decompositions ( lexical conceptual structures, or LCSs) made up of features and functions of the primitive vocabulary. Here the approach concurs with the main traditions in lexical semantics(Miller and JohnsonLaird 1976 ; Lehrer and Kittay 1992 ; Pinker 1989 ; Pustejovsky 1995 , to cite only a few . parochial examples ) As the mental encoding of meaning, conceptual structure must include all the . A sample : nonsensorydistinctions of meaning made by natural language I . CS must contain pointers to all the sensorymodalities, so that sensoryencodings and correlated (seenext section may be accessed ). 2. CS must contain the distinction betweentokens and types, so that the concept of an individual (say a particular dog) can be distinguished from the concept of the type to which that individual belongs (all dogs, or dogs of its breed, or dogs that it lives with , or all animals) . 3. CS must contain the encoding of quantification and quantifier scope . 4. CS must be able to abstract actions (say running) away from the individual performing the action (say Harry or Harriet running) . 5. CS must encodetaxonomic relations (e.g., a bird is a kind of animal) . 6. CS must encodesocial predicatessuch as " is uncle of ," " is a friend of ," " is fair ," and " is obligated to." " " 7. CS must encode modal predicates , such as the distinction between is flying, " " can " and " " isn' t ' " can t fly . flying , fly , I leaveit to my readersto convince themselves that none of theseaspectsof meaning can be representedin sensoryencodings without using special annotations (such as , or footnotes); CS is, at the very least, the systematicform in which pointers, legends such annotations are couched. For a first approximation, the interface between CS and syntax preservesembedding relations among constituents. That is, if a syntactic constituent X express es the CS constituent X ' , and if another syntactic constituentY express es the CS constituent Y' , and if X contains Y, then, as a rule, X ' contains Y' . Moreover, a verb (or other argument-taking item) in syntax corresponds to a function in CS, and the subject and object of the verb normally correspond to CS argumentsof the function . Hence much of the overall structure of syntax corresponds to CS structure. (Some instancesin which relative embeddingis not preservedappear in Levin and Rapoport 1988and Jackendoff 1990 , chapter 10.) Unlike syntax, though, CS has no notion of linear order: it must be indifferent as to whether it is expressedsyntactically in , say, English, where the verb precedes

-SpatialInterface of the Linguistic TheArchitecture

the direct object, or Japanese , where the verb follows the direct object. Rather, the 7 embeddingin CS is purely relational. At the same time, there are aspectsof CS to which syntax is indifferent. Most prominently , other than argument structure, much of the conceptual material bundled up inside a lexical item is invisible to syntax, just as phonological features are. As far as syntax is concerned , the meaningsof cat and dog (which have no argument structure) are identical, as are the meanings of eat and drink (which have the same argument structure) : the syntactic reflexes of differences in lexical meaning are . extremely coarse In addition , some bits of material in CS are absent from syntactic realization ), is (3) . , given by Talmy ( 1978 altogether. A good example (3) The light flashed until dawn. . But this repetition is The interpretation of (3) contains the notion of repeatedflashes not coded in the verbflash : Thelight flashed normally denotesonly a single flash. Nor is the repetition encodedin until dawn, because , Bill slept until dawndoes , for instance of notion . Rather, the not imply repeatedacts of sleeping (a) repetition arisesbecause ; (b) the light until dawn givesthe temporal bound of an otherwise unbounded process make these c to and bounded therefore event and a is ; ) ( temporally point flashed " coercion" 1991 Jackendoff or construal of a ; (Pustejovsky compatible, principle This notion of . in time out 1991 by repetition ) interprets the flashing as stretched repetition, then, appearsin the CS of (3) but not in the LCS of any of its words. The upshot is that the correspondencebetween syntax and CS is much like the correspondencebetweensyntax and phonology. Certain parts of the two structures are in fairly regular correspondenceand are communicated by the interface module, but many parts of each are invisible to the other. Even though CS is universal, languagescan differ in their overall semantic patterns . First , languagescan have different strategiesin how , in at least three respects bundle , Talmy up conceptual elementsinto lexical items. For example they typically ) documents how English builds verbs of motion primarily by bundling up ( 1980 motion with accompanying manner, while Romance languagesbundle up motion primarily with path of motion , and Atsugewi bundles up motion primarily with the type of object or substanceundergoing motion . Levinson (chapter 4, this volume) shows how the Guugu Yimithirr lexicon restricts the choice of spatial frames of referenceto cardinal directions (see section 1.8) . These strategies of lexical choice . ( This is affect the overall grain of semanticnotions available in a particular language of course in addition to differencesin meaning among individual lexical items across , such as the differences among prepositions discussed by Bowerman, languages chapter 10, this volume.)

T RayJackendot

Second , languagescan differ in what elementsof conceptual structure they require the speakerto expressin syntax. For example , French and Japanese require speakers to differentiate their social relation to their addressee always , a factor largely absent from English. Finnish and Hungarian require speakersto expressthe multiplicity (or , using iterative aspect , a factor absent from English, as seenin repetition) of events (3) . On the other hand, English requiresspeakersto expressthe multiplicity of objects . by using the plural suffix, a requirement absent in Chinese Third , languages can differ in the specialsyntactic constructions they useto express particular conceptual notions. Examples in English are the tag question (They shoot ' " " ' horses , don t they?), the One more construction (One more beer and I m leaving ) " " (Culicover 1972 ), and the The more . . . , the more construction ( The more you drink , the worseyou feel ). These all convey special nuancesthat go beyond lexical mean Ing . " level -specific" semantic 1 have argued (Jackendoff 1983 ) that there is no language of representation intervening between syntax and conceptual structure. Language differences in semantics of the sort listed are in localized the interface specific just between syntactic and conceptual structures. 1 part company here with Bierwisch ( 1986 ), Partee ( 1993 ), and to a certain extent Pinker ( 1989 ) . Within my approach, a , in part becausethe syntax- CS interface module separatesemanticlevel is unnecessary has enough richnessin it to capture the relevant differences ; 1 suspectthat these other theories have not considered closely enough the properties of the interface. However, the issuesare at this point far from resolved . The main point , on which Bierwisch, Pinker, and 1agree(I am unclear about Partee ), is that there is alanguageindependent and universal level of CS, whether directly interfacing with syntax or mediated by an intervening level.

1.5 SpatialRepresentation
- the encoding of objects and their configurations For the theory of spatial representation - we are on far shakier ground. The best articulated (partial) in space theory ' of spatial representation I know of is Marr ' s ( 1982 ) 3-D model, with Biedermans " " ( 1987 ) geonic constructions as a particular variant. Here are some criteria that a spatial representation(SR) must satisfy. I . SR must encode the shapeof objects in a form suitable for recognizing an object at different distancesand from different perspectives , that is, it must solve the classic 8 of . problem object constancy 2. SR must be capable of encoding spatial knowledge of parts of objects that cannot be seen , for instance , the hollownessof a balloon.

The Architecture

of the Linguistic - Spatial Interface

3. SR must be capableof encoding the degrees of freedom in objects that canchange their shape , for instance , human and animal bodies. 4. SR must be capable of encoding shapevariations among objects of similar visual , making explicit the range of shape variations characteristic of type, for example different cups. That is, it must support visual object categorizationas well as visual object identification. 5. SR must be suitable for encoding the full spatial layout of a sceneand formediating " among alternative perspectives( What would this scene look like from over " there? ), so that it can be used to support reaching, navigating, and giving instructions (Tversky, chapter 12, this volume) . 6. SR must be independentof spatial modality , so that haptic information , information from auditory localization, and felt body position (proprioception) can all be brought into registration with one another. It is important to know by looking at an object where you expect to find it when you reach for it and what it should feel like when you handle it . , criteria 5 and 6 go beyond the Marr and Biederman theories of Strictly speaking . But there is nothing in principle to prevent thesetheories from serving object shape as a component of a fuller theory of spatial understanding, rather than strictly as theories of high-level visual shape recognition. By the time visual information is converted into shapeinformation , its strictly visual character is lost- it is no longer ' - nor , as Marr stress es retinotopic , for example , is it confined to the observers point 9 ofview . SR contrasts with CS in that it is geometric (or even quasi-topological) in character , rather than algebraic. But on the other hand, it is not " imagistic" - it is not to be " " thought of as encoding statuesin the head. An image is restricted to a particular point of view, whereasSR is not . An image is restricted to a particular instance of a ' category (recall Berkeley s objection to imagesas the vehicle of thought : how can an image of a particular triangle stand for all possible triangles?! O ), whereasSR is not. An image cannot representthe unseenparts of an object- its back and inside, and ' s view other the parts of it occluded from the observer by objects- whereasSR does. An image is restricted to the visual modality , whereas SR can equally well encode information receivedhaptically or through proprioception. Nevertheless , even though SRs are not themselves imagistic, it makessenseto think of them as encoding image schemas : abstract representationsfrom which a variety of imagescan be generated . Figure 1.2 postulates a separatemodule of imagistic (or pictorial ) representation one level toward the eye from SR. This correspondsroughly to Marr ' s 2t -O sketch. It is specifically visual; it encodeswhat is consciouslypresent in the field of vision or visual imagery (Jackendoff 1987 , chapter 14 ) . The visual imagistic representation is

Ray JackendofT

restricted to a particular point of view at anyone time; it doesnot representthe backs and insides of objects explicitly . At the sametime, it is not a retinotopic representation becauseit is normalized for eye movementsand incorporates information from both eyesinto a single field, including stereopsis . (There is doubtlessa parallel imagistic representationfor the haptic faculty , encoding the way objects feel, but I am not aware of any researchon it .) It is perhapsuseful to think of the imagistic representationas " perceptual" and SR as " cognitive" ; the two are related through an interface of the general sort found in the languagefaculty : they sharecertain aspects , but each has certain aspectsinvisible to the other. Each can drive the other through the interface: in visual perception, an imagistic representation gives rise to a spatial representation that encodesone' s ; in visual imagery, SRs give rise to imagistic representations understanding of the visual scene . In other words, the relation of images to image schemas(SRs) in the to thoughts. Image schemas are present theory is much like the relation of sentences not skeletal images , but rather structures in a more abstract and more central form of representation . 11 This layout of the visual and spatial levels of representation is of course highly , I have not addressedthe well-known division of visual oversimplified. For instance " and the " where " " labor between the what system , which deal, roughly system ' , with object identification and object location respectively (O Keefe and speaking Nadel 1978 rod 1994 ; Ungerleider and Mishkin 1982 ; Farah et al. 1988 ; Jeanne ; Landau and Jackendoff 1993 ). My assumption, perhaps unduly optimistic , is that such division of labor can be captured in the present approach by further articulation of the visual-spatial modules in figure 1.2 into smaller modules and their interfaces , much as figure 1.2 is a further articulation of figure 1.1.

1.6 Interface between CS andSR We comeat last to the mappingbetween CS and SR, the crucial link between the 12 visualsystem and the linguisticsystem . What do these two levels share , suchthat it
is possiblefor an interface module to communicate betweenthem? The most basic unit they share is the notion of a physical object, which appearsas a geometrical unit in SR and as a fundamental algebraic constituent type in CS. 13In addition , the Marr -Biedermantheory of object shapeproposesthat object shapesare decomposedinto geometric parts in SR. This relation maps straightforwardly into the part -whole relation , a basic function in CS that of course generalizesfar beyond object parts. The notions of place (or location) and path (or trajectory) playa basic role in CS ; Jackendoff 1983 (Talmy 1983 ; Langacker 1986 ); they are invoked, for instance , in

The Architecture of the Linguistic -Spatial Interface

locational sentences such as The book is lying on tile table (place ) and The arrow flew can be checked against through tile llir past my llead (path) . Becausethesesentences visual input , and because locations and paths can be given obvious geometric counterparts, it is a good bet that these constituents are shared between CS and SR. 14(The Marr - Biederman theory does not contain placesand paths becausethey arise only in encoding the behavior of objects in the full spatial field, an aspect of visual cognition not addressedby thesetheories.) The notion of physical motion is also central to CS, and obviously it must be representedin spatial cognition so that we can track moving objects. More specula tively, the notion of force appearsprominently in CS (Talmy 1985 ; Jackendoff 1990 ), and to the extent that we have the impression of directly perceiving forces in the visual field (Michotte 1954 ), these too might well be shared between the two 1S . representations Our discussionof interfacesin previous sectionsleadsus to expect someaspectsof each representationto be invisible to the other. What might someof theseaspectsbe? Section 1.4 noted that CS encodesthe token versustype distinction (a particular dog vs. the category of dogs), quantificational relations, and taxonomic relations (a bird is a kind of animal), but that theseare invisible to SR. On the other hand, SR encodes all the details of object shapes , for instance , the shapeof violin or a butter knife or a ' s ears. These German shepherd features do not lend themselves at all to the geometric sort of algebraic coding found in CS; they are absolutely natural to (at least the spirit of ) SR. In addition to generalmappings betweenconstituent types in CS and SR, individual matchings can be learned and stored. ( Learned and stored) lexical entries for physical object words can contain a spatial representation of the object in question, in addition to their phonological, syntactic, and conceptual structure. For instance , the entry for dog might look something like (4) . (4) Id ~gl + N , - V , + count , + sing, . . Individual , Type of Animal , Type of Carnivore Function: (often) Type of Pet SR: [3-D model wi motion affordances ] : Auditory [sound of barking] Phono: Syntax: CS:

In (4) the SR takes the place of what in many approaches (e.g., Rosch and Mervis " 1975 ; Putnam 1975 ) has been informally called an image of a prototypical instance " of the category. The difficulty with an image of a prototype is that it is computa: it does not meet the demands of object shape identification tionally nonefficacious laid out as criteria 1- 4 in the previous section . A more abstract spatial representation ,

Ray Jackendoff a. One way to view (4)

+ + CS Syntax IPhonology I+SA


LANGUAGE
? ? ?

b. Anotherway to view (4) LANGUAGE

+ Syntax IPhonology I+[~ !:~ ~ ~


.CONCEPr

Figure1.3 Two waysto viewtheintegrationof spatialstructures . into lexicalentries along the lines of a Marr 3-D model, meetsthesecriteria much better; it is therefore a more satisfactory candidate for encoding one' s knowledgeof what the object looks like. As suggestedby the inclusion of " auditory structure" in (4), a lexical entry should encode(pointers to ) other sensorycharacteristicsas well. The idea, then, is that the " meaning" of a word goes beyond the features and functions available in CS, in particular permit ting detailed shape information in a lexical SR. (A word must have a lexical CS; it may have an SR as well.) Such an approach might be seen as threatening the linguistic integrity of lexical items: as . But an alternative suggested by figure 1.3a, it breaks out of the purely linguistic system view of entries like (4) places them in a different light . Suppose one deletes the phonological and syntactic structures from (4) . What is left is the nonlinguistic " " knowledge one has of dogs- the concept of a dog, much of which could be shared by a nonlinguistic organism. Phonological and syntactic structures can then be viewed as further structures tacked onto to this knowledge to make it linguistically , as suggestedin figure 1.3b. With or without language , the mind has to expressible have a way to unify multimodal representationsand store them as units (that is, to establish long-term memory " binding " in the neuroscience sense ); (4) represents just such a unit . The structures that make this a " lexical item" rather than just a " concept " : the simply representan additional modality into which this concept extends . linguistic modality Having establishedgeneral properties of the CS- SR interface, we must raise the ? The question of exactly what information is on either side of it . How do we decide overall premise behind RepresentationalModularity , of course is that each module , is a specialist , and that each particular kind of information belongs in a particular module. For instance , details of shape are not duplicated in CS, and taxonomic relations are not duplicated in SR. For the general case , we can state a criterion of : all other if a certain kind of distinction is encodedin SR, , economy things being equal

The Architecture of the Linguistic -Spatial Interface

it should not also be encodedin CS, and vice versa . I take this maximal segregation to be the default assumption. Of course , all other things are not equal. The two modules must share enough structure that they can communicate with each other- for instance , they must share at least the notions mentioned at the beginning of this section. Thus we do not expect , as a baseline . , that the information encodedby CS and SR is entirely incommensurate Let us call this the criterion of interfacing. What evidencewould help decidewhether a certain kind of information is in CS as well as SR? One line of argument comesfrom interaction with syntax. Recall that CS is by hypothesis the form of central representation that most directly interacts with syntactic structure. Therefore, if a semanticdistinction is communicatedto syntax, so that it makes a syntactic difference , that distinction must be present in CS and not SR. that this criterion . just ( Note applies only to syntactic and not lexical differences As pointed out in section 1.4, dog and cat look exactly the sameto syntax.) Let us call this the criterion of grammatical effect. A secondline of argument concernsnonspatial domains of CS. As is well known : Talmy 1978 ; Jackendoff 1976 , 1983 (Gruber 1965 ; Lakoff and Johnson 1980 ; 1986 the semantics of Langacker ), many nonspatial conceptual domains show strong . Now if a particular semanticdistinction parallels to the semanticsof spatial concepts appearsin nonspatial domains as well as in the spatial domain, it cannot be encoded in SR alone, which by definition pertains only to spatial cognition . Rather, similarities between spatial and nonspatial domains must be captured in the algebraic structure of CS. I will call this the criterion of nonspatialabstraction.

-Mag Distinction 1.7 A SimpleCase : TheCount


A familiar example will make thesecriteria clearer. Consider the count-massdistinction . SR obviously must make a distinction betweensingle individuals (a cow), multiple individuals (a herd of cows), and substances (milk )- thesehave radically different and spatial behavior over time (Marr and Biederman appearances , of course, have little or nothing to say about what substances look like.) According to the criterion of economy, all else being equal, SR should be the only level that encodes these . differences But all elseis not equal. The count-massdistinction has repercussions in the marking of grammatical number and in the choice of possible determiners (count nouns usemany and few, massnouns usemuch and little , for example ) . Hence the criterion of grammatical effect suggests that the count-massdistinction is encodedin CS also. Furthermore, the count-massdistinction appearsin abstract domains. For example , * muchthreat but the threat is grammatically a count noun (many threatsf ), semantically

RayJackendoff
very similar adviceis a massnoun (much advicej* many advices ). Becausethe distinction ' t " look between threats and advice cannot be encoded spatially- it doesn like " - the anything only place to put it is in CS. That is, the criterion of nonspatial extensionapplies to this case . In addition , the count-mass distinction is closely interwoven with features of temporal event structure such as the event-processdistinction ( Verkuyl 1972 , 1993 ; ; Hinrichs 1985 Dowty 1979 ; Jackendoff 1991 ; Pustejovsky 1991 ) . To the extent that eventshave a spatial appearance , it is qualitatively different from that of objects. And distinctions of temporal event structure have a multitude of grammatical reflexes. Thus the criteria of nonspatial extension and grammatical effect both apply again to argue for the count-massdistinction being encodedin CS. A further piece of evidencecomes from lexical discrepanciesin the grammar of count and mass nouns. An example is the contrast between noodles (count) and spaghetti (mass )- nouns that pick out essentially the same sorts of entities in the world . A single one of these objects can be described as a singular noodle , but the massnoun forcesone to usethe phrasal form stick (or strand) of spaghetti . (In Italian , spaghettiis a plural count noun, and one can refer to a single spa ghetto.) Becausenoodlesand spaghetti pick out similar entities in the world , there is no reasonto believethat they havedifferent lexical SRs. Hencethere must be a mismatch somewherebetweenSR and syntax. A standard strategy (e.g., Bloom 1994 ) is to treat them as alike in CS as well and to localize the mismatch somewherein the CS- syntax interface. Alternatively , the mismatch might be betweenCS and SR. In this scenario , CS has the option of encoding a collection of smallish objects (or even largish objects such asfurniture ) as either an aggregateor a substance , then syntax follows suit by treating the concepts in question as grammatically count or mass . 16 , respectively Whichever solution is chosen , it is clear that SR and syntax alone cannot make sense of the discrepancy . Rather, CS is necessary as an intermediary betweenthem. 1.8 Axes and Framesof Reference We now turn to a more complex casewith a different outcome. Three subsetsof the vocabulary invoke the spatial axesof an object. I will call them collectively the " axial " vocabulary. I . The " axial parts" of an object- its top, bottom, front , back, sides , and ends behavegrammatically like parts of the object, but , unlike standard parts such as a handleor a leg, they have no distinctive shape . Rather, they are regions of the object (or its boundary) determined by their relation to the object' s axes . The up- down axis determines top and bottom , the front -back axis determines front and back, and

The Architecture of the Linguistic -Spatial Interface

a complex set of criteria distinguishing horizontal axes detennines sides and ends ; Landau and Jackendoff 1993 (Miller and Johnson-Laird 1976 ). " 2. The " dimensional adjectives , long, thick, and deep and their nomi high, wide nalizations height, width, length, thickness , and depth refer to dimensions of objects measuredalong principal , secondary and , , sometimeswith referenceto tertiary axes the horizontality or verticality of these axes (Bierwisch 1967 ; Bierwisch and Lang 1989 ). 3. Certain spatial prepositions, such as above , below , next to, in front of, behind , and out a detennined , left of, alongside right of, pick region by extending the reference ' . For instance , in front of X denotes a object s axes out into the surrounding space region of space in proximity to the projection of X' s front -back axis beyond the ; Landau boundary of X in the frontward direction (Miller and Johnson-Laird 1976 and Jackendoff 1993 Landau 8 this volume . contrast inside X makes ; , chapter , , ) By referenceonly to the region subtendedby X , not to any of its axes ; near X denotesa " region in proximity to X in any direction at all. Notice that many of the axial " are prepositions morphologically related to nouns that denote axial parts. It has been frequently noted (for instance , Miller and Johnson- Laird 1976 ; Olson and Bialystok 1983 ; and practically every chapter in this volume) that the axial vocabulary is always used in the context of an assumedframe of reference . Moreover, the choice of frame of referenceis often ambiguous; and becausethe frame determines the axesin tenDSof which the axial vocabulary receives its denotation, the axial vocabulary too is ambiguous. The literature usually invokes two frames of reference : an intrinsic or object -centeredframe. Actually the situation is centeredframe, and a deictic or observer more complex. Viewing a frame of referenceas a way of determining the axes of an object, it is possibleto distinguish at least eight different available frames of reference , which in (many of these appear as special casesin Miller and Johnson- Laird 1976 turn cites Bierwisch 1967 and Fillmore 1971 others . ; Teller 1969 ; , among ) A . Four intrinsic frames all make referenceto properties of the object: I . The geometric frame usesthe geometry of the object itself to determine the . For instance axes , the dimension of greatestextensioncan determine its length 1.4a . (figure ) Symmetrical geometry often implies a top- to -bottom axis dividing the symmetrical halvesand a side-to-side axis passingfrom one half to the other (figure 1.4b) . A specialcaseconcernsanimals, whosefront is intrinsically marked by the position of the eyes . 2. In the motion frame, thefront of a moving object is determined by the direction of motion . For instance , the front of an otherwise symmetrical double-ended tram is the end facing toward its current direction of motion (figure 1.4c) .

,
1 ~~ ~~ ~ ~~ ~ f":'
--

\t'(
.;.

RayJackendoff

w. WI..

f~
Two intrinsic framesdependon functional properties of the object. The canon ical orientation frame designatesas the top (or bottom ) of an object the part which in the object' s normal orientation is uppermost (or lowermost), even if it does not happen to be at the moment. For instance , the canonical orientation of the car in figure 1.4d has the wheelslowermost, so the part the wheels are attached to is the canonical bottom , even though it is pointing obliquely upward in this picture. Intrinsic parts of an object can also be picked out according to the canonical encounterframe. For instance , the part of a house where the public enters is

The Architecture of the Linguistic -Spatial Interface

l'r
:J1 1 .

Figure1.5 . frames reference Environmental functionally the front (figure 1.4e) . (Inside a building such as a theater, the front is the side that the public normally faces , so that the front from the inside front from the outside.) than the the wall of a different building may be the Four environmentalframes project axesonto object basedon properties of the environment: 1. The gravitational frame is determined by the direction of gravity , regardlessof , the hat in figure 1.5a the orientation of the object. In this frame, for instance is on top of the car. 2. The geographical frame is the horizontal counterpart of the gravitational frame, imposing axes on the object based on the cardinal directions north , south, east, and west, or a similar system(Levinson, chapter 4, this volume) . 3. The contextual frame is available when the object is viewed in relation to , another object, whose own axesare imposed on the first object. For instance has . The a which is drawn on a 1.5b page figure geometric pictures page figure an intrinsic side-to -side axis that determines its width , regardlessof orientation . The figure on the page inherits this axis, and therefore its width is measured in the samedirection. 4. The observerframe may be projected onto the object from a real or hypothetical es the front of the object as the side . This frame establish observer " We might call this the orientation 1.5c. in as , figure facing the observer " , such as Hausa, mirroring observer frame. Alternatively , in some languages

_ . _ ~f ~ , t "

(.

. - -

, fr8 'l\~

Ray Jackendoff

's the front of the object is the side facing the same way as the observer " front , as in figure 1.5d. We might call this the orientation -preservingobserver frame." It should be further noted that axesin the canonical orientation frame (figure 1.4d) are derived from gravitational axesin an imagined normal orientation of the object. Similarly , axes in the canonical encounter frame (figure 1.4e) are derived from a ' hypothetical observers position in the canonical encounter. So in fact only two of the eight frames, the geometric and motion frames, are entirely free of direct or indirect environmental influence. One of the reasons the axial vocabulary has attracted so much attention in the literature is its multiple ambiguity among frames of reference . In the precedingexamples alone, for instance , three different usesof front appear. Only the geographical frame (in English, at least ) has its own unambiguousvocabulary. Why should this be? And what does it tell us about the distribution of information betweenCS and SR? This will be the subject of the next section. Before going on, though, let us take a moment to look at how frames of reference are used in giving route directions (Levelt, chapter 3, this volume; Tversky, chapter 12 , thi ~ volume). Consider a simple case of Levelt' s diagrams such as figure 1.6. The route from circle I to circle 5 can be describedin two different ways: " " (5) a. Geographic frame: From I , go up/ forward to 2, right to 3, right to 4, down to 5. " b. " Observer frame: From I , go up/ forward to 2, right to 3, straight/ forward to 4, right to 5. The problem is highlighted by the step from 3 to 4, which is describedas " right " in " " (5a) and straight in ( 5b) . The proper way to think of this seems to be to keep track of hypothetical traveler' s " " orientation . In the geographic frame, the traveler maintains a constant orientation , ' so that up always means up on the page ; that is, the traveler s axes are set contextually by the page (frame B3) .
2 3 - - - - o- -

r 1

Figure1.6 ' Oneof Levelt s " maps ."

4 1 5

The Architecture of the Linguistic -Spatial Interface

" frame where the direction from 2 to 3 is observer The puzzling case is the ' ~ , " " " " and the samedirection from 3 to 4 is " , , straight or forward . Intuitively , right as Levelt and Tversky point out , one pictures oneself traveling through the diagram. " " From this the solution follows immediately: forward is determined by the ob' s last move that is server , , using the motion frame (A2 ) . The circles, which have no intrinsic orientation , play no role in determining the frame. If they are replaced by ' landmarks that do have intrinsic axes , a third possibility , as in Tversky s examples ' , that of setting the traveler s axescontextually by the landmarks (frame 83 emerges again) . And of course geographicalaxes(frame 8 I ) are available as well if the cardinal directions are known. "

of Axial Vocabulary 1.9 LexicalEncoding


Narasimhan ( 1993 ) reports an experiment that has revealing implications for the semantics " of the axial vocabulary. Subjectswere shown irregular shapes( Narasimhan " of the sort in figure 1.7, and asked to mark on them their length, width , figures ) height, or some combination of the three. Becauselength, width, and height depend ' . revealedsubjects on choice of axes , responses judgments about axis placement . Previous experimental This experiment is unusual in its use of irregular shapes research on axial vocabulary with which I am familiar (e.g., Bierwisch and Lang 1989 ; Levelt 1984 ) has dealt only with rectilinear figures or familiar objects, often ' only in rectilinear orientations. In Narasimhan s experiment, the subjects have to compute axesof novel shapeson-line, basedon visual input ; they cannot simply call up intrinsic axesstored in long-term memory as part of the canonical representation of a familiar object. ' . In But of course linguistic information is also involved in the subjects responses the choice of to mark influences that the is asked the dimension , subject particular . blases axis, as might be expectedfrom the work of Bierwisch and Lang ( 1989 ) Length the subject in favor of intrinsic geometric axes (longest dimension), while height -based contextual blases the subject toward environmental axes (gravitational or page ) . Thus, confronted with a shapesuch as figure 1.8a, whose longest dimension is oblique to the contextual vertical, subjects tended to mark its length as an oblique, and its height as an environmental vertical. Sometimessubjects even marked these ! axeson the very samefigure; they did not insist by any meanson orthogonal axes . The linguistic input , however, was not the only influence on the choice of axes Details in the shapeof the Narasimhan figure also exerted an influence. For example , figure 1.8b has a flattish surface near the (contextual) bottom . Some subjects (8% ) apparently interpreted this surfaceas a basethat had beenrotated from its canonical orientation ; they drew the height of the figure as an axis orthogonal to this base , that

Ray Jackendoff No base Flat base Tilted base

Up -down axis
Maximum (vertical )

Up -down axis

Vertical
, Maximum

Observer ' s line of sight

The Architecture of the Linguistic -Spatial Interface

is, as a " canonical vertical." Nothing in the linguistic input created this new possibility : it had to be computed on-line from the visual input . As a result of this extra possibility, the shapepresentedthree different choicesfor its axis system , as shown in the figure. We see , then, that linguistic and visual input interact intimately in determining ' in this experiment. However, the hypothesis of Representational subjects responses does not allow us to just leave it at that. We must also ask at what level Modularity of representation (i.e., in which module) this interaction takes place. The obvious choicesare CS and SR. The fact that the subjectsactually draw in axesshowsthat the computation of axes must involve SR. The angle and positioning of a drawn axis is continuously variable, in a way expected in the geometric SR but not expected in the algebraic feature complexes of CS. ' response How does the linguistic input get to SR so that it can influence the subjects ? That is, at what levels of representation do the words length, width, and height specify the axes and frames of referencethey can pick out? There are two possibilities: I . The CS hypothesis . The axes could be specified in the lexical entries of length, width, and height by features in CS such as [ ::f: maximal] , [ ::f: vertical], [ ::f: secondary ]; ' the frames of reference could be specified by CS features such as [ ::f: contextual] , in the CS- SR interface would then map features [ ::f: observer ] . General correspondences into the geometry of SR. According to this story, when subjectsjudge the axes of Narasimhan figures, the lexical items influence SR indirectly, via these general interpretations of the dimensional features of CS. (This is, I believe , the approach advocated by Bierwisch and Lang.) 2. The SR hypothesis . Alternatively, we know that lexical items may contain elements of SR such as the shapeof a dog. Hence it is possiblethat the lexical entries of length, width, and height also contain SR components that specify axesand frames of reference directly in the geometric format of SR. This would allow the axesand reference frames to be unspecified(or largely so) in the CS of thesewords. According to this , when subjectsjudge the axesof Narasimhan figures, the SR of the lexical hypothesis items interacts directly with SR from visual input . I propose that the SR hypothesis is closer to correct. The first argument comes ' from the criterion of economy. Marr ( 1982 , and Narasimhan s experiment ) demonstrates confirms, that people use SR to pick out axesand frames of referencein novel figures. In addition , people freely switch frames of referencein visuomotor tasks. For , we normally adopt an egocentric (or observer example ) frame for reaching but an environmental frame for navigating; in the latter , we seeourselvesmoving through a

Ray Jackendoff

17 stationary environment, not an environment rushing past. Theseare SR functions, not CS functions. Consequently , axes and frames of referencecannot be eliminated that a CS feature SR. This means from systemfor thesedistinctions at best duplicates information in SR it cannot take the place of information in SR. Next consider the criterion of grammatical effect. If axesand frames of reference to encodethem in CS. But can be shown to have grammatical effects , it is necessary seem to be few grammatical there in this domain, unlike the count-mass system , of the effects . The only thing specialabout the syntax English axial vocabulary is that dimensional adjectivesand axial prepositions can be precededby measurephrases , as in three incheslong, two miles wide (with dimensional adjectives ), andfour feet behind the wall, sevenblocks up the street (with axial prepositions) . Other than dimensional , the only English adjective that can occur with a measurephrase is old; adjectives such pragmatically plausible casesas * eighty degreeshot and * twelvepounds heavy are ungrammatical. Similarly , many prepositions do not occur with measurephrases (* ten inchesnear the box); and those that do are for the most part axial (though away, 18 as in a mile awayfrom the house , is not) . Thus whether a word pertains to an axis does seemto make a grammatical difference . No grammatical effectsseemto depend on . But that is about as far as it goes which axis a word refers to , much lesswhich frame of referencethe axis is computed in , at least in English. 19Thus the criterion of grammatical effect dictates at most that CS needsonly a feature that distinguishes axesof objects from other sorts of object parts; the axial vocabulary will contain this feature. Distinguishing axes from each on grammatical other and frames of referencefrom each other appearsunnecessary . grounds , consider the use of axis systems Turning to the criterion of nonspatial extension and frames of referencein nonspatial domains. It is well known that analoguesof spatial axes occur in other semantic fields, and that axial vocabulary generalizes to these domains (Gruber 1965 ; Langacker 1986 ; ; Talmy 1978 ; Jackendoff 1976 I of are Lakoff 1987 only one-dimensional, ) . But all other axis systems know for example , weights, ranks, and comparative adjectives , numbers, temperatures . A cognitive system with more than one dimension etc. more less ) ( / beautiful/salty/exciting/ is the familiar three-dimensional color space , but languagedoes not express of axial in sort differences color using any vocabulary. Kinship systemsmight be another multidimensional case , and again the axial vocabulary is not employed. In English, when a nonspatial axis is invoked, the axis is almost always up/ down , my mood is up, etc.) . , lower rank, of higher beauty, lower temperature (higher number Is there a referenceframe? One' s first impulse is to say that the referenceframe is gravitational - perhaps becausewe speak of the temperature rising and falling and of rising in the ranks of the army, and becauserise and fall in the spatial domain

-Spatial TheArchitecture of theLinguistic Interface


pertain most specifically to the gravitational frame. But on secondthought, we really wouldn ' t know how to distinguish among reference frames in these spaces . What would it mean to distinguish an intrinsic upward from a gravitational upward, for ? example About the only exception to the use of the vertical axis in nonspatial domains is time, a one-dimensional systemthat goesfront to back.2O Time is also exceptional in that it doesdisplay referenceframe distinctions. For instance , one speaksof the times " " " " beforenow, where beforemeans prior to , as though the observer(or the front of an event) is facing the past. But one also speaksof the hard times before us, where " " before means subsequentto , as though the observer is facing the future. A notion of frame of referencealso appears in social cognition , where we speak of adopting another' s point of view in evaluating their knowledge or attitudes. But compared to spatial frames of reference , this notion is quite limited : it is analogous to adopting an observer referenceframe for a different (real or hypothetical) observer ; there is no parallel to any of the other seven varieties of reference frames. Moreover, in the social domain there is no notion of axis that is built from these frames of reference . Thus again an apparent parallel proves to be relatively . impoverished In short, very little of the organization of spatial axes and frames of referenceis recruited for nonspatial concepts . Hence the criterion of nonspatial extension also gives us scant reasonto encodein CS all the spatial distinctions among three-dimensional axesand frames of reference . All we need for most purposesis the distinction betweenthe vertical and other axes , plus some special machinery for time and perhaps for social point of view. Certainly nothing outside the spatial domain calls for the richnessof detail neededfor the spatial axial vocabulary. Our tentative conclusion is that most of this detail is encoded only in the SR component of the axial vocabulary, not in the CS component; it thus parallels such lexical SR componentsas the shapeof a dog. Let me call this the " Mostly SR hypothesis ." A skeptic committed to the CS hypothesis might raise a " functional " argument against this conclusion. Perhapsmultiple axes and frames of referenceare available in CS, but we do not recruit them for nonspatial conceptsbecausewe have no need for them in our nonspatial thought . Or perhapsthe nature of the real world does not lend itself to such thinking outside of the spatial domain, so such conceptscannot be usedsensibly . If one insists on a " functional " view, I would urge quite a different argument. It would often be extremely useful for us to be able to think in terms of detailed variation of two or three nonspatial variables, say the relation of income to educational level to age , but in fact we find it very difficult . For a more ecologically plausible case , why do we inevitably reduce social status to a linear ranking, when it so clearly

Ray Jackendoff

involves many interacting factors? The best way we have of thinking multidimensionally is to translate the variablesin question into a Cartesian graph, so that we can multidimensional spatial intuitions to the variation in question- we can our apply . This suggests that CS is actually relatively poor seeit as a path or a region in space in its ability to encodemultidimensional variation ; we have to turn to SR to help us . encodeit . This is more or lesswhat would be predicted by the Mostly SR hypothesis That is, the " functional " argument can be turned around and used as evidencefor the Mostly SR hypothesis . The caseof axesand frames of referencethus comesout differently from the case of the count-massdistinction . This time we conclude that most of the relevant distinctions are not encodedin CS, but only in SR, one level further removed from syntactic structure. This conclusion is tentative in part becauseof the small amount of linguistic evidence adduced for it thus far - one would certainly want to check the data out we cross linguistically before making a stronger claim. But it is also tentative because do not have enough formal theory of SR to know how it encodesaxesand frames of reference . It might turn out , for instance , that the proper way to encodethe relevant distinctions is in terms of a set of discrete(or digital ) annotations to the geometry of SR. In such a case , it would be hard to distinguish an SR encoding of thesedistinctions from a CS encoding. But in the absenceof a serioustheory of SR, it is hard to . know how to continue this line of research

1.10 FinalThoughts
To sort out empirical issuesin the relation of languageto spatial cognition , it is useful to think in terms of Representational Modularity . This forces us to distinguish the levels of representationinvolved in language , abstract conceptual thought, and spatial to take the issue of how theselevels communicate with and seriously cognition , one another. In looking at any particular phenomenon within this framework , the crucial question has proved to be at which level or levels of representationit is to be . We have examinedcases where the choice betweenCS and SR comesout in encoded different ways. This shows that the issueis not a simple prejudged matter ; it must be . evaluated for each case For the moment, however, we are at the mercy of the limitations of theory. Compared to the richnessof phonological and syntactic theory, the theory of CS is in its , is hardly infancy; and SR, other than the small bit of work by Marr and Biederman even in gestation. This makes it difficult to decide among (or even to formulate) competing hypothesesin any more than sketchy fashion. It is hoped that the present volume will spur theorists to remedy the situation.

~ tic-Spatial The Architecture of the I ,ingul Interface


Acknowledgments I am grateful to Barbara Landau, Manfred Bierwisch, Paul Bloom , Lynn Nadel, Bhuvana Narasimhan, and Emile van der Zee for extensivediscussion , in person and in correspondence , came from participants in surrounding the ideasin this chapter. Further important suggestions the Conferenceon Spaceand Language sponsoredby the Cognitive Anthropology Research Group at the Max Planck Institute for Psycholinguisticsin Nijmegen in December1993and of course from the participants in the Arizona workshop responsiblefor the presentvolume. This researchwas supported in part by National ScienceFoundation grant IRI -92- 13849 to Brandeis University, by a Keck Foundation grant to the Brandeis University Center for , and by a fellowship to the author from the John Simon Guggenheim Complex Systems Foundation. . Notes I . This is an oversimplification, becauseof the existenceof languagesthat make use of the visual/gestural modalities. SeeEmmorey (chapter 5, this volume) . 2. Various colleagueshave offered interpretations of Fodor in which some further vaguely es the conversion. I do not find any support for theseinterpretations specifiedprocessaccomplish in the text. 3. Of course , Fodorian modularity can also solve the problem of communication among modules by adopting the idea of interface modules. However, becauseinterface modules as conceived here are too small to be Fodorian modules (they are not input -output faculties), there are two possibilities: either ( I ) the scaleof modularity has to be reducedfrom faculties to , along lines proposed here; or else(2) interfacesare simply an integrated part representations of larger modules and need not themselvesbe modular. I take the choice betweenthese two , but also in part an empirical one. possibilities to reflect in part a merely rhetorical difference 4. Caveatsare necessary concerning nonconcatenativemorphology such as reduplication and Semitic inflection , where the relation betweenlinear order in phonology and syntax is unclear, to say the least. 5. To be sure, syntactic featuresare frequently realized phonologically as affixeswith segmental content; but the phonology itself has no knowledge of what syntactic features theseaffixes . express 6. Fodor ' s claims about informational encapsulation are largely built around evidence that semantic es of lexical retrieval /pragmatic information does not immediately affect the process and syntactic parsing in speechperception. This evidenceis also consistent with Representational Modularity . The first pass of lexical retrieval has to be part of the mapping from ' auditory signal to phonological structure, so that word boundaries can be imposed; Fodor s discussionshows that this first pass usesno semantic information . The first pass of syntactic parsing has to be part of the mapping from phonological to syntactic structure, so that candidate semantic interpretations can subsequentlybe formulated and tested ; this first pass uses no semantic information either. See Jackendoff 1987 , chapters 6 and 12, for more detailed discussion .

Ray Jackendoff 7. It is surely significant that syntax sharesembeddingwith CS and linear order with phonol ogy. It is as though syntactic structure is a way of converting embedding structure into linear order, so that structured meaningscan be expressed as a linear speechstream. 8. As a corollary , SR must support the generation of mentally rotated objects, whoseperspective with respectto the viewer changesduring rotation . This is particularly crucial in rotation on an axis parallel to the picture plane becausedifferent parts of the object are visible at different times during rotation - a fact noted by Kosslyn ( 1980 ). ' " 9. Somecolleagues have objectedto Marr s characterizingthe 3-D sketchas " object-centered , arguing that objects are always seenfrom some point of view or other- at the very least the ' s. However I " " observer , interpret object-centered as meaning that the encoding of the object is independent of point of view. This neutrality permits the appearanceof the object to be computed as necessaryto fit the object into the visual sceneas a whole, viewed from any arbitrary vantage point . Marr , who is not concerned with spatial layout but only with identifying the object, does not deal with this further step of reinjecting the object into the scene . But I seesuch a step as altogether within the spirit of his approach. 10. A different sort of example , offered by Christopher Habel at the Nijmegen spaceconference " " (seeacknowledgments ) : the image schema for along, as in the road is along the river, must include the possibility of the road being on either side of the river. An imagistic representation must representthe road being specifically on one side or the other. II . It is unclear to me at the moment what relationship this notion of image schemabears to that of Mandler ( 1992and chapter 9, this volume), although there is certainly a family resemblance . Mandler ' s formulation derivesfrom work such as that of Lakoff ( 1987 ) and Langacker ( 1986 ), in which the notion of level of representation is not well developed , and in which no explicit connection is made to researchin visual perception. I leaveopen for future researchthe question of whether the presentconception can help sharpen the issueswith which Mandler is . concerned 12. This section is derived in part from the discussionin Jackendoff 1987 , chapter 10. 13. Although fundamental, such a type is not necessarilyprimitive . Jackendoff 1991decomposes the notion of object into the more primitive feature complex [material, + bounded, - inherent structure] . The feature [material] is shared by substances and aggregrates ; it distin them all from situations events and states times and various sorts of abstract , , ( ), spaces guishes entities. The feature [ + bounded] distinguishes objects from substances , and also closedevents es. The feature [ - inherent structure] distinguishes objects (or accomplishments ) from process from groups of individuals , but also substances from aggregates and homogeneousprocess es from repeatedevents . 14. On the other hand, it is not so obvious that places and paths are encoded in imagistic representation becausewe do not literally see them except when dotted lines are drawn in cartoons. This may be another part of SR that is invisible to imagistic representation . That is, placesand paths as independententities may be a higher-level cognitive (nonperceptual ) aspect of spatial understanding, as also argued by Talmy (chapter 6, this volume) . 15. Paul Bloom has asked ( personalcommunication) why I would considerforce but not , say " " anger to be encoded in SR becausewe have the impression of directly perceiving anger as

~ tic-Spatial The Architecture of the IJmgul Interface


- direction of force well. The difference is that physical force has clear geometric components to encodeother spatial and often contact betweenobjects- which are independentlynecessary entities suchas trajectories and orientations. Thus force seems a natural extensionof the family of spatial concepts . By contrast, anger has no such geometrical characteristics ; its parameters belong to the domain of emotions and interpersonal relations. Extending SR to anger, therefore . , would not yield any generalizationsin terms of sharedcomponents 16. This leavesopen the possibility of CS- syntax discrepanciesin the more grammatically . I leavethe issueopen. problematic caseslike scissorsand trousers 17. For a recent discussion of the psychophysics and neuropsychology of the distinction . between environmental motion and self-motion , see Wertheim 1994 and its commentaries to Wertheim, however, does not appear to addressthe issue crucial the of , , present enterprise how this distinction is encoded so that further inferencescan be drawn from it - namely, the of distinguishing referenceframes. cognitive consequences 18. Measure phrasesalso occur in English adjective phrasesas specifiersof the comparatives moref-er than and as . . . as, for instance ten poundsheavier ( than X ) , threefeet shorter ( than X ) , six timesmore beautiful ( than X ) ,fifty timesasfunny ( as X ) . Here they are licensednot by the adjective itself, but by the comparative morpheme. 19. Bickel 1994a , however, points out that the NepaleselanguageBelhare makes distinctions of grammatical casebasedon frame of reference . In a " personmorphic" frame for right and is the visual field divided into two halves , with the division line running through the left , observerand the referenceobject; this frame requires the genitive casefor the referenceobject. In a " physiomorphic" frame for right and left, the referenceobject projects four quadrants whosecentersare focal front , back, left , and right; this frame requires the ablative casefor the referenceobject. I leave it for future researchto ascertain how widespreadsuch grammatical distinctions are and to what extent they might require a weakeningof my hypothesis . 20. A number of people have pointed another nonvertical axis system , the political spectrum, which goes from right to left. According to the description of Bickel 1994b , the Nepalese languageBelhare is a counterexampleto the generalization about time going front to back: a transverseaxis is used for measuring time, and an up- down axis is used for the the conception of time as an opposition of past and future. References

Bickel in spatialdeixisand the typologyof reference . frames , B. ( 1994a ). Mappingoperations , CognitiveAnthropologyResearch Group, Max PlanckInstitute for Working paperno. 31 . , Nijmegen Psycholinguistics Bickel : Where to orient , and culture , B. ( I 994b , cognition ). Spatial operationson deixis oneselfin Belhare (revisedversion , Cognitive Anthropology ). Unpublishedmanuscript Research . , Nijmegen Group, Max PlanckInstitutefor Psycholinguistics Biederman : A theoryof humanimageunderstanding . , I. ( 1987 ). Recognition by- components - 147 . Review , 94(2), 115 Psychological Bierwisch . Foundations , M. ( 1967 ). Some semanticuniversalsof German adjectivals of . , 3, 1- 36 Language

T RayJackendot
Bierwisch, M . ( 1986 . In F. Klix and ) . On the nature of semantic fonn in natural language H. Hagendorf (Eds.), Human memoryand cognitivecapabilities : Mechanisms andperformances , 765- 784. Amsterdam: Elsevier / North-Holland . Bierwisch, M ., and Lang, E. (Eds.) ( 1989 . Berlin: Springer. ) . Dimensionaladjectives Bloom, P. ( 1994 : The role of syntax-semanticsmappings in the acquisition of ) . Possiblenames nominals. Lingua, 92, 297- 329. Culicover, P. ( 1972 : On the derivation of sentenceswith systematically ) . OM -sentences - 236. , 8, 199 unspecifiableinterpretations. Foundationsof Language ) . Word meaningand Montague grammar. Dordrecht: Reidel. Dowty , D . ( 1979 Farah, M ., Hammond , K ., Levine, D ., and Calvanio, R. ( 1988 ) . Visual and spatial mental . Cognitive Psychology imagery: Dissociable systemsof representation , 20, 439- 462. Fillmore , C. ( 1971 ) Santa Cruz lectureson deixis. Bloomington : Indiana University Linguistics Club. Fodor , J. ( 1975 . ) The languageof thought. Cambridge, MA : Harvard University Press Fodor , J. ( 1983 . ) Modularity of mind. Cambridge, MA : MIT Press Gruber , J. ( 1965 Institute of Technology ). Studiesin lexical relations. PhiD . diss., Massachusetts . Reprinted in Gruber , Lexical structures in syntax and semantics , Amsterdam: North Holland , 1976 . Hinrichs , E. ( 1985 ) . A compositional semanticsfor Aktionsarten and NP referencein English. Ph.D . diss., Ohio State University . Jackendoff, Ray ( 1976 . Linguistic Inquiry, 7, ) . Toward an explanatory semanticrepresentation 89- 150 . Jackendoff, R. ( 1983 . ). Semanticsand cognition. Cambridge, MA : MIT Press Jackendoff, R. ( 1987 and the computationalmind. Cambridge, MA : MIT Press . ) . Consciousness Jackendoff, R. ( 1990 . Cambridge, MA : MIT Press . ). Semanticstructures Jackendoff, R. ( 1991 ). Parts and boundaries. Cognition, 41, 9 45. Jackendoff, R. ( 1992 . ) . Languages of the mind. Cambridge, MA : MIT Press Jackendoff, R. (forthcoming). The architecture of the language faculty . Cambridge, MA : MIT . Press Jeanne rod , M . ( 1994 ) . The representing brain: Neural correlates of motor intention and - 201. . Behavioral and Brain Sciences , 17, 187 imagery . ) . Image and mind. Cambridge, MA : Harvard University Press Kosslyn, S. ( 1980 Lakoff , G . ( 1987 . ,fire , and dangerous ) . Women things. Chicago: University of Chicago Press Lakoff , G., and Johnson . , M . ( 1980 ). Metaphorswelive by. Chicago: University of ChicagoPress " " " " Landau, B., and Jackendoff, R. ( 1993 ) . What and where in spatial languageand spatial , 16, 217- 238. cognition . Behavioraland Brain Sciences

The Architecture of the Linguistic - Spatial Interface

. Vol. 1. Stanford , CA: Stanford , R. ( 1986 grammar of cognitive ). Foundations Langacker . UniversityPress . .) ( 1992 Lehrer , Hinsdale ,fields, andcontrasts , NJ: Erlbaum , A., and Kittay, E. (Eds ). Frames . In A. van Doom, limitations in talking about space Levelt , W. ( 1984 ). Someperceptual . Utrecht : Coronet .), Limits in perception W. van de Grind, and J. Koenderink (Eds . Books . In Papers Levin - fourth from the twenty , T. ( 1988 , B., and Rapoport ). Lexicalsubordination . . : 275 289 the , Universityof Chicago Chicago Linguistics Society Chicago regional meeting of . of Linguistics Department . Psychological Review : 2. Conceptual Mandler , 99, primitives , J. ( 1992 ). How to build a baby - 604 . 587 . : Freeman . SanFrancisco Marr, D. ( 1982 ). Vision Universitaires . 2d ed. Louvain : Publications dela causalite Michotte , A. ( 1954 ). La perception . de Louvain -Laird, P. ( 1976 . Cambridge andperception Miner, G., andJohnson , MA: Harvard ). Language . UniversityPress . in the useof length Narasimhan , width , and height , B. ( 1993 ). Spatialframesof reference . , BostonUniversity manuscript Unpublished . Oxford: Oxford as a cognitive O' Keefe map , L. ( 1978 , J., and Nadel ). The hippo campus . UniversityPress . . Hinsdale Olson , NJ: Erlbaum , E. ( 1983 , D., and Bialystok ). Spatialcognition . es . New York: Holt, Rinehart Paivio , and Winston , A. ( 1971 process ). Imageryand verbal . Erlbaum 1979 Hinsdale NJ: , , , Reprint . In E. Reuland andW. Abraham andsemantic structures Partee , B. ( 1993 properties ). Semantic . Dordrecht : structure . Vol. 2, Lexicaland conceptual .), Knowledge and Language , 7- 30 (Eds Kluwer. . Cambridge structure : Theacquisition andcognition Pinker , of argument , S. ( 1989 ). Learnability . MA: MIT Press . Cognition , 41, 47- 81. , J. ( 1991 ). The syntaxof eventstructure Pustejovsky . . Cambridge lexicon , MA: MIT Press , J. ( 1995 ). Thegenerative Pustejovsky " " . In K. Gunderson of meaning Putnam , mind , and (Ed.), Language , H. ( 1975 ). Themeaning - 193 . Press . Minneapolis : Universityof Minnesota , 131 knowledge in the internal structureof : Studies Rosch , C. ( 1975 , E., and Mervis ). Family resemblances - 605 . . Cognitive , 7, 573 Psychology categories . In D. Waltz (Ed.), : A synopsis , L. ( 1978 ). The relation of grammarto cognition Talrny for Computing in naturallanguage issues Theoretical , vol. 2, NewYork: Association processing . Machinery

' Ray Jackendoft Talmy, L . ( 1980 ) . Lexicalization patterns: Semantic structure in lexical forms. In T . Shopen (Ed.), Languagetypology and syntactic description , vol. 3. New York : Cambridge University Press .

structures . In H. Pick and L. Acredolo(Eds .), Spatial , L. ( 1983 ). How language Talmy space orientation : Theory . NewYork: Plenum Press . , research , andapplication in language and thought . In Papers , L. ( 1985 Talmy ). Forcedynamics from the Twenty -first . Chicago : Universityof Chicago . Department Regional Meetingof theChicago LinguisticSociety of Linguistics . Also in Cognitive Science . , 12( 1988 ), 49- 100 ' s work on German Teller and extension of Manfred Bierwisch , P. ( 1969 ). Somediscussion . Foundations . , 5, 185 217 adjectivals of Language . In D. Ingle , L., andMishkin, M. ( 1982 Ungerleider ) Two corticalvisualsystems , M. Goodale , and R. Mansfield .), Analysis behavior . Cambridge MA: MIT Press . (Eds of visual natureof theaspects . Dordrecht : Reidel . , H. ( 1972 Verkuyl ). On thecompositional . Cambridge : Cambridge . , H. ( 1993 Verkuyl ). A theoryof aspectuality UniversityPress Wertheim , A. ( 1994 ). Motion perceptionduring selfmotion: The direct versusinferential - 311 . Behavioral revisited andBrainSciences . , 17 , 293 controversy

Chapter How

2 Space Gets into Language ?

Much

Manfred Bierwisch

2.1

Introduction

We can talk about spatial aspectsof our environment with any degreeof precision we - unlike pictures, maps, blueprints, and the want, even though linguistic expressions like - do not exhibit spatial structure in any relevant way. This apparent paradox is . For the simply due to the symbolic, rather than iconic, character of natural language same reason , we can talk about color , temperature, kinship , and all the rest, even though linguistic utterances do not exhibit color , temperature, kinship, and so on. raisesthe by no meanstrivial question where and The apparent paradox nevertheless . The present chapter will be concerned with certain how space gets into language aspectsof this problem, pursuing the following question: Which components of natural languageaccommodatespatial information , and how? Looking first at syntax, we observethat completely identical structurescan express : both spatial and clearly nonspatial situations, as in ( la ) and ( lb ), respectively ' ( I ) a. We entered Saint Peter s Cathedral. b. We admired Saint Peter' s Cathedral. The contrast obviously dependson the meaning of enter versusadmire. Comparing ( la ) with (2), we notice, furthermore , that identical or at least very similar spatial eventscan be expressed by meansof rather different syntactic constructions: ' (2) We went into Saint Peter s Cathedral. The conclusion that syntactic elementsand relations do not accommodatespatial information seemsto be confronted with certain objections, though. Thus the PP at the end has a temporal meaning in (3a) but a spatial one in (3b), depending on its syntactic position :

Manfred Bierwisch

the letter. , shesigned (3) a. At the end


b. She signedthe letter at the end. One cannot, however, assignthe contrast betweenspatial and nonspatial interpretation to the position as such, as is evident from pairs like those in (4) : (4) a. With this intention , she signedthe letter. b. Shesignedthe letter with this intention. What we observein (3) and (4) is rather the effect the different syntactic structure has on the compositional semanticsof adjuncts (the details of which are still not really understood), determining different interpretations for the PP in (3) . Pending further clarification , we will nevertheless conclude that phrasestructure does not reflect spatial information per se. Another problem shows up in caseslike ( 5), differing with respectto place and goal: . (5) a. Er schwammunter DernSteg (He swam under the bridge.) location b. Er schwammunter den Steg . He swam under the . ( bridge ) directional It is, of course, not the contrast betweenIml and Inl , but rather that betweendative and accusative that is relevant here. This appears to be a matter of the syntactic , however, the crucial distinction can be reducedto a component. In the presentcase systematicdifferencebetweena locative and a directional reading of the preposition unter, each associatedwith a specificcaserequirement (seeBierwisch 1988fordiscussion ) in languageswith rich morphology . I will take up this issue in section 2.7. Whereascasecan thus be shown to be related to spaceonly as an indirect effect, this does not hold for the so-called notional or content cases . In any case , syntax and morphology as such do not reflect spatial information . Hencethe main area to be explored with respectto our central question is thesemantic component, in particular the field of lexical semantics . As already mentioned with to I it is the word meaning of enter that carries the spatial aspect . Similarly , respect ( ), the contrast betweenplace and goal in (5) is ultimately a matter of the two different readingsof unter. Further illustrations could be multiplied at will , including all major lexical categories . This does not mean, however, that there is a simple and clear distinction between spatial and nonspatial vocabulary. As a matter of fact, most words that undoubtedly have a spatial interpretation may alternatively carry a nonspatial reading under certain conditions. Consider (6) as a casein point : (6) He entered the church.

? How Much SpaceGets into Language

Besidesthe spatial interpretation corresponding to that of ( Ia ), (6) can also have an interpretation under which it means he becamea priest, where church refers to an institution and enter denotesa changein social relations. The verb to enter thus has a spatial or nonspatial interpretation depending on the reading of the object it combines " " with . This is an instanceof what Pustejovsky( 1991 ) calls co- compositionality, that is, a compositional structure where one constituent determinesan interpretation of the other that is not fixed outside the combinatorial process . In other words, we must not only account for the spatial information that enter projects in cases like ( Ia ) and one reading of (6), but also for the switch to the nonspatial interpretation in the second reading of (6) . To conclude these preliminary considerations , in order to answer our central question, we have to investigatehow lexical items relate to space and eventually project theserelations by meansof compositional principles.

2.2 LexicalSemantics andConceptual Structure


Let me begin by placing lexical and compositional semanticsin the more general perspectiveof linguistic knowledge, that is, the internal or I -languagein the senseof ), which underlies the properties of external or E-languageof setsof Chomsky ( 1986 . Following the terminology of Chomsky ( 1993 ), I -languageis to linguistic expression be construed as a computational systemthat detenninesa systematiccorrespondence betweentwo different domains of mental organization: (7) A -P +- - I -language- - + C-I A -P comprises the systemsof articulation and perception, and C-I , the systemsby which experienceis conceptually organized and intentionally related to the external and internal environment. I -language provides two representational systems , which " " " " Chomsky calls phonetic fonn (PF) and logical form (LF ), that constitute the interfaceswith the respectiveextralinguistic domains. Because there is apparently no direct relation that connects spatial infonnation to sound structure, bypassing the , I will have correspondenceestablishedby the computational system of I -language nothing to say about PF, except where it will be useful to compare how it relates to A -P with the far more complex conceptual phenomenathat concern us. Given PF and LF as interface levels , detennined by I -languageand interpreted in terms of APand C-I , respectively the , correspondencebetweenthem is established . With this overall orientation by the syntactic and morphological operations of I -language in mind , one might consider the (speciesspecific ) languagecapacity as emerging from brain structures that allow for the discrete , recursive mapping between two representational systemsof different structure and origin . Assuming universal grammar (UG ) to be the formal characterization of this capacity, we arrive at the

Manfred Bierwisch

schema from theconditions , whereI -language followinggeneral emerges specified by UG throughthe interactionwith the systems of APand C-I:
( 8) A - P +- - +- lPF + - - SYNTAX - - + LFJ+- - +- C - I y

I -language

~ va
This schemais meant as a rough orientation , leaving crucial questionsto be clarified. Before I turn to details of the relation between I -language and C-I , two general remarks about UG and the organization of I -languagemust be made. First , for each of the major components of I -language , universal grammar (UG ) must provide two specifications : I . A way to recruit the primitive elementsby which representationsand operations of the component are specified ; and 2. A general format of the type of representationsand operations of the component. The most parsimonious assumption is that specification 2 is fixed across languages , from the conditions of the as such. In other words , emerging language capacity the types of representation and the operations available for I -languageare given in advance .! As to specification I , three types of primes are to be distinguished: I . Primes that are recruited from and interpreted by A -P; 2. Primes that are recruited from and interpreted by C-I ; and 3. Primes that function within I -languagewithout external interpretation . It is usually assumedthat type I , the primes of PF, namely, phonetic features and , are basedon universally fixed options in UG . Alternatively , one prosodic categories might think of them as being recruited from the auditory input and articulatory patterns by means of certain constraints within UG , which provides not the repertoire of these features but rather some sort of recipe to make them up . This view would be mandatory if in fact UG were not restricted to acoustic signalsbut allowed also for systemslike sign language . Although the details of this issuego beyond the , the notion of conditions or constraints to construct scopeof the present discussion primes of I -language seemsto be indispensableif we addresstype 2, the primes in terms of which I -languageinterfaceswith C-I , and if semanticrepresentationsare to go beyond a highly restricted core of lexical items. I will return to theseissuesbelow. As for type 3, which must comprise the featuresspecifying syntactic and morphologi cal categories , thesemust be determined directly by the conditions on syntactic and morphological representationsand operations falling under type 2, varying only to

How Much SpaceGets into Language ?

. This the extent to which they can be affected by intrusion from the interface levels in fact be the case for which conditions syntactic might morphological categoriesby take up conceptual content, for example , in number, person, and so forth . Second , the computation determined by I -languagedoes not in general proceedin terms of primitive elementsbut to a large extent in terms of chunks of them fixed in lexical items. Lexical items are idiosyncratic configurations, varying from languageto , which must be learned on the basis of individual experience , but which are language determined by VG with respectto their general format in accordancewith specifications 1 and 2. I will call the set of lexical items, together with the general conditions " LS of I . LS is not a to which they must conform , the " lexical system ( ) language of I , alongside phonology, syntax, morphology , and separatecomponent language semantics ; rather, it cuts acrossall of them, combining information of all components of I -language . The general format that VG determinesfor lexical items is (9) : (9) [PF (le), GF (le), LF (le)], where PF (le) determinesa representationof Ie at PF; LF (le) consistsof primes of LF specifiedby Ie; GF (le) representssyntactic and morphological properties of Ie. I will have more to say about the organization of lexical entries at the end of section 2.2. (9) also indicates the basic format of linguistic expressionsin general , if we assumethat PF (le), LF (le), and GF (le) can representinformation of any complexity in accordancewith the two requirementsnoted above. With regard to the crucial question how C- I relates to I -language , there is a remarkable lack of agreementamong otherwise closely related approaches. According to the conceptualframework of Chomsky ( 1981 , 1993 , 1986 ), LF is a level of syntactic in whose status lies its the interface with conceptual representation particular forming LF in structure. (In Chomsky 1993 is fact the level of , only syntactic representation to which independent , systematic conditions apply .) The basic elementsof LF are lexical items, or rather their semantic component, and the syntactic features associated with them. In other words, the primes of LF , which according to type 2 above connect I -language to C-I , are to be identified with word meanings , or more LF technically, with the part of lexical items, including complex items originating es of " sublexical syntax" as from incorporation , head movement, or other process discussed , whatever internal , for example , by Hale and Keyser ( 1993 ) . In any case structure should be assignedto the semanticsof lexical items is essentiallya matter of . C-I , not structurally reflectedin I -language In contrast to this view, Jackendoff ( 1983and subsequentwork ), following Katz ( 1972 ) and others, assignslexical items a rich and systematicinternal structure, which is claimed to be linguistically relevant. I will adopt this view, arguing that there are

structural phenomenadirectly involved in I -languagethat turn on the internal structure " of lexical items. I call the basic elementsof this structure " semantic primes, assuming theseare the elementsidentified in type 2 that connect I -language to C-I . Supposenow that we call the representationalsystembasedon semantic primes the " semantic form " SF of I - parallel to PF, which is based on phonetic ( ) language ), and hence primes. We will consequentlyreplace schema(9) of lexical items by ( 10 the overall schema(8) by ( II ) : ( 10 ) [PF (/e), GF (/e), SF(/e)] with PF (/e) a configuration of PF, SF(/e) a configuration of SF, and GF (/e) a specification of morphological and syntactic properties
( 11) A - P +- - +- lPF + - - SYNTAX - - + SF) +- - +- C - I y I - language

~ va
at LF according The systemof SYNTAX is now to include the information represented 2 to (8) . Before I take up some controversial issuesthat are related to these assumptions , I will briefly illustrate their empirical motivation . The basic idea behind the organization of knowledge suggestedin ( II ) is that I -language needsto be distinguished from the various mental systemsthat bear on . More specifically A -P and C-I , respectively , the conceptual interpretation c of a linguistic expressione is determined by the semantic form of e and the conceptual knowledge underlying C- I . As this point is crucial with respect to our central question . What I want to show is , I will clarify the problem by meansof someexamples twofold . On the one hand, the interpretation of an expressione is detennined by its semanticform SF(e), which is basedon the semanticform of lexical items exhibiting a systematic , linguistically relevant internal structure. On the other hand, the conceptual interpretation of e, which among other things must fix the truth and satisfaction conditions, depends in crucial respectson commonsensebeliefs, world knowledge, -independentand must be assignedto C-I . and situational aspects , which are language in ( 12 To begin with the secondpoint , compare the sentences ): ( 12 ) a. He left the institute an hour ago. b. He left the institute a year ago. In ( 12a ) leave is (most likely) interpreted as a physical movement and institute as while the time adverbial a year ago of ( 12b ) turns leave into a change in place, affiliation and institute into a social institution . The two interpretations of leave the

? into Language Gets HowMuchSpace


institute are casesofco - compositionality as already illustrated by sentence (6) above. and where For extensive discussion of these phenomena , encyclopedic linguistic ) . The most , Bierwisch ( 1983 , for example ) and Dolling ( 1995 knowledge interact, see ) is, however, that the choice between the locational and the striking point of ( 12 social interpretation is determined by the contrast betweenyear and hour. This has , with the meaning of theseitems as such, whether linguistic nothing to do , of course world but with or otherwise, knowledge about changesof location or institutional affiliation and their temporal frames. In a similar vein, the physical or abstract interpretation of lose and moneyin ( 13 ) : adverbial in the different world on adjuncts knowledge coming through depends ) a. John lost his money through a hole in his pocket. ( 13 b. John lost his money by gambling at cards. c. John lost his money by speculatingat the stock market. Notice incidentally, that his moneyin ( 13a ) refers to the coins or notes John is carrying ) it is likely to refer to all his wealth, again due to encyclopedic along, while in ( 13c domain. a certain about knowledge first now to the point concerning the internal structure of SF(le), I will Turning illustrate the issue by looking more closely at leave , providing at the same time an . To begin with , ( 14 outline of the format to be assumed for SF representations ) indicates the slightly simplified semanticform of leaveas it appearsin ( 12 ): ( 14 ) [x DO [BECOME [ NEG [x A Ty ]]]] , SF consists of functors and arguments that combine by functional Generally speaking . The basic elementsof SF in the sensementioned in type 2 above application DO BECOME like are constants , AT , and so forth and variables like x , y, z. More , , DO is a relation between an individual x and a proposition p with the specifically " " ), pis conceptual interpretation that could be paraphrasedby xperforms p . In ( 14 the proposition [BECOME [ NEG [x AT f ))] , where BECOME defines a transition into a state s characterizedby the condition that x be not at y . In short, ( 14 ) specifies ' the complex condition that x brings about a change of state that results in x s not , seeBierwisch being at y . For a systematicexposition of this framework in general in BECOME of DO and the and for 1988 , see Dowty particular ), interpretation ( all the elements that noted at this be 1979 . It should ) are showing up in ( 14 , point , ) ( . Thus with differing conceptual interpretations highly abstract and hencecompatible ), or an institutional affiliation , as in [x AT y] might be a spatial relation , as in ( 12a be interpreted by a spatial movement BECOMEs can x DO 12b . , ]] [ [ ) Correspondingly ( or a change in social position , depending on the conceptual content of the resulting state s.

Manfred

Bierwisch

But why should the lexical meaning of leavebe representedin the manner of ( 14 ), rather than simply as [x LEAVE y], if the conceptual interpretation must account for more specificdetails anyway? This brings us to the linguistic motivation of the internal structure stipulated for SF(Ie) . An even remotely adequateanswer to this question would go far beyond the scopeof this chapter, henceI can only indicate the type . Consider first ( 15 of motivation that bearson ( 14 ), ) by pointing out two phenomena which is ambiguous betweena repetitive and a restitutive reading: ) John left the institute again. ( 15 Under the repetitive reading, ( 15 ) statesthat John leavesthe institute for (at least) the second time, while under the restitutive reading ( 15 ) states only that John brings about of his not being at the institute , which obtained before. These two interpretations can be indicated by ( 16a , where x must be interpreted ), respectively ) and ( 16b AGAIN is a shorthand for the SF to be and the institute and where John , by y by assignedto again: ( 16 ) a. [AGAIN [x DO [BECOME [ NEG [x A Ty ]]]]] b. [x DO [BECOME [AGAIN [ NEG [x AT y]]]]] For discussionof intricate details left out here, seevon Stechow ( 1995 ) . Two points are to be emphasized , however. First , the ambiguity of ( 15 ) carries over to both the physical and the institutional interpretation ; it is determinedby linguistic, rather than , it could not be represented , if leave extralinguistic, conceptual conditions. Second were to be characterizedby the unanalyzedlexical meaning [x LEA VE y] . The secondphenomenon to be mentioned concernsthe intransitive use of leaveas in ( 17 ): ) John left a year ago. ( 17

Two observationsare relevant here. First , the variabley of ( 14 ) can be left without a in case it must be determined value which , interpreted by contextual syntactically conditions providing a kind of neutral origo . Second , the state [x AT y] under this condition is almost automatically restricted to the locative interpretation , which servesas a kind of default. Once again, although for different reasons , the global the relevant structure. LEA VE would fail to x y] provide representation[ The optionality of the object of leaveon which ( 17 ) relies brings in , furthermore, the intimate relationship between SF(le) and GF (le), or more specifically , the relationship between variables in SF(le) and the syntactic argument structure (or subcategorizatio , to useearlier terminology) . Supposewe include a specification of the SF variable, optionally or obligato rily interpreted by syntactic constituents, as one ) would be a more component into the syntactic information GF (le), such that ( 18 : complete lexical entry for leave

? How Much SpaceGets into Language


( 18) / Ieave / ~ .

~ ~ ~"V ~ ~! J Ix .~

DO [ BE CO M [ NEG [x AT y ]]] ! Ey .

PF (le)

GF (le)

SF(Ie)

Here x and y specify the obligatory subject position and the optional object position of leave , identifying the semantic variables to be bound by the corresponding , respectively constituents. Technically, x and y can in fact be considered as syntactic lambda operators, abstracting over the pertinent variables, such that assigningtheta roles, or argument positions for that matter, amounts semantically to functional , see , for example , Bierwisch ( 1988 ). application . For details of this aspect 2.3 Remarkson Modularity of Knowledgeand Representation The main reason to distinguish SF from syntactic representations , including LF , is the linguistically relevant internal structure of lexical items connectedto the conceptual . The compositional structure claimed for interpretation of linguistic expressions SF is very much in line with the proposals of Jackendoff ( 1983 , 1987 , and chapter 1, this volume) about conceptual structure (CS), with one important difference , however . The problem is for the relation of languageand space , which has consequences " this. Although what Jackendoff calls " lexical conceptual structure (LCS) is- details - very close in spirit to the SF information SF(Ie) of lexical items, he explicitly aside claims that conceptual structure (CS; and hence LCS) is an extralinguistic level of . Hence CS . In other words, CS is held to be external to I -language representation must obviously be identified with C-I (or perhaps a designated level of C-I ) .3 The architecture sketchedin ( 11 ): ) is thus to be replacedby something like ( 19 ) Audition ( 19

Articulation

Vision , / " " + + + + + A CS PS SS ./ ""


uditiol1
l J y

Locomotion

I -language Jackendoffproposesa principled distinction betweensystemsor modules of representation ), and supporting the levels of representation indicated by the labels in ( 19 rules representedby the arrows. This proposal is interface systemsor correspondence " connectedto what he calls " representationalmodularity , suggestingthat autonomy of mental modules is a property of representationalsystemslike phonological structure ), conceptual structure (CS; but also articulation , (PS), syntactic structure (SS . Autonomous modules of vision, etc.), rather than complex faculties like I -language this sort are then connected to each other by interface or correspondencesystems ,

which- by definition - cannot be autonomous, as it is their very nature to mediate betweenmodules. ' I -language , in Jackendoff s conception, comprisesPS, SS , and the correspondence rules connecting them to their adjacent levels but not CS. The bulk of correspondence , rules relating PS and SS , on the one hand, and SS and CS, on the other, are lexical items. While this is a plausible way to look at lexical items, it createsa conceptual problem. How can lexical items as part of the correspondencerules belong to I -language , if SF(le), or rather LCS, does not? To put it differently , either CS (and hence LCS) is not included in I -language or lexical items belong to the system of rules included in linguistic knowledge, but not both.4 correspondence One might , of course , argue that the problem is not conceptual, but merely on the appropriate characterization of I -language , which terminological, turning cannot in be schematized as 22 the lexical system not only cuts across simply ( ); the subsystems within I -language . , but also acrosslanguageand other mental systems I do not think this is the right solution , though, for at least three reasons . First , there seem to be substantial generalizations that crucially depend on the linguistic nature of SF(le), the principles of argument-structure being a major casein , then, SF(le) point . (This is a contention clearly sharedby Jackendoff.) In this respect is no lesspart of I -languagethan PF(le), or even GF (le) . Second , the phenomenadiscussedabove in connection with the interpretation of leave enter , , institute, and so on could not reason ably be explained without accounting for their fairly abstract linguistic structure and the specific distinctions that depend on factual knowledge. In other words, there seems to be a systematicdistinction betweenlinguistic and extralinguistic factors determining conceptual and referential interpretation . If thesedistinctions are not captured by two levelsof representation SF and C I in my terminology then two aspectsof CS must be distinguished in somewhat similar ways. But this would spoil the modular autonomy of CS and its extralinguistic status. Third , the nature of correspondencerules in general remains rather elusive. To some extent, they must belong to the core of linguistic knowledge based on the principles of UG , but they appear also to depend on quite different principles of mental organization. Although one might argue that this is just a consequence of actual fact, that linguistic knowledge is not a neatly separated system of mental organization, it seemsto me this conclusion can and in fact must be avoided. Let me return , in this regard, to the initial claim schematizedin (7), namely that I -language (based on UG ) simply determines a systematiccorrespondencebetween the domains APand C-I . In this view, I -language is altogether a highly specific interface mediating two independent systems of computation and representation .

? How Much SpaceGets into Language

Under this perspective , PF and SF are theoretical constructs sorting out those aspects of APand C-I that are recruited by UG in order to compute the correspondencein question. Hence PF (le) and SF(le) representstructural conditions projected into configurations . There are no correspondencerules connecting in APand C-I , respectively SF(le) to its conceptual interpretation , or PF (le) to articulation for that matter. Rather, the componentsof PF (le) and SF(le) as such provide the interface of APand -internal computation . It is the aim of this chapter to make this C-I with the language . view more precisewith respectto the subdomain of C-I representingspace Notice , first of all , that the difficulties concerning the status of CS are largely due to the notion of representational modularity , which is intended to overcome the ' ) concept of modularity . Replacing the inadequaciesencountered by Fodor s ( 1983 overall languagemodule by a number of representationalsystems , each of which is construed as an autonomous module, Jackendoff is forced to posit interface systems as well. Instead of speculating about the nature of these intermodular systems(are ?), I suggestwe go back to the they supposedto be encapsulatedand impenetrable notion of modularity first proposed by Chomsky ( 1980 ), characterizing systemsand . of tacit knowledge, rather than levelsof representation subsystems The notion of level of representation need by no meanscoincide with that of an autonomous module. To be sure, there is no systemof knowledgewithout representations to which it applies. But neither must one module of knowledge be restricted to one level of representation , nor must a level of representation belong to only one module of knowledge. I will not go here through the intricate details of subsystems and levelsof syntactic representation , where no simple correlation betweenlevelsand I want to indicate that , in a more general sense modules obtains. Instead, , different , determining systemsof rules or principles can rely on the same system of representation . What I have in mind , however, different aspectsof actual representations might best be illustrated by examplesfrom different nonlinguistic domains. A simple of digits. The same sequence case is the representational system consisting of sequences , might happen to be your birth date, your office phone number , say 12121942 or your bank account. Each of theseinterpretations belongs to a different subsetof , subject to different restrictions. For none of them is the fact that the sequences number is divisible by 29 relevant; each subsetdefinesdifferent neighbors , different constituents, and so on. Such interpretations of the same representation are based on different rules or systemsof knowledge, exploiting the same representational resources . Notice that certain operations on the representation would have the same effect for each of the interpretations, becausethey affect the shared properties of the representationalsystem , while others would have different effectson alternative : recruitings, as illustrated in (20a) and (20b), respectively

The notes exhibit simultaneously a position within the tonal systemand, because , of " their " names , within the Latin alphabet. Again, different rules apply to the two interpretations. This case is closer to what I want to elucidate than the different interpretation of digits . First , the tonal and the graphemic interpretation of the representation , albeit under different interpretations. Second , the apply simultaneously two interpretations rely on different cutouts of the shared representation . Although s all notes have alphabetic names , not all letters are representableby notes. Third , the more complete interpretation (in this casethe tonal one) determinesthe full representation , from which the additional interpretation recruits designated components , 6 imposing its own constraints. Obviously, even though theseillustrations are given in terms of external representations , it is the internal structures and the pertinent knowledge they are based on that we are interestedin. In this respect , digits and notes are comparable to language , . Moreover, while the examples rely on rules and exhibiting an E- and an I -aspect elementsthat are more or lessexplicitly defined, knowledge of languageis essentially basedon tacit knowledge. However, the artificial character of the twofold interpretation in our examples by no means excludes the existence of the same structural relationship with respectto systemsof implicit knowledge. In other words, the conceptual considerationscarry over to I -languageas well as other mental systems . It might be objected that the representationsconsideredabove are not really identical under their different interpretations, especiallyif we try to identify the information contained in their I -representation : digits representingdatesare grouped according to ; and so forth . In day, month , and year; telephone numbers, according to extensions - digits, notes, and so on- must be construed as other words, the relevant elements annotated in some way with respect to the rules of different systemsof knowledge. This seemsto me correct, but it does not change the fact that we are dealing with annotations imposed on configurations of the same representational system . Both of the and indication of affiliation aspects identity representationalsystem specific - are crucial with respect to the way in which different modules of knowledge are

? How Much SpaceGets into Language

. These considerations lead to what interfaced by a given representational system ' " " " might be called modularity of knowledge, in contrast to Jackendoff s representational " modularity . The moral should be obvious, but some comments seemto be indicated. First , the notion of interface- or correspondencefor that matter- is a relative . I -languageas a whole is a system concept, dependingon which modules are at issue that establish es an interface betweenAPand C I , with languagecapacity basedon UG providing the requisite knowledge. Furthermore, I -languagemust be interfaced . This sort of interface is not basedon rules that map with APand C-I , respectively one representationonto another, but rather on two types of knowledge that participate . In other words, PF and SF are the in one and the samerepresentationalsystem interfaces of I -languagewith APand C-I , respectively , which does not exclude the levels of C I further that APor , as we will see representation support possibility below. Second ) , if this is so, then the levelsof PF and SF are each determined by (at least two modules of knowledge, imposing conditions on, or recruiting elementsof , each other, possibly adding annotations in the sensementioned above. One might, of course , distinguish different aspects of one representation by setting up different . While this may be helpful for descriptive purposes levels of representation , it must . not obscurethe sharedelementsand properties of the representationalsystem PF we PF this at le under more , recognize (le) as ( ) perspective specifically Looking APIt is based on on the linguistic aspectimposed temporal patterns determined by articulation and perception, which include various aspectssuch as effectsof the particular ' speakers voice, emotional state, and so on. Theseare determined by their own . but are, so to speak , ignored by I -language subsystems interest here, we will now recognize which is of to SF le primary ( ), Turning finally it as the designatedaspectof C 1 to which I languageis directly related, using configurations . This leaves open of C-I as elements of its own, linguistic representation various possibilities concerning ( 1) how SF components recruit elementsor configurations of C-I ; (2) what annotations of SF must be assumed ; and (3) how rules and principles of C 1will contribute to the interface representationwithout being reflected in I -language . We will turn to thesequestionsin the sectionsbelow. To conclude this section, I want to schematizethe view proposed here by a slight modification of (8) :
( 22) . . . +- - + lPF + - - SYNTAX - - + SFJ +- - + . . . Y
- v- - - I ' - - -

A-P

I -language

y -

- -

C-I

Manfred Bierwisch

The main point is, of course , that SF is governed by conditions of I -language as well as those of C- I , although the aspect concerned need not be identical. (Parallel considerationsapply to PF.) The dots in (22) indicate the (largely unknown) internal organization of C-I , to which we turn now.

2.4 TheConceptualization of Space


What interests us is the internal representationof spaceand the knowledge underlying " it , which we might call " I -space , corresponding to I -language , and contrasting " " with physical, external or Espace. I -spacein this sense must be assumedto control and draw on information from a variety of sources ; it is involved primarily in visual perception and kinesthetic information , but it also integrates information from the vestibular system , auditory perception, and haptic information . All these systems provide nonspatial information as well. Vision integrates color and texture; haptic and kinesthetic information distinguish, among other things, plasticity and rigidity ; and so forth . I will therefore assume , following Jackendoff (chapter I , this volume), that I -spaceselects information from different sourcesand integratesit in a particular system of spatial representation(SR) . As a first approximation, SR should thus be construed as an interface representationin the sense ; that is, as mediating just discussed betweendifferent perceptual and motoric modalities, on the one hand, and the conceptual systemC-I , on the other, comparable to the way in which PF reconciles articulation and audition with I -language . Before looking more closely at the status of SR and its role for the relation betweenI -spaceand I -language , I will provisionally indicate the format and content to be assumedfor SR. , SR should meet the following conditions: According to generalconsiderations I . SR is based on a (potentially infinite ) set of locations, related according to three , with a topological and metrical structure imposed on this orthogonal dimensions set. 2. Locations can be occupied by spatial entities (physical objects and their derivates like holes, including regions, or shadows , substances , and events ), such that Loc (x ) is a function that assignsany spatial entity x its place or location. Spatial properties of physical entities are thus related to the structure imposed on the set of locations. 3. In general , Loc (x ) must be taken as time-dependent , such that more completely Loc (x , t) identifies the place of x at time t , presupposingstandard assumptionsabout time intervals. (Motion can thus be identified by a sequence of placesassignedto the samex by Loc (x , t ) .) 4. In addition to dimensionality, topological structure, and metrical structure, two further conditions are determined for locations:

? How Much SpaceGets into Language

a. orientation of the dimensions , marking especially a directed vertical dimension (basedon gravitation ); b. orientation with respectto a designatedorigo and/ or observerand intrinsic conditions of objects (canonical position or motion ) . , the dimenDepending on how physical objects are perceived and conceptualized . All sionality of their locations can be reduced to two , one, or even zero dimensions of this would have to be made precise in a serious theory of SR. The provisional outline given by conditions 1- 4 abovecan serve , however, as a basisfor the following remarks. Notice that although SR is transmodal in the sense already mentioned and must be considered as one of the main subsystemsthat contribute to the conceptual and intentional organization of experience , it should still clearly be distinguishedfrom the for at least two interrelated reasons . First , SR level of conceptual structure (CS) is assumedto be domain specific , representingproperties and distinctions that are , while conceptual structure must provide a representation strictly bound to spatial experience for experienceof all domains, including not only color , taste, smell, and auditory perception, but also emotion, social relation , goals of action, and so on, that is, information not bound to sensory domains in direct ways. Second , the type of representation at SR is depictive of or analogous to what it representsin crucial , while CS is abstract, propositional , algebraic, that is, nondepictive. All that respect " in the is neededfor a representationalsystemto be depictive is a " functional space senseexplained in Kosslyn ( 1983 ), which we have in fact assumedfor SR in conditions 1 and 2 . Becausethe distinction betweenthe depictive nature of SR and the propositional character of CS is crucial for the further discussion , let me clarify the point by the following simplified example: (23) a. 0 D~ b. i. A OVER B & B LEFT -OF C ii . A OVER B & C RIGHT -OF B iii . B LEFT -OF C & B OVER A

(24) a. A correspondsto O . B correspondsto D . C correspondsto ~ . b. x OVER y correspondsLoc (x )


+

Loc (y) (23a) is a pictorial representationof a situation for which (23b) gives three possible indicated in (24)- the , provided the correspondences propositional representations " conceptual lexicon apply, together with the principles that relate the functional structure" underlying (23a) to the compositional structure of the representationsin

Manfred Bierwisch

(23b) . Presupposingan intuitive understanding of the correspondencein question, which could be made precisein various ways, I will point out the following essential differencesbetweenthe format of (23a) and (23b) : I . Whereasthere is an explicit correspondencebetweenunits representingobjects in (23a) and (23b)- establishedby (24a)- there are no explicit units in (23a) representing the relational concepts OVER , LEfT OF , and so on in (23b), nor are there explicit elementsin (23b) representingthe properties of the objects in (23a), that is, the circle, the square , and so on. 2. The different distance between the objects is necessarilyindicated in (23a), even though in necessarilyimplicit way; it is not indicated in (23b), where it could optionally be added but only in necessarilyexplicit manner (e.g., by adding coded units of measurement ). 3. Additional properties or relations specified for an object in (23b) require a repeated " " representationof the object in question, while no such anaphoric repetition showsup in (23a); for the samereason , (23b) requires logical connectivesrelating the elementarypropositions, while no such connectivesmay appear in (23a). 4. Finally , (23b) allows for various alternative representations correspondingequivalently to the unique representation in (23a), while (23a) would allow for variations that need not show up in (23b), for example , by different distances between the objects. In general , the properties of (23a) are essentially those of mental models in the sense discussedby Johnson- Laird ( 1983 , and chapter II , this volume) and by Byrne and Johnson Laird ( 1989 ), who demonstrate interesting differences between inferences basedon this type of representation , as opposedto inferencesbasedon propositional of 23b . representations type ( ) Returning to SR, it seemsto be a plausible that it constitutes a conjecture pictorial representation in the senseof (23a), with objects representedin terms of 3 0 models in the senseof Marr ( 1981 ), or configurations of geons as proposed by Biederman ( 1987 , and chapter ) . SeeJackendoff ( 1990 I , this volume) for further discussion . It differs from CS by formal properties like I . to 4., allowing for essentially different operations based on its depictive character, which supports an analogical relation to conditions of Espace. The next point to be noted is that SR as construed here is a level of representation , not necessarilyan autonomous module of knowledge. Given the variety of sourcesit , it seemsin fact plausible to assumethat SR draws on different systemsof integrates mental organization. According to the view proposed in the previous section, SR might rather be considered as one aspect of a representational system shared by different modalities, visual perception providing the most fundamental as well as the

? How Much SpaceGets into Language

most differentiated contribution . This leavesopen whether, and to what extent, the SR aspect of the representational system is subject to or participates in operations ) like imaging or mental rotation of objects, which are argued by Kosslyn et ale( 1985 . to be not only depictive, but also modality -specific This leavesus with the question of how SR relatesto the overall systemC-I and the level of conceptual structure in particular . If the comments on the propositional character of CS and the depictive nature of SR are correct, then SR and CS cannot . On the other hand, SR be two interlocked aspectsof the samelevel of representation to the extent to which it is to be identified with Johnsonmust belong to C-I , because Laird ' s system of mental models, it supports logical operations similar in effect to those basedon the propositional -level CS, albeit of a different character. The obvious . This conclusion is that C-I comprises at least two different levels of representation conclusion should not be surprising; it has in fact a straightforward parallel in 1, where PF and SF also constitute two essentiallydifferent representational language systemswithin the sameoverall mental capacity. To carry this analogy one step further , what I have metaphorically called the " " conceptual lexicon (24) correspondsin a way to the lexical entries. Just as PF (le) indicates how the corresponding SF(Ie) is to be spelled out at the level of PF, the pertinent 3-D model determinesthe representationof a given concept on the level of SR. More generally, and in a less metaphorical vein, the correspondencebetweenSR and CS must provide the SR rendering of the following specifications for spatial conditions: I . Shapeof objects, that is, proportional metrical characteristicsof objects and their ); parts with respectto their conceptually relevant axesor dimensions(3 D models with the of characteristics metrical that is 2. Size of objects, , objects interacting relevant shapecharacteristics ; 3. Place of objects, that is, relations of objects with respectto the location of other objects; and 4. Paths of (moving) objects, that is, changesof place in time. , , for instance Obviously, specifications1- 4 are not independentof eachother. Shape is to some extent determined by size and place of parts of an object; paths- as of places ; and so forth . Jackendoff (chapter I , this already mentioned- are sequences volume) points out further aspectsand requirements to be added, which I need not repeat here. The main purpose of the outline given above is to indicate the sort of CS information that SR is to account for , without trying to actually specify the format of representations , let alone the principles or rules by which the relevant knowledge is organized.

Manfred

Bierwlsch

I will conclude this sketch of the status of I -spacewith two comments that bear on the way spatial information is conceptually structured and eventually related to SF, and hence to I -language . First , it is worth noting that commonsenseontology , , the sortal and type structure of concepts . namely , is entrenchedin someway in I -space More specifically, the informal rendering of SR in conditions 1- 4 at the beginning of this section freely refers to objects, events , places , properties, relations, and so on or in fact I because the corresponding ontology , suppose legitimately, , necessarily holds also for SR. This observation, in turn , is important for two reasons : ( I ) in spite of its domain specificity, SR shareswith general conceptual organization basic onto logical structures; and (2) by virtue of this common ground, SR not only provides entities in terms of which intended reference in C-I can be established and interpreted; it also participates in a general framework that underpins the interface with general conceptual structure. I will assume , for example , that 3-D models spell out properties in SR that general conceptual knowledge combines with nonspatial knowledge about specific types of physical objects. Thus the commonsensetheory about cats will include conditions about the characteristic behavior, the taxonomic classification, and so forth of cats, along with access to the shapeas specifiedin SR. I will return to this problem in the next section. My secondcomment has two parts. ( I ) I want to affirm that spatial representation as discussedthus far respondsto properties and relations of physical objects, that is, to external conditions that constitute real, geometrical space . We are dealing with , one might say, basedon spatial perception of various sorts, spacein the literal sense as mentioned above. This leads to (2) the observation that spatial structures are to extensivelyemployed in many other conceptualdomains. Time appearsnecessarily be conceptualizedas one-dimensional, oriented spacewith eventsbeing mapped onto intervals just like .objects being mapped onto locations. Hierarchies of different sorts, such as social, evaluative, taxonomic, and so on, are construed in spatial terms; further domains- temperature , tonal scales , loudness , color - come easily to mind. More complex analogiesin the expressionof spatial, temporal, possessional , relations have been discussed , for example , by Gruber ( 1976 ) and by Jackendoff ( 1983 ). The conclusion from this observation is this. The basic conditions of I -spaceas listed at the beginning of this section seemto be available as a general framework underlying different domains of experience , which immediately raises the question of how this generalizedcharacter of basic spatial structures is to be explained. Becausetaxonomies, social relations, and even time do not rely on the same sourcesof primary , the transmodal aspect in question clearly must exceed I -space (in the experience sense assumedthus far ), functioning as an organizing structure of generalconceptual knowledge.

? How Much SpaceGets into Language

Basic structures of spatial organization must therefore either 1. constitute a general schema of conceptual knowledge imposed on different domains according to their respectiveconditions; or 2. originate as an intrinsic condition of I -spaceand are projected to other domains on demand. According to alternative 1, actual three-dimensional spaceis the prevailing, dominant instantiation of an abstract structure that exists in a senseindependent of this instantiation as a result of experience ; according to alternative 2, the structure emerges in the primary domain. The choice between these alternatives has clear empirical , but it is a difficult choice , and phylogeneticrespects impact in structural , onto genetic to make, given the present state of understanding of conceptual structure. I tentatively : ( 1) I -spaceis assumethat alternative 2 is correct for the following two reasons not only a privileged instantiation of spatial structure but is also the richest and most detailed instantiation of spatial structure, compared to other domains. Whereas 1space is basically three-dimensional, other domains are usually of reduced dimensionality , as Jackendoff (chapter 1, this volume) remarks. Orientation with respectto frame of referenceis accordingly reduced to only one dimension. (2) While size and place carry over to the other domains with scalar and topological properties, shape has only very restricted analogy in other domains. I will thus consider the full structure of I -spaceas intrinsic to this domain due to its specific input , rather than as an abstract potential that happens to be completely instantiated in I -spaceonly . These structural considerations might be supplementedby onto genetic and phylogenetic considerations , which I will not pursue here. In any case , whether imported to I -spaceaccording to alternative 1, or exported from it according to alternative 2, dimensionality and orientation require appropriate structures of other domains, or rather of conceptual structure in general , to correspond common sense with to to. This is similar to what has been said earlier respect ontology , with its type and sortal distinctions. It might be useful to distinguish two types of transfer of spatial structure. I will consider as implicit transfer the dimensionality and orientation of domains like time or social hierarchies , whose conceptualization follows these patterns automatically , that is, without explicit stipulation . In contrast, explicit transfer shows up in cases where dimensionality is used as a secondary organization, imposing an additional . The notion of color spaceor property spaceis based structure on primary experience on this sort of explicit transfer. The boundary betweenexplicit and implicit transfer need not be clear in advanceand might in fact vary to some extent, which would be of alternative 2. In what follows , I will not deal with explicit a natural consequence transfer but will argue that implicit transfer is a major reason for the observation

noted at the outset , namely , that there is no clear distinction between spatial and nonspatial terms . The relations expressed, for example , by in , enter , or leave are not restricted to space because of the implicit transfer of the framework on which they are based.

2.5 Types of Space Relatednea in Conceptual Structure


Let us assume , to conclude the foregoing discussion , that the conceptual-intentional system(C I ) provides a level of representation(CS) by which information of different modules is integrated, looking more closely at the way in which spatial information is accommodatedin CS. Notice first of all that assumptionsabout the properties of CS can only be justified by indirect evidencebecause , by definition , CS dependson various other systemsrelating it to domain-specific information . There seemsto be indicated , however, that CS is propositional in nature, in the sense generalagreement above and discussedin more detail, for example Fodor 1975 , by ( ) and by Jackendoff , 1990 , and chapter I , this volume) . The two main sourcesrelied on in specifying ( 1983 CS are languageand logic. On the one hand, CS is modeled as tightly as possible in accordancewith the structure of linguistic expressions to be interpreted in CS; on the other hand, it is made to comply with requirements of logical inferencesbased on situations and texts. As to the general format of CS, two very general assumptionswill be sufficient in the presentcontext. First , CS is basedon functor -argument-structure, with functional application being the main (and perhaps only) type of combinatorial operation. Hence CS does not rely on sequential ordering of elements but only on nesting according to the functor -argument structure. There are various ways in which these , a particularly explicit version being Kamp and assumptions can be made precise 1993 . Second I will , ) Reyle ( supposethat CS exhibits a fairly rich sortal structure common sense provided by ontology . Both assumptionsshould allow CS to be interfaced with the semanticform (SF) of linguistic expressions earlier. , as discussed I will refrain from speculationsabout the primitive elementsof CS, with two exceptions : ( I ) the primes of SF must be compatible with basic or complex units of CS, if the assumptions about SF and its embedding in CS are correct; and (2) CS must accommodateinformation from various domains, including SR, possibly treating for , specificationsof 3-0 modelsas basicelementsthat feature in CS representations example . I will return to exception 2 shortly . Note , furthermore, that CS must not be identified with encyclopedicknowledge in . Although commonsensetheories by which experienceis organized and explained general must have access to representationsof CS, their format and organization are

HowMuchSpace Gets into Language ?


to be distinguished from bare representational aspectsof CS. It has been suggested ; Pustejovsky 1991 (e.g., Moravcsik 1981 ) that commonsensetheories are organized by explanatory factors according to Aristotelian categorieslike structure, substance , function , and relation. It remains to be seenhow this conjecture can be made explicit in the formal nature of commonsense knowledge. Pending further clarification , I will I simply assumethat C determinesrelevant aspectsof CS on the basis of principles that organize experience . Turning next to the way in which CS and commonsenseknowledge integrate I -space , three observationsseemto me warranted: I . Commonsense , ontology -requiresphysical entities to exhibit spatial characteristics . including in particular shapeand sizeof objects and portions of substance This observation distinguishes " aspatial" conceptual entities- mental states , informational structures (like arguments or and social institutions , songs , poems ), from those subject to spatial characterization. Although these aspatial entities are invested with spatial characteristics by the physical objects implementing them, it should be clear enough that , for example , a poem as a conceptual entity is to be from the letters that distinguished printed representit . 2 . Encyclopedic knowledge mayor may not determine particular definitional or characteristic spatial properties within the limits set by ( I ) . This observation simply notes that spatial entities are divided into those whose , and those without specifications typical or essentialproperties involve spatial characteristics of this sort. Dog, snake , child, table, or pencil expressconcepts of the first type, while animal, plant, tool,furniture exemplify conceptsof the secondtype, which, although inherently spatial, are not characterizedby particular spatial information . Actually observation 2 does not set up a strictly binary, but rather a gradual distinction , dependingon the specificity of shape , size , and positional information . Thus the concept of vehicle is spatially far less specific than that of cat or flute , but it still contains spatial conditions absent in the concepts of machineor musical instrument , even though theseare not aspatial. Also , the specifity of spatial properties seemsto , as Landau (chapter 8, this volume) vary in the course of onto genetic development , showing that young children initially tend to invest conceptsin general with argues spatial information . 3. Conceptual units may specify spatial properties or relations without involving any nonspatial properties of entities they can refer to. While observations I and 2 distinguish conceptual entities with respect to their participation in spatial conceptualization, observation 3 separatesconceptual units that specifypurely spatial conditions for whatever entities fall within their range from conditions that inextricably involve additional conceptual information . Thus square ,

Manfred Bierwisch

, circle, top (in one reading) expressstrictly or exclusively spatial conceptswhile edge or dog cup include- in addition to shapeand size information - further systematic conceptual knowledge. It should be borne in mind that we are talking here about conceptual units, using linguistic expressionsonly as a convenient way of indication . For the time being, we ignore variability in the interpretation of lexical items, which might be of various sorts. Thus lexical items expressingstrictly spatial concepts are extensively used to refer to " typical implementations" like corner, square , margin, and so on. Expressions for aspatial concepts , on the other hand, for example , social institutions like parliament or informational structures like novel or sonata , are used to refer to spatial , as already mentioned. Theseare problems objects where they are located or represented of conceptual shift of the sort mentioned in section 2.2, which must be analyzed in their own right . The different spatial character of conceptsdiscussedthus far can be schematically summarizedas follows: Example fear , hour, duration animal, robot, instrument horse , man, violind , margin, height square Observation 1 distinguishesbetween(25a) and (25b- d); observation 2 separates (25d) " " from (25a c) . Extrinsically spatial refers to conceptsthat require spatial properties but do not specify them; " intrinsically spatial" indicates the specificationof (someof ) these properties. It should be noted that intrinsically spatial properties might be . SeeKeil ( 1987 ) typical or characteristic, without being definitional in the strict sense for relevant discussion . As already mentioned, the distinction between (25b) and (25c) is hence possibly to be replaced by various steps according to the specificity of spatial information . The main point is that concepts can involve more or less specificspatial information , but neednot fix it , even if they are essentiallyspatial. It is worth noting that the samedistinctions (with similar provisos) apply to other domains of conceptual organization, color and time being casesin point : (26) Type of color -relatedness a. No relation b. Extrinsic c. Intrinsic d. Strict Example live, hour, height liquid, animal, tool blood, zebra , sky red, black, colorlessness (25) Type of concept a. Aspatial b. Extrinsically spatial c. Intrinsically spatial . Strictly spatial

? How Much SpaceGets into Language

(27) Type of time-relatedness a. No relation b. Extrinsic c. Intrinsic d. Strict

Example number , water, lion , travel fear , commettee death, inauguration , beat hour, beginning , duration

There are numerous problems in detail, which would have to be clarified with respect to the particular domains in question. The point at issueis merely that the observations . 1- 3 noted above are not an isolated phenomenonof space Thus far I have illustrated the distinctions in question with respect to objects of different sorts. The observations apply, however, in much the same way to other onto logical types, such as properties, relations, and functions; (28) gives a sample illustration : (28) a. b. c. . Aspatial Extrinsic Intrinsic Strict Property clever , sober , famous colored , wet, solid striped, broken, open upright, long, slanting Relation , during acknowledge kill , show , write close , pierce, squeezed under , near, place

Notice, once again, that we are talking about concepts , not about the nouns, verbs, them. In addition to distinctions blurred by this , prepositions expressing adjectives . Thus difficulties must be observed further long, as shown in the appendix practice, es actually a three place relation , rather than a property . The main below, express point should be clear, however. Conceptsof different types are subject to the distinctions related to observations 1- 3. The distinctions discussedthus far are directly related to two additional observations important in the present context. First , there are, on the one hand, concepts ' " with a fairly rich array of different conditions- Pustejovskys ( 1991 ) qualia structure " for , explanation. Concepts example integrated into theories of commonsense artifacts like car or elevator but also of natural kinds like dog or raven , combine more , about function or lessspecificshapeand size information with knowledge , behavior, substance , and so on that might be gradually extended on the basis of additional . On the other hand, there are relatively spare concepts such as near, experience stand , , basedon highly restricted conditions of only one or two domains. Let square " " and " me call thesetwo kinds " rich concepts , for the sakeof discussion spareconcepts . There is, of course , no sharp boundary here, but the differenceis relevant in two : ( I ) spareconceptsmight in fact enter into conditions of rich concepts , with respects rich conceptsbeing subject to further elaboration, while spareconceptsare just what they are; and (2) it is essentiallyrich conceptsthat constitute commonsensetheories:

Manfred Bierwisch

although spare concepts like in or long can feature in explanations, they do not , record and circle, we notice that circle is explain anything . Contrasting, for example which relies, however, on knowledgeexplaining in record information of the , shape part of detail sound storage (in varying degrees ), while nothing (beyond mere geome , the distinction of rich and spare try) is explained by circle. For almost trivial reasons between extrinsic and with the distinction but is not identical relates to ) ( concepts . Strictly spatial concepts to as intrinsic spatial concepts , opposed strictly spatial concepts not vice versa. but can be integrated into intrinsically spatial ones , Related to this is the secondobservation. Specificationsrepresentedin SR can be " " " " relied on in CS in two ways, which I will call explicit and implicit . Detailed shape information , for instance , representedin SR by 3 D models, enters the pertinent which meansthat neither the internal structure of 3-D models nor conceptsimplicitly , " " " the properties reconstructing them like " four -legged or long-necked enter CS , but rather the shape information as a whole. In contrast, strictly representations , far , tall , and so on must explicitly representthe relevant spatial conceptslike behind in terms of conceptual primitives . One might take this as a corollary conditions spatial : of the classification illustrated in (25) in the following sense Strictly spatial conceptsrepresentspatial information explicitly in terms of conceptual primes; intrinsically spatial conceptsrepresentspatial information implicitly , that is, encapsulatedin configurations of SR. The moral of all of this with respect to our initial question would thus be something like the following . CS extracts information from SR in two ways: ( I ) encapsulated in SR configurations that are only treated holistically, defining, so to speak , an open set of primes in terms of conditions in SR, and (2) explicitly representedby means of conceptual primes that directly recruit elementsof SR. Becausewe have further assumedthat CS is the interface of C-1 with I -language , it follows that SF has to SR. I will return to this point below. Although I take this moral two typesof access to be basically correct as a kind of guideline, there are essentialprovisos to be made, even if the notion of explicit and implicit representation can be made formally precise . can be overcome , and even if the usual problems with borderline cases A major problem to be faced in this connection is the fact that in CS strictly spatial (i.e., explicit) conceptsmust appropriately combine with implicit spatial information . Thus, for the complex conceptsexpressed by short man, long table, or steeproof, the or information of short , , steepmust be able to extract the relevant long strictly spatial dimensional and orientational information from the encapsulatedshaperepresentation of man, table, or roof A useful proposal to overcomethis problem is the notion of object schematadevelopedin Lang ( 1989 ). An object schemaspecifiesthe conditions that explicit representationscould extract from encapsulatedshape informa -

? Gets intoLanguage HowMuchSpace


tion , in particular , dimensionality, canonical orientation and subordination of axes relative to eachother. Even though an object schemais lessspecificthan a 3-D model, it is not just a simplification of the model, but rather its rendering in terms of primes of the strictly spatial sort. An object schemamakes 3-D models respond to explicitly . Notice that there are default schemataalso for extrinsically , so to speak spatial concepts spatial conceptsthat do not provide a specified3-D model, as combinations like ). ) and Lang ( 1989 long instrumentshow. For details seeBierwisch and Lang ( 1989 A final distinction emergingfrom the observationsabout I -spaceand C-I should be of the implicit transfer imposing basic structures of I -space noted. As a consequence on other domains, which we noted above, it seems plausible to assumethat explicitly spatial concepts like in, length, and around do in fact relate to I -space and other domains to which the pertinent structures are transferred. In other words, we are led to a distinction between elementsof CS that are exclusively interpreted in SR and elementsthat are neutral in this respect , being interpreted by structures of SR that transfer to other domains. The latter would include only explicit concepts , which are . strictly spatial only if interpreted in I space Not surprisingly, we found a fairly rich typology of different elementsand configurations thereof in CS, depending only on the way in which SR as a representational systemrelatesto I -spaceas well as other cognitive domains. I would like to stressthat the observations from which this typology derives , are not stipulated conditions but about the architecture of subsystemsof of basic assumptions simply consequences C-I and their internal organization.

: Outlineof a Program 2.6 BasicSpatialTenns


Assuming that the relation of spatial cognition and conceptual structure is to be construed along the lines sketched thus far , the central question we posed at the outset boils down to two related questions: 1. How is I -spacereflectedin CS? ? 2. How are spatial aspectsof CS taken up in SF We have already dealt with question 1. A partial answer to question 2 is implied by the assumption that SF and CS, although determined by distinct and autonomous , but systemsof knowledge, neednot be construed as disjoint representationalsystems modules of to different rather as ways to recruit pertinent configurations according knowledge. Pursuing now question 2 in more detail, I will stick to the assumption made earlier, that SF can be thought of as embeddedin CS, such that the conditions on the format of SF representationsoutlined in section 2.2 would carry over to the format of CS, unlessspecific additional requirementsare motivated by independent

evidenceconcerning the nature of CS. Such additional requirementsmight relate, for . , to commonsenseontology and the sortal systemit induces example 2 is which elementsof the main issue raised With theseprerequisites , by question in I . An additional CS are recruited for lexicalization language point concerning further grammaticalization in terms of morphological categorieswill be taken up in section 2.7. I will restrict the issueof lexicalization to strictly spatial conceptsfor two reasons : ( I ) to go beyond obvious, or even trivial , statementswith respectto encapsulated information of intrinsically spatial concepts , including the intervening effectsof , would by far exceedthe limits of this chapter; and (2) understanding object schemata the lexicalization of strictly spatial conceptswould be a necessary precondition in any case . Given theseconsiderations , the following researchstrategy seemsto be promising, and has in fact been followed implicitly by a great deal of researchin this area. First we define the systemof basic spatial terms ( BST, for short) of a given language , and . The notion of then we look at the properties they exhibit with respectto question 2 basic spatial terms has beenborrowed from Berlin and Kay ' s ( 1969 ) basiccolor terms . Becausespace is a far and is similar in spirit , though different in certain respects more complex than color , BSTs cannot, for example , as , be restricted to adjectives basic color terms can. Basic spatial terms can be characterizedby the following criteria: I . BSTs are lexical items [ pF(le), GF (le), SF(le)] that belong to the basic (i.e., ; morphologically simple), native, core of the lexical systemof a given language 2. In their semantic form [SF(le)], BSTs identify strictly spatial units in the sense above. discussed are not , violating Thus short, under , side , lie are BSTs, while hexagonaland squeeze . It should be emphasizedthat BST is a purely criterion I and criterion 2, respectively heuristic notion with no systematic impact beyond its role in setting up a research strategy. Hence one might relax or change the criteria should this be indicated in order to arrive at relevant generalizationsor insights. Thus my aim in assumingthese criteria is not to justify the delimitation they define, but rather to rely on them for . practical reasons It is immediately obvious that the two criteria , even in their rather provisional of BSTs: form , lead to various systematicallyrelated subsystems I . Linguistically , BSTs belong to different syntactic and morphological categories , and perhaps classifiers and inflections for (verbs, nouns, prepositions, adjectives Case ); 2 . Conceptually, BSTs are interpreted by different aspects of space (size , shape , of motion etc. . size , , ) place, change

How Much SpaceGets into Language ?

Of particular interest is, of course , the relation betweenlinguistic ( 1) and conceptual 2 whether ( ) subsystems , systematicor incidental. Ultimately , a researchstrategy taking BSTs as a starting point is oriented toward (at least) three aims, all of which are related to our central question: . Identification of the conceptual repertoire available to BSTs. This includes in particular the question whether universal grammar provides an a priori system of potential conceptual distinctions that can be relied on in the SF of BSTs- parallel to what is generally assumedfor PF primes- or whether the distinctions made in SF are abstracted from actual experienceand its conceptualization. . Identification of basic patterns, either strict or preferential, by which UG organizes BSTs with respectto their SF, as well as their syntactic and morphological properties. . Identification of systematicoptions that distinguish languageswith respectto the repertoire and the patterns they rely on. This problem might be couched in terms of parametersallowing for a restricted number of options, or simply as different ways to idiosyncratically exploit the range of possibilities provided by principles of C-I and UG . As a preliminary illustration , I will have a look at the reason ably well understood structure of dimensional adjectives (DAs , for short) like long, high, tall , short, and low, the interpretation of which combines conditions on shape and size . Generally , a DA picks out a particular, possibly complex, dimensional aspect of the speaking entity it applies to and assignsit a quantitative value. Characteristically, DAs come in antonymous pairs like long and short, specifying somehow opposite quantitative values with respect to the same dimension. Thus the sentencesin (29) state that the maximal dimension of the boat is above or below a certain norm or average , : respectively (29) a. The boat is long. b. The boat is short. The opposite direction of quantification specified by antonymous DAs creates rather intriguing consequences , however, as can be seenin (30) : (30) a. b. c. d. The boat is twenty feet long and five feet wide. * The boat is ten feet short and three feet narrow. The boat is ten feet longer than the truck. The boat is ten feet shorter than the truck .

In other words, a measurephrase like tenfeet can naturally be combined only with the " positive" DA - hencethe deviancy of (30b)- exceptfor the comparative, where it combines with the positive as well as the negative DA . Theseand a wide range of

58

ManfredBierwisch

other phenomena discussedin Bierwisch ( 1989 ) can be accounted for , if DAs are assumedto involve three elements : ( 1) an object x evaluated with respect to a spe cified dimension; (2) a value v to be compared with ; and (3) adifferencey by which x either exceeds or falls short of v. While x and yare bound to argument positions to be filled in by syntactic constituents the DA combineswith , v is left unspecifiedin the positive and made available for a syntactically explicit phrase by the comparative morpheme. Using the notational conventions illustrated in ( 18 ), the following entries for long and short can be given: (31) jlongj Adj x (j ) [[QUANT [MAX x ]] = [ v + y]] I Deg (32) jshortj Adj x (j ) [[QUANT [MAX x ]] = [v - y]] I Deg As in ( 18 , x and j are operators binding semantic variables ), the entry for leave to syntactic arguments where the optional degreecomplement is morphologically , marked by the grammatical feature Deg that selectsmeasurephrasesand other degree . Semantically, long and short are identical except for the different complements functor + as opposed to - . The common functor MAX picks up the maximal dimension of the argument x , which then is mapped onto an appropriate scaleby the operator QUANT . The scalar value thus determined must amount to the sum or differenceof v and y , where the choice of the value for v is subject to rather general semanticconditions responsiblefor the phenomenaillustrated by (29) and (30) . One option for the choice of the variable v is Nc, indicating the norm or averageof the class C which x belongs to. It accounts for the so-called contrastive reading that shows up in (29), while in (30) v must be specifiedas the initial point 0 of the scale selectedby QUANT . Three points can be made on the basis of this fairly incomplete illustration . First , the semantic form of dimensional adjectives , providing one type of BSTs, has a nontrivial compositional structure in the sense introduced in section 2.2, from which crucial aspectsof the linguistic behavior of these items can be derived. Second , the elementsmaking up the SF of theseitems have an obvious interpretation in terms of the structural conditions provided by SR, even though this interpretation is anything but trivial . Especially the way in which MAX and other dimensional operators like VERT or SEC for the vertical or secondary dimension of x are to be interpreted follows intricate conditions spelledout in detail in Lang ( 1989 ) . Third , the entries (31) and (32) immediately account for the fact that long and short apply not only to spatial

How Much SpaceGets into

! Ian"guage

entities in the narrower sensebut to all elementsfor which a maximal dimension is defined, such as a long trip , a short visit, a long interval, and so on, due to the projection of spatial conditions to other domains in the sensediscussedabove. Note that the choice of the scaleand its units determined by QUANT must be appropriately of the interpretation of MAX . I will place this initial specifiedas a consequence illustration of BSTs in a wider perspectivein the appendix, looking at further conditions for basic patterns and their variation .

Dof Space 2.7 Grammaticalizatio


The elementsand configurations consideredthus far are supposedto be part of the . As part of the interface, they determine directly the semantic form of I -language ; their impact on the computational conceptual interpretation of linguistic expressions structure of I language , via argument positions, is only indirect and does , for example not dependon their spatial interpretation as such. The problem to be consideredbriefly in this section concernsthe relation between elementsof the morphosyntactic structure of I -language and spatial interpretation . As rationale for this question, there are categoriesof I -language that clearly enter , strictly morphological and syntactic relations and operations such as agreement of to conditions related that are but concord, and categorial selection , obviously , and tense are obvious casesin , number, gender conceptual interpretation . Person point . Before taking up this problem with respectto spatial properties, I will briefly consider the status of grammatical categorieswith semanticimpact more generally. The problem to be clarified is the need to reconcile two apparently incompatible claims. On the one hand, morphological and syntactic primes, type 3 as indicated in section 2.2, differ from phonetic featuresand semanticcomponentsby the lack of any extralinguistic interpretation , their content being restricted to their role within the . On the other hand, there cannot be any doubt computational systemof I -language do tense or that , for example , person have semanticpurport in someway. The way out of this apparent dilemma can be seen by looking more closely at . [ :t Plural] is clearly a feature that enters the morpho number as a paradigm case . The details of inflection , syntactic computation of English and many other languages concord, and agreementthat depend on this feature need not concern us here; it is clear enough that theseare strictly formal conditions or operations. It is equally clear there must be some kind of an operator in SF related to [ + Plural] that imposesa condition on individual variables turning their interpretation into a multiplicity of individuals , although the details once again need not concern us. The relation between clear in casesof conflict , such as the pluralia tantum thesetwo aspectsbecomes

Manfred Bierwisch

" ), but to a single object in of (33), where " glasses refers to a set of objects in ( 33a (33b) : were collected by the waiter. (33) a. Their glasses . were sitting on his nose b. His glasses " " Obviously, the feature [ + Plural] of glasses cannot be responsiblefor the set reference in (33a), as it must be lacking in (33b) . Another type of conflict is illustrated by " " shown by (34a ), but does not 34 ( ), where who must allow for set interpretation , as " each other" : antecedent the required by plural provide (34) a. Who was invited? (Eve, Paul, and Max were invited.) b. * Who does not talk to each other? (Eve and Paul.) Further types of dissociation betweenmorphological number and semantic individual / set interpretation could easily be added. The conclusion to be drawn from these observations is obvious. The feature [ :t: Plural] is related to , but not identical to , the presenceor absenceof the semanticset operator. More specifically, [ + Plural] in the default causeis related to the operator SET; [ - Plural] to the lack of this operator. in some How this relation is to be captured is a nontrivial problem, which resembles respectsthe phonological realization of [ :t: Plural] and other morphological categories . Thus the suffix / - s/ is the default realization of [ + Plural] for English Nouns, but is, of course, just as different from [ + Plural] as SET is. Notice , however, that both the phonological realization and the semanticinterpretation of the default case might be instrumental in fixing the morphological category in acquisition as well as . Similar, albeit more complex, accounts might be given for categories in languagechange its relation to sex and animateness and like gender , or tenseand its relation to . temporal reference More generally, for morphological categories , the following terminological convention to be useful: seems A semanticcondition - that is, a configuration of primes of SF- is grammaticalized, if there is a morphological category M to which Cisrelated by certain rules or conditions R. The conditions R should be considered as the semantic counterpart to inflectional morphology , which relates morphological categoriesto configurations in PF. I am not going to make serious proposals as to the formal nature of R at the moment. The simplest assumption would be to associatea morphological category, such as in [ + Plural], with someelementin SF, such as SET, in a way that will be suspended be then would the association of . The potential suppression specificallymarked cases of the autonomous character of the morphological category, whereas a consequence

? HowMuchSpaceGets into Language


its actual realization indicates the conceptual purport of the formal category inquestion . Instead of pursuing these speculations , I will briefly look at the grammaticalization of spatial componentsin the sense specifiedin the above convention. Two candidates are of primary interest in this respect : ( I ) casesystemsincluding ; and (2) classifier systems , sufficiently rich distinctions of so-called notional cases . We must expect in general not a , respectively corresponding to location and shape , but rather straight and simple realization of spatial information by thesecategories a more or lesssystematicmapping, whose transparencywill vary, depending on how entrenchedthe morphological categoriesare in autonomous computational relations like concord and agreement . That notional casesare related to spatial information about location is uncontroversial and has beenthe motivation for the localistic theory of casementioned earlier. In agglutinative languageslike Hungarian, there is no clear boundary separating . The semantic information related to locational and directional postpositions from cases caseslargely matchesthe schemaof the corresponding prepositions discussed in the appendix, as shown in simple caseslike (35) : ' ' (35) a. ahaz - ban in the house . the housein -ben b. Budapest ' in ' Budapest -re c. Budapest ' ' to Budapest Even though things are far less transparent in more elaborate systems , it is sufficiently clear that place information can be grammaticalizedby inflectional categories . For an extensivestudy of complex casesystems(including Lak and Tabassarian ) that is relevant under this perspective , even though it is committed to a different theoretical - 37, part 1) . framework , seeHjelmslev ( 1935 Classifier systemsare subject to similar variations with respect to differentiation and grammatical systematization . A characteristic example is Chinese , where classifiers are obligatory with numerals for syntactic reasons , and related to shape in caseslike (36) : iects) (36) a. liDO(longish. thin ob_ yi tiao lie one CL street ' one street' liang tiao he two CL river ' two rivers'

Bierwisch Manfred

b. zhang(planar objects) liang zhang xiangpian two CL photograph ' ' two photographs san zhang zhuozi table three CL ' three tables' c. kuai (three-dimensional objects) yi kuai zhuan one CL brick ' one brick ' san kuai feizao three CL soap ' ' three cakesof soap The SF conditions to which theseclassifiersare related are not particular 3-D models but rather abstract object schemata of the sort mentioned above, which must be available, among others, for dimensional adjectivesof English or German, for Tzeltal positional adjectivesdiscussedin the appendix, but also for positional verbs like lie, sit, or stand, albeit in different modes of specification. Even though the details need clarification , it should be obvious that shape information can correspond to grammatical . categories I will conclude thesesketchy remarks on the grammaticalization of spacewith two . more general considerations concerning the range and limits of these phenomena . The first position takes There are, in fact, two opposite positions in this respect the as structure computational structure of I -language immediately supporting spatial and the categoriesof syntax and morphology . A tradition directly relevant is the locationist theory of case , according to which not only notional but also structural , contact, casesare to be explained in terms of spatial concepts like distance lines is given these account ambitious . The most orientation and coherence along , in Jakobson is less in Hjelmslev ( 1935 37), a slightly rigorous proposal developed ( 1936 ) . While thesetheories are concernedwith caseonly, more recent proposals of ) extend so-called cognitive grammar as put forward , for example , in Langacker ( 1987 case the.lo cationist to I will restrict . in to considerations myself syntax general spatial structural to the related of the properties varying range phenomena theory. To cover of case , an extremely abstract construal of spacemust be assumedthat has little , if any, connection to spatial cognition as sketched in section 2.4. Spatial structure is thereby turned into a completely generalsystemof formal distinctions that makesthe explanation either vacuous or circular. Even more crucially , the way in which caseis

? How Much SpaceGets into Language

related to spatial conditions is notoriously opaque and indirect. In many languages caseis involved in the distinction betweenplace and direction , as mentioned above (seeappendix for illustration ) . On the other hand, the dative/ accusativecontrast of /e (into the /e (in the school) versus in die Schu German for example , in de, Schu form of locative and connected to the semantic formal condition school), is a purely directional in, respectively ; it does not by itself expresslocation or direction. This is " borne out by the fact that " zur Schule (to the school) requires the dative, even though it is directional. The conclusion to be drawn here has already been stated. Cases , and person, and morphological categoriesin general , like number, gender, tense are elements of the computational structure that may correspond to conceptual distinctions, but that do not in general representthose distinctions directly . In other words, spatial distinctions as representedin SF can correspond to elements of grammatical form , as should be expected , but are clearly to be distinguished from them. The secondposition , which is in a way the opposite of the first one, is advocated by Jackendoff (chapter I , this volume) . Comparing two options with regard to the , Jackendoff argues that axial systemsand the pertinent frames of encoding of space referenceare representedin spatial representation but generally not in conceptual structure. The claim, presumably, applies to spatial structure in general. It is basedon the following consideration. A clear indication for the conceptual encoding of a given distinction is the effect it has on grammatical structure. As a casein point , Jackendoff for morphosynnotes the count-mass distinction , which has obvious consequences tactic categories in English. That comparable effects are missing for practically all spatial distinctions, at least in English, is then taken as an indication that they are not . I agree with representedin conceptual structure, but only in spatial representation of the pertinent Jackendoff in assumingthat grammatical effectsindicate the presence distinctions in conceptual structure. But it seemsto me that the conclusion is the opposite becausethe major spatial patterns are no less accessiblefor grammatical effectsthan conceptual distinctions related to person, number, gender, tense , definiteness discussed . the mass distinction Given or the count , shape may , provisos just ; and sizemay correspond ; location may correspond to notional case correspond to classifiers to degreeand constructions like comparative, equative, and so on. Whether and which spatial distinctions are taken up explicitly by elementsof semantic form and whether these correspond, furthermore , to effects in computational aspectsof I -language , is a matter of languageparticular variation . English keepsmost of them . But this does not mean that they are excluded within the limits of lexical semantics from grammatical effectsin other languages , nor that they are excludedfrom conceptual . and semanticrepresentationsof English expressions

Manfred Bierwiscb

2.8

Conclusion

The overall view of how language accommodatesspace that emergesfrom these considerationsmight be summarizedas follows: I . Spatial cognition or I -spacecan be considered a representational domain within the overall systemof C-I of conceptual and intentional structure integrating various perceptual and motoric modalities. 2. Representationsof I -spacemust be integrated into propositional representations of conceptual structure, where in particular shape , size , and location of objects and the situations in which they are involved will be combined with other aspectsof commonsenseknowledge. Conceptual representation of spatial structure provides, among other things, more abstract schemataspecifying the dimensionality of objects and situations, the axesand frames of referenceof their location , and metrical scales with respectto which sizeis determined. 3. Linguistic knowledge or I -languageinterfaces with conceptual structure, recruiting configurations of it by basic components of semantic form , where strictly spatial conceptsare to be identified as configurations that interpret elementsof SF by exclusively spatial conditions on objects and situations. 4. Spatial information " visible" in I -language is thus restricted to strictly spatial , all other spatial information being supplied conceptsand their combinatorial effects by representations of ~ -I and the commonsense knowledge on which they are . based 5. The computational categoriesof I -language , which map semanticform onto phonetic form , seemto fall into two types: syntactic categories , which serve the exclusively , which , and morphological categories computational conditions of I -language in SF to in or less more (or PF configurations transparent ways may correspond for that matter) . The distinction between these two types of categories varies for obvious reasons , depending on the systematicity of the correspondencein question. Thus tense , person, and number are usually more transparent than (abstract) case or infinite categoriesof verbs. Categoriesof the combinatorial system , however transparent their correspondencemight be to elements of the interfaces of I -language with other mental systems , are neverthelesscomponents of the formal structure of I language . With all the provisos required by the wide range of unsolved or even untouched problems, the question raised initially might be answeredas follows: I -spaceis accommodatedby semanticform in terms of primitives interpreted by . strictly spatial concepts

~ How Much SpaceGets into Language

Appendix
In what follows , I will illustrate the types of questions that arise with respect to the program sketched in section 2.6 by looking somewhat more closely at locative prepositions and dimensional adjectives , relating to place and shape, respectively .

Locative Prepositiol W To begin with , I will consider a general schema that covers a wide range of phenomenashowing up within the systemof locative propositions. By meansof the notational conventions introduced in ( 18 ) and (31) above, the lexical entry for the follows: as be stated in can preposition (37) / in / [ - V , - N , . . .] .i (j ) [x [LOC (INT y])] I [ + Obj] ), the ) and Wunderlich ( 1991 , based on Bierwisch ( 1988 According to this analysis , including the relation LOC semanticform of in is composedof a number of elements and the functor INT , which specifiesthe interior of its argument. In other words, instead of a simple relation IN , we assumea compositional structure, which I will . now motivate by a number of comments

Icture Intuitively Stn a IM I Argument Variables , SF(le) of in (andin fact of prepositions andthe relatum and , respec x entities two relates in general y , identifyingthe theme ) that is to be checked by a complement specified tively. The relatumy is syntactically : a complement of that(38 . Suppose case for objective representation such ) is a simplified DEF Ui[GARDEN ] Ui [DP, + Obj, . . .] I ) Ithe garden (38 interpretation of the noungarden the SF constants , whose conceptual GARDEN abbreviates , DEF indicates includes objectschema , a two dimensional , amongother things thePP with 38 37 the . realized thedefiniteness ( ) yields ) ( Combining by operator : 38 saturated 37 is of in (39 by ( ) position ( ) ), wherethe objectargument i. [DEF Ui ] ]]]] ] : [x [LOC [INT Ui , . . .] ) fin the gardenl [ pP (39 eitherby the head The remainingargument positioni. of this PP is to be saturated ofa the modifiedby thePP subject copulathat takesthe ), or by , asin (40a ) and (40b PPaspredicate , asin (4Oc ): (40) a. the manin the garden . b. Themanis waitingin thegarden . c. The manis in the garden

Manfred Bierwisch

The main point to be noted here is the way in which the saturation of argument positions imposes conditions on the variables provided by the lexical SF(/e) of in. I will take up the consequences of this point shortly . A final remark on the argument positions of in concerns the optionality of its object, indicated by bracketing y in (37) . It accounts for the intransitive use in cases like (41), where y is left as a free variable in SF(/e) and will be specified by default conditions applying in C-I without conditions from SF. (41) He is not in today . SemanticPrimes The variablesx and y in (37) are related by the constants LOC and INT . Both are explicitly spatial in the sense that they identify conceptualcomponents that representsimple (possibly primitive ) spatial conditions. The interpretation of in can thus be stated more preciselyas follows: (42) a. x LO Cp identifies the condition that the location of x be (improperly) included in p identifies a location determined by the boundaries of y, that is, the bINTy interior of y Three commentsare to be made with respectto this analysis. First , additional conditions applying to x and y will affect how LOC and INT are interpreted in C-I . Relevantconditions include in particular the dimensionality of the object schemaconceptually imposed on x and y , alongside with further conceptual knowledge. Thus the actual location of the theme in (43b) would rather be expressed by underif it were identical to that in (43a) : (43) a. The fish is in the water. b. The boat is in the water. A similar casein point is the following contrast: (44) a. He has a strawberry in his mouth. b. He has a pipe in his mouth. Both " water" and " mouth " are associatedwith a three-dimensional object schemain (43a) and (44a) but conceptualizedas belonging to a two-dimensional surfacein (43b) and (44b) . Knowledge about fishes , boats, fruits , and pipes supports the different construal of both INT and LOC . Somewhatdifferent factors apply to the following cases : (45) a. There are somecoins in the purse. b. There is a hole in the purse.

? Getsinto Language How Much Space

67

In (45a) purse relies on the object schema of a container; in (45b) the conditions . Notice that in (45) it is only the coming from hole enforce the substanceschema interpretation of INT that varies, while in (43) and (44) the inclusion determined by LOC differs accordingly. The differencesresulting from theme or relatum may enter , but (47b) does . Thus from (45a) and (46) the conclusion (47a) derives into inferences not follow from (45b) and (46) : . (46) The purse is in my briefcase . (47) a. There are somecoins in my briefcase . b. There is a hole in my briefcase I do not think that water, mouth, purse are lexically ambiguous; although the way in which conceptual knowledge creates the differences in question is by no means a , there is no reason to assumethat trivial issue , it must be left aside here. In any case in is ambiguous between(37) and someother lexical SF(/e) . The different interpretations illustrated by (42)- (47), to which further variants could easily be added, are due to conditions of I -spaceand conceptual knowledge not reflectedin the lexical SF(/e) of in. Second , the conditions identified by LOC and INT are subject to implicit transfer : to domains other than I -space (48) a. He came in November. b. severalstepsin the calculation . c. The argument applies only in this case dreadings in linguistics e. He lost his position in the bank. Again , the specification of the theme and/ or the relatum provides the conditions on which LOC and INT are interpreted. Examples like those in (48) indicate, however, that the notion of BST crucially dependson how implicit transfer of spatial structures , but is construed. In one possibleinterpretation , in is a BST only if it relatesto I -space not if it relates (in equally literal fashion) to time or institutions . It seemsto me an important observation that in under this construal of BST is not an exclusivelyspatial term, but I do not think that this terminological issuecreatesseriousproblems. I will thus continue to use BST without additional comment. And third , the range of I -spaceconditions identified by INT dependson the distinctions a given language happens to representexplicitly in SF by distinct primes. Thus English and German, for example , contrast INT with a prime ON with roughly the following property : ON y identifies a location that has direct contact with (the designatedside of ), but does not intersect with , y .

Manfred Bierwisch

This yields the different interpretations of , for example , the nail in the table and the nail on the table- assumingthat SF(Ie) of on is [x LOC [ONy ]]- whereasin Spanish el clavo en la mesawould apply to both casesbecausethere is no in/ on contrast in , such that the surface of the table could provide the location identified by Spanish INT . The Pattern of Locative PrepositiO18 I have assumedthroughout that the categorization inherent in the primes of SF determines the compositional structure of SF . Hence the variation in patterns of according to general principles of I -language lexical representationsI will briefly look at are fully detennined by the basic elements involved. What is nevertheless of interest is the systematicityof variation theselexical exhibit. representations The first point to be noted is the obvious generalization about locative prepositions , all of which instantiate schema(49), where F is a variable ranging over functors that specify locations determined by y : (49) [x LOC [Fy]] Not only do in and on fit into (49), specifyingFby INT and ON , respectively , but also constants to other near, under at over and several , , , prepositions, using pertinent replace F. It is not obvious, however, whether schema(49) covers the full range of conditions that locative prepositions can impose. Thus Wunderlich ( 1991 ) claims that , for example , along, across , and around are more complex, introducing an additional condition , as illustrated in (62) : (y) .i [[x LOC [PROX yll : [x PARALLEL (50) jalongj [ - V , - N , . . .] [MAX y]]] PROXy and MAX y detennine the proximal environment and the maximal extension . If this is correct, the generalschemaof locative prepositions is of y, respectively (51) instead of (49) : (51) [[x LOC [Fyll : [xCyll where C is a condition on x and y , as exemplified in (50), all of which must Cmight be a configuration of basic elements have a direct, explicit spatial interpretation , in order to keep to the limits of BST. Another systematicaspectof locative prepositions concernstheir relation to directional counterparts, as shown for English and German examplesin (52) : (52) a. They were in the school. They went into the school. . . Sle gingen in die Schule Sle waren in der Schule was under the table. The ball rolled under the table. b. The ball Der Ball rolite unter den Tisch. Der Ball war unter DernTisch.

? How Much SpaceGets into Language

, the directional preposition identifies a path whose end is specified by Semantically the corresponding locative preposition. Let CHANGE p be an operator that turns the proposition p into the terminal state of changeor path . The general schemaof a standard directional preposition would then be (53) : (53) CHANGE [[x LOC [Fy]] : [xCy]] where CHANGE [ . . . ] identifies a transition whose final state is specifiedby [ . . . ] The relevant observation in the presentcontext is the systematicstatus of CHANGE in lexical structure. Besidesmere optionality in caseslike under , which , over, behind can be used as locative or directional prepositions, the occurrence of CHANGE is connected to - to in onto, into. In languages like Russian , German, and Latin is CHANGE with appropriate morphological case , , largely related to accusative introduced devices . notational of the to be checkedby the object preposition Using in phonology, the relation in question can be expressedas in (54) for German in: (54) lint Dir ] [ - V , - N , IX y .i [ < CHANGE ) [x LOC [INT yll ] I Obl ] [ - IX
This means that in is either directional , assigns - oblique case and contains the CHANGE component , or it is locative , assigns + oblique case and does not contain CHANGE .

Typological Variation Thus far , the generalpatterns of prepositions have beenconsidered as the frame by which lexical knowledge of a given language is organized. Crosslinguistic comparison revealsvariations of a different sort, one of which concerns " what might be called " lexical packaging, that is, the way components of basic schema(49) are realized by separateformatives. A straightforward alternative is found , for example , in Korean , as can be seen in (55) , taken from Wunderlich ( 1991 ): ' - ta - (ui )- ui - e iss kkotpyong i (55) Ch aeksang there Pres be Nom Gen top Loc vase desk ' There is a vaseon the desk.' ' The relatum ch aeksang(optionally marked for genitive) functions as complement of the noun ui, which identifies the top or surface of its argument and provides the complement of the locative element e. In other words, LOC and F of (49) are realized by separateitems with roughly the entries in (56), yielding (57) :

Manfred

Bierwisch

(56 ) a. Iwuil

[ + N , . . . , L]

-OF x] X [ TOP I ) (Gen N i [zLOC [ N ]]

b. lei

[ - V, - N , . . .]

I [L] ' -OF [ DESK i [z LOC [ TOP (57 ) ch aeksang (ui)wui-e [ - N , - V , . . .] ]]] The details ad hoc , includingthe featureL of the noun wui, are somewhat , but the main point shouldbe clearenough : e and wui combineto createa structurethat is closelyrelatedto the SF of Englishon or Germanauf A differenttype of packaging for locativeconstructions is found in Tzeltal and other Mayan languages . Like Korean Tzeltal has a , , completelyunspecific general locativeparticle as ta; additionalspecification doesnot come , realized , however , by nominaltermsidentifyingpartsor aspects of the relatum , but ratherin termsof positional - somewhat information , that indicatemainly positionaland shape adjectives like sit, stant , but with a remark !, lie in English ably more differentiated variety of . 51 form : Levinson 1990 ( ) givesexamples specifications ( ) ' ' ' te k ib (58 ) a. Waxal ta ch uj te -jar . Loc wood the water upright plank ' Thewater is ' on the jar standing plank. ' b. Nujul boch ta te k ib -down gourd -bowl Loc the water -jar upside 'The -jar .' down on the water gourdis upside Waxalandnujulbelongto about250positionals , derivingfrom some70 roots representing andpositionalcharacteristics Brown 1994 for discussion shape (see ). A highly provisionalindicationof waxaland the only locativeprepositionta would look like (59 ): x [ UPRIGHTCYLINDRIC x] ) a. Iwaxall (59 [ + N , + V . .] b. ltal [ - N, - V ] y i [z LOC [ ENVy]] ENV abbreviates an indicationof any (proximal . The PP ta ch'uj te' in ) environment asan adjunctwith the predicate waxalas shownin (60 ) combines ), which then (77a to the NP te k 'ib, to yield(58a applies ): ' ' x [[UPRIGHT CYLINDRIC x] : (60 ) waxalta ch uj te [ + N , + V , . . .] PLANK ]]]] [x LOC [ENV [ WOOD

How Much SpaceGets into Language ? Although various details are in need of clarification , the relevant issue- the type of packaging of SF material - seems to be perspicuous . I will not go into further typo logical variations related to the way in which general principles of semantic form accommodate locational information in basic spatial terms of different languages, but rather will take a look at issues that arise with respect to terms encoding aspects of explicit shape information .
Dime _ Here Based onal Adjectives briefly add some points to the analysis of DAs sketched in section 2 .6 . of / ong given

I will

on the analysis

in ( 31 ) and

repeated

here as ( 61 ) :

( 61) /long /

Adj

x ( j ) [ [QUANT

[MAX x ]] = [ v + y]]

I
Deg
I will
some

keep
of the

to the same sort


points have

of comments
been taken

given
up

with
above

respect
.

to prepositions

, although

already

Variables that
an

and

Argument adjectives
) x to

Structure in English
optional

As

already

mentioned

dimensional
object ( or event

are syntactically
complement of the

, ( 61 ) express es the fact two - place predicates , relating


DA that specifies adegreey ,

an

realized in ( 63 ) :

by appropriate

measure

phrases

, as in ( 62 ) , or more

complex

expressions

as

( 62) a. a six - foot - long desk


b . The
c . His

field
speech

is 60 yards
was only

long
fifteen

and

30 yards
long

wide .
.

minutes

( 63) a. The car is just as long as the garage .


b . The
c . The

stick

is long
is

enough
twice as

to touch
long as

the ceiling
the sonata

.
.

symphony

A particular and the

point

that

distinguish that

es DAs

from to

locative

Ps concerns earlier

the

variable to this

particular , DAs

conditions

variable like
too

are semantically . This


make

apply three - place in fact

it , as mentioned , rather than

. Due

relations when
to

two - place morphology


:

relations or the

prepositions
construction

becomes
the variable

visible

comparative

accessible

syntactic

specification

( 64) a. John is two feet taller than Bill .


b . The car is two In a way , than v under Bill feet too long for this garage . that explicitly specify the

and for

this garage syntactic

are complements .

variable

particular

conditions

SemanticPrimes The variables x , y, and v are related in (61) by meansof the four constants QUANT , MAX , = , and + , of which only MAX has a specifically spatial interpretation , identifying the maximal dimension with respect to the shape of y , while QUANT , = , and + identify quasi-arithmetical operations underlying quantitative , scalarevaluations quite generally. More specifically , [QUANT Y ] is a function that maps arbitrary dimensions Y on an appropriate abstract scale , and = and + have the usual arithmetical interpretation with respect to scalar values . In other words, long is a spatial term only insofar as MAX determinesdimensional conditions that rely on shapeand size of objects or events ; the shapeand the size information in contained long and short are defined by MAX , on the one hand, and by QUANT , = , and + or - , on the other. Hence semantically, shapeand sizeare interlocked in ways that differ remarkably from their interpretation in SR. Also , the quantitative conditions may carry over to various other domains: old and young are strictly temporal ; heavyand light are gravitational ; and so forth . The Pattern of DimensionalAdjectives The characteristicproperties of D As show up more clearly if we look at the general schemaof their SF, which automatically accounts for the fact that they usually come in antonymous pairs as already noted: (65) [[QUANT [DIM y]] = [v :t x ]] The secondpoint of variability in (65) besidesthe :t alternation is indicated by DIM , which marks the position for different dimensional components . Where long/ short pick out the maximal dimension, high/ low pick out the actually vertical axis by means of VERT , and tall combines both MAX and VERT . As a matter of fact, the constants replacing the variable DIM in (65) turn an adjective into a spatial term like tall or thin, a temporal term like young or late, a term qualifying movement, like fast and slow, and so forth . It might be noted that the interpretation of the different dimensional constants requires the projection of an appropriate object schemaon the term providing the value for x : a tall sculptureinducesa schemawhosemaximal dimension is vertical for , which does not provide this condition by itself. As ball would not allow sculpture for a schemaof this sort, a tall ball is deviant. For details of this mechanismseeLang ( 1989 ). . Typological Variation Thus far , we have consideredvariation within schema(65) . I will now indicate someof the possibilities to modify the schemaitself in various ways. An apparently simple modification is shown by languageslike Russian, which do not allow measurephraseswith DAs . 10 m long could not come out 10 m dlinnij ; measure phrasescan only be combined with the respectivenouns, that is, by constructions like

? Getsinto Language How Much Space

73

dlinna 10 metrov, corresponding to length ofmeters . This suggeststhat Russian DAs do not have a syntactic argument position for degreecomplements , preserving otherwise schema84. Things seemto be a bit more complicated, though: measure with comparativesare possible , although only in termsof prepositional phrases phrases translates into the adjectival construction na 2 m with na. 2 m longer, for example , dlinnej. I cannot go into the details of this matter. We have already seena much more radical variation of schema(65), exemplifiedby . Here, not only the degreeargument position is dropped, Tzeltal positional adjectives but the whole quantificational component, retaining only [ DIM x ], but supplying it with a much more detailed system of specifications , as indicated provisionally in (59a ) . This is not merely a matter of quantity ; rather, it attests a different strategy to recruit conditions on shape and position of objects. Where the twenty-odd DAs of most Indo -European languages rely on object schemata in a rather abstract and indirect way, the positional adjectivesof Tzeltal include fairly specific , strictly spatial specificationsof objects to which they apply . Although organizing principles and actual details of Tzeltal positional adjectives remain to be explored, rather subtle, but clear distinctions determining alternativesin DAs of German, Russian, Chinese , and Korean have been isolated in Lang ( 1995 ). , while Object schematain Chinese seemto be based on proportion of dimensions Korean takes observer orientation as prominent ; a similar preferencedistinguishes German and Russian. Let me summarizethe main points of this rather provisional sketch of basic spatial terms. First , among the entries of the core lexical system of I -language , there is a illustrated in section 2.5. Their subsystemof items that are strictly spatial in the sense semanticform [SF(/e)] consistsexclusivelyof primes that are explicitly interpreted in . Even though the delimitation of this subsystemis terms of conditions of I -space subject to intervening factors, suchas implicit or explicit transfer of interpretation , its . elementsplaya theoretically relevant role for the linguistic representationof space with respectto the linguistic properties Second , there are characteristic consequences of theseitems, as shown by the appearanceof degreephrases , and argument structure of theseterms must be structure of the SF more generally. Hence the compositional assumedto belong to I language , their basic elementsbeing components of a representational aspectdetermined by VG . Finally , there is remarkably systematicvariation among different languageswith respectto both the choice of basic distinctions recruited for lexicalization and the different types of packaging according to more , then, the analysis of basic spatial terms, even though it general patterns. In general could be illustrated only by two types of cases , promises to give us a more detailed much into . of how ) spacegets ( language understanding

Manfred Bierwisch Acknowledgments The presentchapter benefitsfrom discussionsat various occasions . Besidesthe membersof the Max Planck ResearchGroup on Structural Grammar , I am indebted to the participants of the project on Spatial and Temporal Referenceat the Max Planck Institute for Psycholinguistics ; further discussions included Dieter Gasde , Paul Kiparsky , Ewald Lang, StephenLevinson, and Dieter Wunderlich. Particular debts are due to Ray Jackendoff, whose stimulating proposals are visible throughout the paper, even if I do not agreewith him in certain respects . Notes I . This view is in line with fundamental developmentsin recent linguistic theory, including the minimalist program proposed in Chomsky ( 1993 ) . Although it is still compatible with the possibility of parametric variation regarding the way options provided by specification 2 are exploited in individual languages , this sort of parametric variation should be considered as bound to lexical information , and thus ultimately to the choice of primitives in the senseof specification I . I will examine more concrete possibilities along these lines in section 2.6. 2. This doesnot necessarily , stipulating LF in imply a proliferation of levelsof representations addition to SF. One might in fact consider LF a systematiccategorization imposed on SF, just as PF must be subject to certain aspectsof syntactic structure. " " 3. Even though Chomsky ( 1993 ) refersto APand C-I occasionallyas perfonnance systems , it should be clear that they must be construed as computational systemswith their own specific representationalproperties. 4. It should be noted that Jackendoff considers the structure (.i.e., PF) phonological . as beto I he , although longing properly language recognizesthe need for correspondencerules connecting it to articulation and perception. " " 5. Thus, in order to honor Schonberg , Alban Berg in his Lyrische Suite introduces a theme that consists of the notes es ( = e ftat)-c h ( = b)-e-g, representing all and only the letters in Schonbergcorresponding to the German rendering of notes. " in the intended senseis the 6. A very special " interface representation systemof numbering ' s famous used in G Odel of the proof incompletenessof arithmetic , where numbers are given two mutually exclusivesystematicinterpretations, one stating properties of the other. References Berlin, B., and Kay , P. ( 1969 : University of Call fomi a Press . ) . Basic color terms. Berkeley Biedennann, I . ( 1987 : A theory of human image understanding. ) . Recognition- by- components - 147 . , 94, 115 PsychologicalReview

- 100 -Verlag 61 . , Berlin , Akademie

Bierwisch mantis cheund konzeptuelle lexikalischer . Einheiten , M. ( 1983 ). Se Reprasentation In R. RuzickaandW. Motsch(cds .), Untersuchungen zur Semantik : StudioGrammatico XXlI ,

? How Much SpaceGets into Language Bierwisch, M . ( 1988 ) . On the grammar ofloca1 prepositions. In M . Bierwisch, W. Motsch, and , I . Zimmennann (Eds.), Syntax, Semantik, und Lexikon: Rudolf Ruzicka zum 65. Geburtstag 1- 65. Berlin: Akademie-Verlag. Bierwisch, M . ( 1989 ) . The semanticsof gradation . In M . Bierwisch and E. Lang (Eds.), Dimensional : Grammatical structure and conceptual interpretation, 71- 261. Heidelberg, adjectives

. NewYork: Springer

- further and further. - muchdeeper Bierwisch , E. ( 1989 longer , M., and Lang ). Somewhat andconceptual structure : Grammatical .), Dimensional and Lang (Eds In Bierwisch adjectives - 514 . . Heidelberg , NewYork: Springer , 471 interpretation of static : The semantics Brown , P. ( 1994 ). The INS and ONS of Tzeltallocativeexpressions . 743 790 32 . location of , , Linguistics descriptions . Journalof Memoryand Laird Johnson and R. M. J. , P. N. ( 1989 ). Spatialreasoning , , Byrne . 575 28 564 , , Language . . NewYork: ColumbiaUniversityPress andrepresentations , N. ( 1980 ). Rules Chomsky : Foris. . Dordrecht andbinding ongovernment , N. ( 1981 ). Lectures Chomsky . . NewYork: Praeger : Its nature , anduse , origin , N. ( 1986 of language ). Knowledge Chomsky . In K. Hale and S. J. Keyser , N. ( 1993 ). A minimalistprogramfor linguistictheory Chomsky : Theview in honorof SyvianBromberger in linguistics .), Essays from Building20, I - 52. (Eds . , MA: MIT Press Cambridge . International sorts , and systematic ambiguity , semantic ). Ontologicaldomains Dolling, J. ( 1995 - 807 -Computer . Studies Journalof Human , 43, 785 . : Reidel . Dordrecht andMontague grammar ). Wordmeaning Dowty, D. R. ( 1979 . . NewYork: Cromwell Fodor, J. A. ( 1975 of thought ). Thelanguage . . Cambridge , MA: MIT Press Fodor, J. A. ( 1983 of mind ). Themodularity . : North Holland . Amsterdam in lexicalrelations Gruber , J. S. ( 1976 ). Studies and the lexicalexpression structure Hale ofsyntac , S. J. ( 1993 ). On argument , K., andKeyser : in honorof SylvianBromberger in linguistics .), Essays . In Hale and Keyser(Eds tic relations ~ 109 . . Cambridge 20 Theview , MA: MIT Press , 53 from Building - 37 . . Arhus: Universitetsforlaget descas , L. ( 1935 ). La categorie Hjelmslev . . Cambridge andcognition Jackendoff , MA: MIT Press , R. ( 1983 ). Semantics . . Cambridge mind andthecomputational , MA: MIT Press Jackendoff , R. ( 1987 ). Consciousness . . Cambridge structures Jackendoff , MA : MIT Press , R. ( 1990 ). Semantic of the : Generalmeanings Jakobson , R. ( 1936 ). Contribution to the generaltheory of case - 103 - 1981 . : 1931 and Slavicgrammarstudies . In R. Jakobson , 59 Russiancases , Russian : Gesamtbe Casuslehre : Beitragzur allgemeinen Berlin , NewYork: Mouton. (Originalversion . Selected Kasus der russischen , 23- 71.) , Vol. 11 Writings deutungen

-Laird, P. N. ( 1983 : Towards a cognitive Johnson science , inference , ). Mentalmodels of language . Cambridge and consciousness : CambridgeUniversity Press ; Cambridge , MA : Harvard . UniversityPress to logic . Dordrecht : Kluwer. , H., and Reyle , U. ( 1993 ). Fromdiscourse Kamp Katz, J. J. ( 1972 . NewYork: Harperand Row. ). Semantic theory Keil, F. C. ( 1987 and category structure . In U. Neisser ). Conceptual (Ed.), Concepts development - 200 andconceptual . Cambridge : Cambridge . , 175 development UniversityPress ' in theminds machine . NewYork: Norton. , S. M. ( 1983 Kosslyn ). Ghosts , S. M., Holtzmann , J. D., Farah , M. J., and Gazzaniga , MS . ( 1985 Kosslyn ). A computa tional analysis of mentalimagegeneration : Evidence from functionaldissociations in split - 341 brain patients . Journalof Experimental : General . , 114 , 311 Psychology of dimensional of spatialobjects . In M. Bierwisch , E. ( 1989 ). The semantics Lang designation and E. Lang (Eds .), Dimensional : Grammatical structure andconceptual adjectives interpretation - 417 . Heidelberg . , 263 , NewYork: Springer : A first look at universalfeatures and typological , E. ( 1995 ). Basicdimensionterms Lang - 100 -Papers variation . FAS in Linguistics . , 1, 66 - 94. . Language , R. W. ( 1987 , 63, 53 ). Nounsandverbs Langacker Levinson . Paper delivered to the , S. C. ( 1990 ). Figureandgroundin Mayanspatialdescription conference Time . Nijmegen : Max PlanckInstitutefor Psycholinguistics , Space , and the Lexicon . , November Marr, D. ( 1981 . SanFrancisco : Freeman . ). Vision Moravcsik ?Journalof Philosophy , J. M. E. ( 1981 , 78 , 5- 24. ). How do wordsgettheirmeanings - 441 . lexicon . Computational , J. ( 1991 , 17 , 409 Pustejovsky ). Thegenerative Linguistics von Stechow in syntax . In E. Urs et al. (Eds .), Thelexicon , A. ( 1995 ). Lexicaldecomposition - 117 in the organization : Selected . , 81 of language papers from 1991Konstanz Conference Amsterdam : Benjarnins . fit into compositionalsyntax and Wunderlich , D. ( 1991 ). How do prepositionalphrases - 621 ? Linguistics semantics . , 29, 591

Chapter

3 Taking and Ellipsis in Spatial Descriptions

Perspective

Willem J. M. Levelt

3.1 Thinkingfor Speaking


There exists happy agreementamong students of languageproduction that speaking normally involves a stageof conceptual preparation. Depending on the communicative . Ideally, this situation , we decide in some way or another on what to express intention choice of content will eventually make our communicative recognizable to our audience or interlocutor . The result of conceptual preparation is technically termed a message ); it is the conceptual entity the speakerwill (or a string of messages formulate. in that is , , eventually express language But there is more to conceptual preparation than considering what to say, or has to be of a particular . There is also microplanning. The message macroplanning kind ; it has to be tuned to the target languageand to the momentary informational . This chapter is about an aspect of microplanning that is of needsof the addressee , namelyperspectivetaking. paramount importance for spatial discourse In an effort to cope with the alarming complexities of conceptual preparation, I ) that is reproduced here as figure 3.1. presenteda figure in my book Speaking( 1989 must be in some kind of propositional It is intended to expressthe claim that messages or " algebraic" format (cf. Jackendoff, chapter 1, this volume) to be suitable for formulation . In particular , they must be composed out of lexical concepts , that is, ' . An concepts for which there are words or morphemes in the speakers language immediatecorollary of this notion is that conceptualpreparation will , to someextent, . . Lexical conceptsdiffer from languageto language be specificto the target language will in and therefore another be nonlexical A lexical concept in one language may . To give one spatial example (from to be expressed need a slightly different message that treat deictic proximity Levelt 1989 ), there are languagessuch as Spanishor Japanese in a tripartite way: proximal -medial-distal. Other languages , such as English or of use distal. Dutch , have a bipartite system , proximal aqui-ahi-alli requires Spanish -there. to construe distance from speakerin a different way than English use of here

sem rep I Cn ~ . 1 nA J Y ' \ D I ~ FO ( p re ) me


Willem J. M. Levelt
Figure 3.1 The mind harbors multiple representationalsystemsthat can mutually interact. But to formulate " " , propositional any representation linguistically requires its translation into a semantic code (reproduced from Levelt 1989 ).

" " Slobin (1987 , which is an elegant ) has usefully called this thinking for speaking synonym for microplanning . Thinking for speaking is always involved when we expressnonpropositional, in particular spatial, information . Figure 3.1 depicts the notion that when we talk about our spatial, kinesthetic, musical, and so on experiences , we cast them in propositional form . This ne" ....essarilyrequires an act of abstraction. When talking about a visual scene , for instance , we attend to entities that are relevant to the communicative task at hand, and generatepredications about theseentities that accurately capture their . This processof abstracting from the visual scenefor spatial relations within the scene " " I will call speaking perspectivetaking . Although this term will in the presentchapter be restricted to its original spatial domain, it is easily and fruitfully generalizedto other domains of discourse(cf. Levelt 1989 ).

3.2 Perspective Tsking


Perspectivetaking as a processof abstracting spatial relations for expressionin language typically involves the following operations: I . Focusing on some portion of the scenewhose spatial disposition (place , path, " " orientation ) is to be expressed ( Talmy 1983 ) . I will call this portion the referent. 2. Focusing on some portion of the field with respectto which the referent' s spatial . I will call this portion the " relatum." disposition is to be expressed 3. Spatially relating the referent to the relatum (or expressingthe referent' s path or orientation ) in terms of what I will call a " perspectivesystem ."

Perspective Taking and Ellipsis in Spatial Descriptions

3.2 This spatial array can be described in myriad ways , depending on the choice of referent, relatum. and perspective .

FigurE

Let me exemplify this by meansof figure 3.2. One way of describing this sceneis ( I ) I seea chair and a ball to the right of it . Here the speakerintroduces the chair as the relatum and then express es the spatial disposition of the ball (to the right of the chair) . Hence, the ball is the referent. The perspectivesystemin terms of which the relating is done is the deictic system , that is, a speaker - centeredrelative system . ! When you focus on the relatum (the chair), your gazemust turn to your right in order to focus on the referent (the ball ) . That is why the ball is to the right of the chair in this system . Two things are worth noticing now. First , you can swap relatum and referent, as in (2) : (2) I seea ball and a chair to the left of it . This is an equally valid description of the scene ; it is only a less preferred one. tend to select smaller and more Speakers foregrounded objects as referentsand larger or more backgroundedentities as relata. Here they tend to follow the Gestalt organization of the scene( Levelt 1989 . , you can take another perspectivesystem ) . Second You can also describethe sceneas (3) : (3) I seea chair and a ball to its left. This description is valid in the intrinsic perspectivesystem . Here the referent' s location ' is expressed in terms the relatum s intrinsic axes . A chair has a front and a back, a left and a right side. The ball in figure 3.2 is at the chair' s left side, no matter from which viewpoint the speakeris observing the scene . Still another perspectivesystem allows for the description in (4) :

Willem J. M. Levelt
(4) I seea chair and a ball north of it . This description is valid if indeed ball and chair are aligned on a north -south dimension ' . This is termed an absolutesystem ; it is neither relative to the speakers nor to the relatum' s coordinate system , but rather to a fixed bearing. The implication of thesetwo observations is that perspectiveis linguistically free. There is no unique way of perspectivetaking . There is no biologically determined one-to-one mapping of spatial relations in a visual sceneto semantic relations in a . And cultures have taken different options here, as linguistic description of that scene Levinson and Brown have demonstrated ( Levinson 1992a ,b; Brown and Levinson . 1993 of Yimithirr are exclusive users of an absolute perspective ) Speakers Guugu are exclusive users of an intrinsic , Mopan speakers , Tzeltal usesa mix system system of absolute and intrinsic perspectives . Similarly, ,a .nd English usesall three systems there are personal style differencesbetween speakersof the same language . Levelt 1982b found that on the same task some use a deictic , , ( ) speakersconsistently system whereas others consistently use an intrinsic perspective system . Finally , the same speaker may prefer one system for one purpose and another system for another purpose as Tversky ( 1991 ) and Herrmann and Grabowski ( 1994 ) have shown. This freedom of perspectivetaking does not mean, however, that the choice of a perspectivesystem is arbitrary . Each perspectivesystem has its specific advantages ' ' and disadvantagesin language use , and these will affect a culture s or a speakers choice. In other words, there is a pragmatics of perspectivesystems . In the rest of this chapter I will addresstwo issues . The first one is pragmatics. I will compare some advantagesand disadvantagesin using the three systemsintroduced above; the deictic, the intrinsic , and the absolute systems . In particular , I will ask how suitable thesesystemsare for spatial reasoning , how hard or easythey are to align betweeninterlocutors , and to what extent the systemsare mutually interactive. The secondissuegoesback to figure 3.1 and to " thinking for speaking ." I defined ' perspectivetaking as a speakers mapping of a spatial representationonto a propositional .A (or semantic ) representation for the purpose of expressingit in language crucially important question now is whether the spatial representationsthemselves are already " tuned to language ." For instance , a speakerof Guugu Yimithirr , who uses absolute well have , may exclusively perspective developedthe habit of representing any spatial state of affairs in an oriented way, whether for languageor not. After all , any spatial scenemay becomethe topic of discourseat a different place and time. ' s absolute orientation . Levinson The speakershould then have rememberedthe scene . On the other ( 1992b ) presents experimental evidence that this is indeed the case hand, I argued above that perspectiveis free. A speaker is not " at the mercy" of a spatial representation in thinking for speaking . In the strongest non-Whorfian

Taking and Ellipsis in Spatial Descriptions Perspective case, spatial representations will be language - independent , and it is perspective taking that maps them onto language specific semantic representations . One way of how speakers operate when they produce spatial ellipsis sorting this out is to study (such as in go right to blue and then 0 to purple , here 0 marks the position where a second occurrence of right is elided ) . I will specifically ask whether ellipsis is generated from a perspectivized or from a perspective - free representation . If the latter turned out to be the case, that would plead for the existence of perspective - free spatial representations . 3.3 Some Properties of Deictic , Intril Bic, and Absolute Perspective

Of many aspects that may be relevant for the use of perspective systems I will discuss the following three : ( I ) their inferential potential , ( 2) their ease of coordination between interlocutors , and ( 3) their mutual support or interference .

3.3.1 Inferential Potential ; Tversky Spatial reasoning abounds in daily life (cf. Byrne and Johnson Laird 1989 search instructions road directions . 1991 , spatial , assembly equipment ) Following instructions, or being involved in spatial planning discourseall require the ability to infer spatial layouts from linguistic description. And the potential for spatial inference . In Levelt ( 1984 is crucially dependenton the perspectivesystembeing used )I ; I will analyzed some essentiallogical properties of the deictic and intrinsic systems . summarizethem here and extend the analysis to the absolute system . Perspectivesystems CoDverseness An attractive logical property is converseness , such as front -back, aboveusually (though not always) involve directional opposites below, north -south. If the two -place relation expressed by one pole is called R and the -1 if R (A , B) ~ R 1(B, A ) . For holds converseness R then the other one by , pole by instance , if object A is above object B, B will be below A . holds for the deictic system and for most cases2of the absolute Converseness . This is demonstratedin figure 3.3. Assuming not for the intrinsic system but , system that it is about noon somewherein the Northern Hemispherewith the sun shining, the shadowsof the tree and ball indicate that the ball is east of the tree. Using this absolute bearing, the tree must be west of the ball , where west is the converseof east. ' also holds for the (three-place) deictic relation. From the speakers Converseness point of view, the ball (referent) is to the right of the tree (relatum) , which necessarily implies that the tree (referent) is to the left of the ball (relatum) . But it is easy to " . The ape can be on the right side ( to the for the intrinsic system violate converseness " " " right ) of the bear at the sametime the bear is on the right side ( to the right ) of the

~
~

@
@
[ . ~

lreq

~ G
.

~~ ~ i .

Takingand Ellipsisin SpatialDescriptions Perspective

83

ape. It is therefore impossible to infer the relation betweenrelatum and referent from , which is a major the relation between referent and relatum in the intrinsic system . drawback for spatial reasoning Tra. - itivity Transitivity holds if from R (A , B) and R(B, C ), it follows that R (A , C ) . . This is the casefor the absolute and deictic systems , but not for the intrinsic system scene depicts This state of affairs is demonstratedin figure 3.4. The flag, tree, and ball " " " the transitivity of " east of in the absolute system and of to the right of in the . For the intrinsic system it is easy to construct a case that violates deictic system . The user of an intrinsic transitivity . This is the casefor the bear, cow, and ape scene systemcannot rely on transitivity . From A is to the right of B, and B is to the right of C, one cannot reliably conclude that A is to the right of C, and so forth . Henceone cannot create a chain of inference , using the previous referent as a relatum for the next one. and transitivity . Converseness Theseare seriousdrawbacks of the intrinsic system . are very desirable properties if you want to make inferencesfrom spatial premises , in following route And spatial reasoningabounds in everydaydiscourse , for instance , and so directions, in jointly planning furniture arrangementsor equipment assembly . for on. I will shortly discussfurther drawbacksof the intrinsic system spatial reasoning 3.3.2 Coordination betweenInterlocutors It is more the exception than the rule that interlocutors make explicit referenceto the and discussion , see perspectivesystemthey employ in spatial discourse(for references but not , Levelt 1989 , 51) . Usually there is tacit agreement about the system used always. An example of nonagreement turned up in an experiment where I asked subjectsto describecolored dot patterns in such a way that other subjectswould be able to draw them from the tape-recordeddescriptions. An exampleof such a pattern is presentedin figure 3.5. Subjectswere instructed to start at the arrow. It turned out . A typical deictic description of this pattern that most subjectsused deictic perspective is the following : Begin with a yellow dot . Then one step up is a greendot and further up is a brown dot . Then right to a blue dot and from there further right to a purple dot . Then one step down there is a red dot. And left of it is a black one. Although the dot pattern was always flat on the table in front of the subject, moves toward and away from the subject were typically expressedby vertical dimension , becauseit is viewerterms (up, down ) . This is characteristic for deictic perspective ; Shepardand . It essentiallytells you where the gazemoves(seeLevelt 1982b centered moves the Hurwitz 1984 up, up, right , right , gaze ) . For the pattern in figure 3.5,

~ 8uU 8uU

t8
~ ~

ai
: '

IUD

IUD

aa

R .i
@
@
~ [

JJtI

m MO MO

lI

' ' . ~ ~ . - . ~
right

andEllipsis in Spatial Perspective Taking Descriptions right

right

t
Figure 3.5 Pattern used in a spatial description task. The nodes were colored ( here replaced by color names ) . On the outside of the arcs are the dominant directional tenDS used in deictic descriptions ; on the inside, the ones useddominantly in intrinsic descriptions.

down, and left. Thesedirectional terms in the description are depicted at the exterior side of the pattern . Notice that all terms would have beendifferent if the pattern had been turned by 90 degrees . But other subjects used the intrinsic system . They described the scene as if they were moving through it or leading you through it . This is a typical intrinsic 3 description. You start at a yellow point . Then go straight to a greendot and straight again to brown. Now turn right to a blue dot and from there straight to a purple dot . From there turn right to red and again right to a black dot. There are no vertical dimension terms here. The description is not viewer-centered , but derives from the intrinsic directions of the pattern itself; the directional terms

Willem J. M . Levelt

. The interior of figure 3.5 would still be valid if the pattern were turned by 90 degrees depicts the directional terms used in this intrinsic description. When I gave the deictic descriptions to subjects for drawing, they usually reproduced ' the pattern correctly. But when I presentedthe intrinsic description, subjects drawings tended to be incorrect, and systematically so. Most reproductions are like the one in figure 3.6, which is a typical example. What has happenedhere is obvious. a deictic perspectiveand forces the intrinsic description The listener tacitly assumes into this deictic Procrustean bed. The incongruent term straight is interpreted as " ." This then is a caseof , , / hearercoordination . failing speaker up Coordination failures can be of different kinds. In this example the listener tacitly assumesone perspectivesystem where the speaker has in fact used a different one. Our deictic and intrinsic systemsare subject to this confusion becausemany of the

subjectended drawinghere (black dot)

t subjectbegan drawinghere (yellow dot) Figure3.6 ' s reconstruction . of the patternin figure3.5 from its intrinsicdescription A subject

) tjj -

~in Spatial Descriptions andEllipsl Perspective Taking


dimensional tenDs are the same or similar in the two systems . But also within the sameperspectivesystemcoordination failure can arise. For the deictic system , a major problem in coordination is that the systemderives 's ' from the speaker viewpoint, that is, the speakers position and orientation in the . And because scene the viewpoints are never fully shared , there is continuous switching back and forth in conversation betweenthe coordinate systemsof the interlocutors . The interlocutors must keep track of their partners' viewpoints throughout . spatial discourse This contrasts with the intrinsic and absolute systems , which are speaker . The intrinsic , however, requires that the interlocutor is aware of independent system the relatum' s orientation . The utterance the ball is to the right of the chair can only ' effectively localize the ball for the interlocutor if not only the chair s position is known , but also its orientation . In a perceptual scene , therefore, the intrinsic system of the relatum on the part of the listener, not only awareness of requires recognition its localization. The felicity of speaker / hearer coordination in the intrinsic system is, therefore, on the shared image of the relatum. First , coordination in the crucially dependent intrinsic systemis only possible if the relatum is oriented. Any object that does not have an intrinsic front is excluded as a basefor the front / back and left/right dimensions , frontness is an interpretative category (Miller and Johnson- Laird 1976 ) . Second , not a strictly visual one. There is no visual feature that characterizesboth the front of a chair and the front of a desk (see figure 3.7a- b) . These properties are , derived from our characteristic usesof theseobjects, and theseuses functional ones

~ left right V left right r front L ~ front 0 ? ? -v V l' CD front


Figure 3.7 The alignment of an object' s left , front , and right side does not dependon its spatial, but on its functional , properties.

Willem J. M . Levelt

can be complex. What we experienceas the front side of a church from the outside ' (figure 3.7c) is its rear or back from the inside. Still worse, the alignment of an object s front , left , and right is not fixed, but dependenton its characteristic use(compare the alignments for chair and desk in figures 3.7a and 3.7b); it may even be undetermined or ambiguous (as is the casefor the church in figure 3.7c). Not all intrinsic systemsshare all of theseproblems. Levinson ( 1992a ) was able to show that speakersof Tzeltal are much more vision-bound in deriving the intrinsic , orientation -determining parts of objects than English or Dutch , which tend to use a more functional approach. Still , the use of intrinsic perspective always requires detailed interpretation of the relatum' s shape , and this has to be shared between interlocutors. Theseproblems do not arise for the deictic and absolute systems . So far we discussed someof the coordination problems in utilizing the deictic or the intrinsic system . What about speaker / hearer coordination in terms of an absolute ? Here, the interlocutors must agree on absolute orientation , for instance on system what is north . Even if such a main direction is indicated in the landscapeas a tilt or a coastline, dead reckoning will be required if successful spatial communication is to take place in the dark , in the fog, farther away from one' s village, or inside unfamiliar ) . The only absolute dimension that is entirely unproblematic dwellings (Levinson 1992b is verticality, for which we have a designatedsensorysystem(and even this one can nowadays be tampered with ; seeFriederici and Levelt 1990for someexperimental results in outer space ) . So evenan absolute systemis not without its drawbacks in spatial communication. 3.3.3 Interaction betweenPerspective Systems When languageusershave access to more than a singleperspectivesystem , additional problems arise. A first problem already appearedin the previous section. Interlocutors must agree on a system , or must at least be aware of the system used by their in . This mechanism failed in the network description task in figure partners speech 3.6. Various factors can contribute to the establishmentof agreement . One important factor is the choice of a default solution. Depending on the communicative task at hand, interlocutors tend to opt for the same solution (Taylor and Tversky 1996 ; ' Herrmann and Grabowski 1994 ) . In addition , a speakers choice of perspectiveis often given away by the terminology typical for that perspective . When a speakeruses terms such as north or east the chosen cannot be deictic or intrinsic . And , perspective there are more subtle differences . I have mentioned the presence of vertical dimension terms in deictic directions in a horizontal plane and their total absencein intrinsic directions (the relevant data are to be found in Levelt 1982b ) . Hence, for thesedescriptions or absence of vertical dimension terms , presence givesaway which perspective . Surprisingly, the subjects in my experiment completely system is being used

I Descriptions andEllipsis in Spatial Perspective Taking


ignored this distinctive information when they drew patterns such as in figure 3.6. ' There are still other linguistic cues . When you say The chair is on Peter s left, you are , and so is the Frenchman who saysla chaiseest a definitely using the intrinsic system la gauchede ma soeur (Hill 1982 , ) or the German who utters Der Stuhl ist zu ihrer Linken (Ehrich 1982 . I am not familiar with any empirical study about the effectiveness ) 's of such linguistic cuesin transmitting the speaker perspectiveto the listener. Two problems that arise with multiple perspectives are alignment and preemption . Different perspectives mayor may not be aligned in a particular situation , and if they are not aligned, one perspective may gain (almost) full dominance, more or less . This is most easily demonstrated from the use of preempting the other perspectives vertical dimension terms, such as in A is above / below B. The basis for verticality is different in the three systemsunder consideration. In the absolute systemverticality is determined by the direction of gravity . In the intrinsic systemit is determined by the top/ bottom dimension of the relatum. In the deictic systemit is probably determined by the direction of your retinal meridian (Friederici and Levelt 1990 ) . In any perceptual situation thesethree basesof verticality mayor may not coincide. Let us consider situations where there is a ball as referent and a chair as relatum and there is an observer .4 The ball can now be abovethe chair with respectto one, two , / speaker or all three of thesebases . The eight possibilities that arise are depicted in figure 3.8.5 The appropriatenessof saying the ball is above the chair varies dramatically for the depicted speakerin the eight scenes . This we know from the work by Carlsonand Irwin 1993 who Radvansky ( ), put subjectsin the positions depicted in figure 3.8 and asked them to name the spatial relation between the referent and the relatum. Although the sceneswere formally the ones in figure 3.8, they varied widely in the " " 6 objects depicted and in backgrounds. Figure 3.8 shows the percentageof above for eachconfiguration . Clearly, absolute perspectiveis quite dominant here responses " above" cases scenes a in absolute perspective dare of absolute ( ) . But in the absence above , intrinsic above keeps having some force, whether or not it is aligned with deictic above (scenes e and g, respectively ) . Deictic abovealone, however, (scene / ) is insufficient to release" above" responses . More generally, the deictic dimension does not seem to contribute much in any combination. But further work by the same authors (Carlson- Radvansky and Irwin 1994 ), in which reaction times of judgments were measuredfor the same kind of scenes , showed that all three relevant systems contribute to the reaction times. The three systemsmutually facilitate or interfere, depending on their alignment. In addition , the reaction times roughly follow the are for abovein absolute perspective judgment data in figure 3.8. The fastest responses . , followed by intrinsic and then deictic aboveresponses These findings throw a new light on a discussion of my " principle of canonical orientation " (Levelt 1984 ) by Garnham ( 1989 ) . I had introduced that principle to

Willem J. M. Levelt WOJj PU


(

:+ II
(

~ w
(

,g

:3

";' a1n : :

Perspective

in Spatial andEllipsis Descriptions Taking


"The ball is in front of the chair."

"The ball is to the left of the chair."

. @

Figure 3.9 According to the principle of canonical orientation , the ball can be intrinsically to the left of the chair in (a) and (c), but not in ( b) . It can be intrinsically infront of the chair in (d) and (f ), but not in (e) .

Willem J. M . Levelt

account for certain cases where the intrinsic systemis " immobilized" when it conflicts with the deictic system . Because the principle is directly relevant to the presentdiscussion of alignment and preemption, I cite it here from the original paper:

The principle of canonical orientation is easily demonstratedfrom figure 3.9. Casesa, b, and c, in the left -hand side of the figure, refer to the intrinsic description the ball is to the left of the chair. According to the principle of canonical orientation this is a possibledescription in ' a ( ) . The description refers to the relatum s intrinsic left /right dimension. That dimension is in canonical orientation to the relatum' s perceptual frame. The perceptual frame for the chair ' s orientation is in this case the normal gravitational field. The chair is in canonical position with respectto this perceptual frame. In particular, the chair' s left/right dimension has a canonical direction , that is, it lays in a plane that is horizontal in the perceptual frame. However, the description is virtually impossible in (b) . Here the left /right dimension of the chair (the relatum) is not in canonical position ; it is not in a horizontal plane, given the perceptual frame. Finally and surprisingly, it is for many native speakersof English acceptableto say the ball is to the left of the chair in caseof (c) . Here the chair is not in canonical position either, but the chair' s left /right dimension is; it is in a horizontal plane of the perceptual . frame. Hence the principle of canonical orientation is satisfiedin this case The state of affairs is similar for the intrinsic description the ball is in front of the chair. This description is fine for (d ) . It is, however, virtually unacceptablefor (e), and this is because the front / back dimension of the relatum (the chair) is not in a canonical, horizontal plane with respectto the perceptual frame. Although in (/ ) the chair is not in canonical position, its front / back dimension is. Hence the description is again possibleaccording to the principle, which agreeswith intuitions of many native speakersof English to whom I showed the scene(the formal experiment has never beendone, though) . " Why does the principle refer to the perceptual frame of orientation of the referent " and not " " , just to the perceptual frame of orientation ? In figure 3.9 it is indeed impossible to distinguish betweenthesetwo. The perceptual frame of the ball is the visual sceneas a whole. Its orientation , and in particular its vertical direction , determines whether some dimension of the relatum (the chair) is in canonical position . More generally, a referent' s perceptual frame of orientation will normally be the
experienced vertical , as it derives from vestibular and visual environmental cues, and

Perspective Taking and Ellipsis in Spatial Descriptions

fly 2 , , , , , " , . r

fly 3 , , , ' , I . .

Figure3.10 to the of canonicalorientation According , fly I can be intrinsicallyto the left of ' s nose and principle 's head John , John fly 2, but not fly 3, canbeabove from Levelt 1984 (reproduced ). will be the samefor referent and relatum. But there are exceptionsin which a dominant visual Gestalt adopts the function of perceptual frame for the referent. This can happen in the sceneof figure 3.10, which is reprinted here from Levelt ( 1984 ). In that paper I argued that it is not impossible in this caseto say about fly 2 in the ' picture: there is afly aboveJohn s head even though the top/ bottom dimension of ' John s head is not in canonical orientation . And this is in agreementwith the principle . To show this, let us consider the figure in some more detail, beginning at the location of fly 1. Here John' s face is a quite dominant background pattern which may becomethe perceptual frame of orientation for the fly . In that case , the principle of canonical orientation predicts that it is appropriate to say, there is afly to the left of John's nose . This is becausethe intrinsic left /right dimension in which the fly is spatially related to John' s nose is canonically oriented with respect to the perceptual frame. It is in a plane perpendicular to the top/ bottom dimension of the face. And fly 2 may similarly take John' s face as its perceptual frame, becauseit is so close to it . If this is a subject' s experience , then it is appropriate to say there is a fly aboveJohn's head , according to the principle . The experimental findings by Carlson- Radvansky and Irwin ( 1993 ; cf. figure 3.8g) now confirm that this can indeed be the case .7 Fly 3 ' ' is further away from John s head and does not naturally take John s head as its . Hence it is less appropriate here to say it is " above" perceptual frame of reference ' John s head. Notice that in these three casesJohn' s head itself has the bed and its normal gravitational orientation as its perceptual frame. Hence the perceptual frame of the referent can be different from the larger perceptual frame in which the relatum

1 , , , fly ? " -

Willem J. M . Levelt

is embedded . In other words, there can be a hierarchy of frames, and it is not neces sanly the casethat the referent and the relatum share a frame. Garnham ( 1989 ) challenged the principle of canonical orientation . Although he in figure 3.9, he rejected those with agreedwith the intuitions concerning the scenes respectto figure 3.10. That allowed him to ignore the distinction betweenthe referent' s and the relatum' s perceptual frame and to formulate a really simple principle, the " framework vertical constraint," which says that " no spatial description may conflict with the meaningsof aboveand belowdefined by the framework in which the related objects are located." But the results by Carlson- Radvansky and Irwin ( 1993 ) for scenese and g in figure 3.8 contradict this because , according to Garnham, above / below derives in this case from the normal gravitational framework. Hence there is a conflict betweenthe meaning of abovein this framework and the description the ball is above the chair, which should make this description impossible according to his constraint, but it does not. The findings are, however, in agreementwith the the experimentsinvolved casessuch as the principle of canonical orientation because one just discussedfor fly 2 in figure 3.10. Garnham' s critique of my 1984formulation of the principle can, in part , be traced back to a vagueness of the term canonicalposition. It does not positively exclude the following strict interpretation : the dimension on which the intrinsic location is made should coincide with the samedimension in the perceptual frame. This is obviously " false, as Garnham ( 1989 , if a vehicle is parked ) correctly pointed out. For instance acrossa street, a bollard [traffic post] to the intrinsic right of the vehicle can still be describedas to its right " (p. 59), even if the perceptual frame for the bollard is given ' by the street (whoseright side is opposite to the vehicle s right side) . The only tenable " " interpretation of canonical position is a weaker one:

With this further specification, then, the principle of canonical orientation seems to be in agreementwith intuition and with experimental data. If in a scenecanonical orientation does not hold , the intrinsic system is evaded by the standard average European (SAE) language user; it is preempted by the deictic or by the absolute .8 system In this section I have discussed various properties of perspectivesystemsthat are of . We have seenthat systemsdiffer in inferential potential and pragmatic significance

andEllipsis in Spatial Descriptions Taking Perspective


in their demands on coordination between interlocutors. We also have seenthat if ' one systemis dominant , concurring systemsare not totally dormant in the speakers ' mind. Their rivalry appearsfrom the kind and speedof a subject s spatial judgments, and the outcome dependson quite abstract properties of the rivaling systems , as is the . of canonical orientation of the implication principle

3.4 Ellipsisin SpatialExpressions


. When we talk about Perspectivetaking is one aspect of our thinking for speaking about we create spatial properties of entities or predications spatial configurations, referents in the scene .. These predications usually relate the entity to some relatum . In short, the process of perspective taking in terms of some perspective system a a onto propositional or semanticone. The latter is the maps spatial representation 's which consists of lexical , that is, conceptsfor which there , concepts speaker message ' . are words in the speakers target language This state of affairs is well exemplified in figure 3.5. The samepattern is expressed ' . Figure in two systematicallydifferent ways, dependenton the speakers perspectives . Depending on the perspective one critical detail (circled) of this example 3.11represents as left or as right . Figure taken, the same referent/ relatum relation is expressed 3.11 express es that the choice of lexical concept (and ultimately of lexical item) depends . It is on the perspectivesystem being used , that is, on thinking for speaking here. It is that the to be clear on the spatial representation underlying assumption important -free; it is neither intrinsic nor deictic. This assumption is itself perspective mayor may not be correct, and I will return to it below. The issuein this section is whether spatial ellipsis originates beforeor after perspective taking . In other words, does the speaker decide not to mention a particular feature of the spatial representation , or rather, does the speakerdecidenot to express ? In the first case we will speak of " deep ellipsis" ; in a particular lexical concept " " the latter case , of surface ellipsis (roughly following Hankamer and Sag 1976on " . " and " surface " anaphora ) deep Compare the following two descriptionsfrom our data. Both relate to the encircled trajectory in the left pattern of figure 3.12, plus the move that precedesit . The first , the secondone is description is nonelliptic with respectto the directional expression . in that respect elliptic " " Full deictic: Right to yellow . Right to blue. Finished. " Elliptic deictic: From pink we go right one unit and place a yellow dot. One, er, one " unit from the yellow dot we place a blue dot .

Willem J. M . Levett

intrinsi deictic perspe perspectiv taking taking ~ ~ lexical lexical concep concept RIGHT LEFT ~ ~ SELECT LEXICAL ) C . ~ ~ word word " " l" eft r"ight
representationfrom a

The crucial feature of the latter , elliptic expressionis that it contains no spatial term that relates the blue dot to the (previous) yellow one. How does the speakercreate this ellipsis? There are, essentially , two possibilities. The first one is that the speaker in scanning the spatial configuration recognizesthat the new visual direction is the sameas the previous one. Before getting into perspectivetaking, the speakerdecides not to prepare that direction for expressionagain. This is deep ellipsis. The second possibility is that the speakerdoesapply deictic perspectiveto the secondmove, thus activating the lexical concept RIGHT a secondtime. This repeatedactivation of the concept then leads to the decision not to formulate the lexical concept a secondtime,

PerspectiveTaking and Ellipsis in Spatial Descriptions

right right

Figure 3.12 Deictic and intrinsic descriptions for two patterns. Can the last spatial tenD (right, straight) be ? deleted

that is, not to repeat the word right . This is surfaceellipsis. Thesetwo alternatives are depicted in figure 3.13. The alternatives can now be distinguished by observing what happensin descriptions from an intrinsic perspective . Here is an instance of a full intrinsic description of the sametrajectory : Full intrinsic : " Then to the right to a yellow node and straight to a blue node." Can the samestate of affairs be describedelliptically ? This should produce something like: Then to the right to a yellow nodeand to a blue node . The answer is not obvious; intuitions waver here. In case of deep ellipsis this should be possible . Just as the previous deictic speaker , the present intrinsic one will scan the spatial sceneand recognizethat the new direction is the sameas the previous one and the speakermay decide not to prepare it again for expression ; it is optional to mention the direction. But in case of surface ellipsis the intrinsic speaker has a problem. In the intrinsic system the direction of the first move is mapped onto the lexical concept RIGHT , whereasthe direction of the secondmove is mapped onto STRAIGHT . Because the latter is not a repetition of the former , it has to be formulated in speech . In other words, the condition for surface ellipsis is not met for the intrinsic speaker ; it is . obligatory to usea directional expression This state of affairs can now be exploited to test empirically whether spatial ellipsis is deep or surfaceellipsis. Does ellipsis occur in intrinsic descriptions of this kind? If

Willem J. M . Levelt
MODEL I " Surface ellipsis " ( ellipsis is perspective - dependent ) move ~ given perspective , is the same ( lexical ) concept to be expressed , i .e. the same directional term to be used ?

next

yes
-

use of directional expression is obligatory

of directional use is optional expression

MODEL2 " Deep ellipsis " (ellipsis is perspective- independent)

move next ~ new the direction of is the direction as the the same move ? move the of preceding no + use of directional is obligatory expression yes + directional use of is optional expression

Figure3.13 Surface ellipsisversusdeepellipsis. Is it reiterating a lexical concept or a spatial direction that ? matters

-+

PerspectiveTaking and Ellipsis in Spatial Descriptions

so, we have an argument for deep ellipsis. And we can create an alternative case where surface ellipsis is possible for intrinsic descriptions, but not deep ellipsis. An exampleconcernsthe encircled trajectory in the right pattern of figure 3.12. A normal full intrinsic description of this trajectory (plus the previous one) is Full intrinsic : Then right to green. And then right to black. Is surface ellipsis possible here, producing " Then right to green. And to black" or some similar expression ? That is an empirical issue . It should be clear that neither deep nor surface ellipsis is possible in a deictic description of this pattern. Take this full deictic description from our data: Full deictic: From white we go up to a greencircle. And from the greencircle we go right to a black circle. Surface ellipsis is impossible here because" right " is not a repetition of the previous directional term (" up" ) . Deep ellipsis is impossible because the trajectory direction is different from the previous one. Hence, if we find ellipsis in such cases , we will have to reject both models. In an experiment reported in Levelt ( 1982a ,b) we had asked 53 subjectsto describe 53 colored dot patterns, among them those in figure 3.12. I will call the circled moves " because in thesepatterns " critical moves the surface and deep models make predictions about them that differ critically for deictic and intrinsic descriptions in the way . Among the test patterns there were 14 that contained such critical just described moves are ; they given in figure 3.14. I checked all 53 subjects to detennine whether made they elliptic descriptions for any of these 14 critical trajectories. I removed all subjectswho did not have a consistent perspectiveover these 14 critical patterns; a ' subject s 14 pattern descriptions should either be all deictic or all intrinsic . This left me with 31 consistent deictic subjects and 13 consistent intrinsic ones,9 and hence with 44 x 14 = 616 pattern descriptions to be checked . In this set I found a total of 43 casesof ellipsis. 10 Theseare presentedin table 3.1. The table presentspredictions and results under both models of ellipsis. For each critical move I determined whether a directional term would be obligatory or optional (i.e., elidible) under the model in deictic and in intrinsic descriptions (such as I did above for the critical moves of the patterns in figure 3.12 ) . Hence there are four casesper model. The table presentsthe actual occurrence of ellipsis for these four caseswithin each model. It should be noticed that the two models make the same predictions with respectto deictic descriptions; if use of a directional term is obligatory under the surface model, it is also obligatory under the deep model and vice versa. But this is not so for the intrinsic descriptions.

100

Willem J. M . Levett

1 re f -

toe t

t ~ ~1

ty"(D t
~ o -- -~ ~ x t ~

o- -~ 3

~ o-

t t1 t ~ -oo

o- .-o~~ ::::9 - -- --..-Q i 1

Figure 3.14 " " Fourteen test patterns containing critical moves , including the two example patterns of or the other examplepattern as a substructure either the one includes test Each pattern figure 3.12. are circled. moves . The critical in two cases rotated ) (though

Takingand Ellipsisin SpatialDescriptions Perspective Table3.1 of Ellipsis underSurface and Deepmodels Distributionof Elliptical Descriptions
Model Description is Directional tenD is obligatory optional
Total

Surfaceellipsis deictic intrinsic Total

Deep ellipsis deictic intrinsic Total

I 24
25

18 0
18

19 24
43

1 24
25

0 18
18

I 42
43

occur, that model is in If a model says" obligatory ," but ellipsis does nevertheless trouble. How do the two models fare? It is immediately obvious from the table that the surface model is out. Where it prescribesobligatory use of a directional term, there are no less then 18 violations among the intrinsic descriptions (i.e., casesof ellipsis) and one among the deictic descriptions, for a total of 19. That is almost half . In contrast, the deep model is in good shape our sample ; there is only one deictic 11All other deictic and all intrinsic violates it . that descriptionsrespectthe description deep model. These findings show that the decision to skip mentioning a direction is really an 's . It precedes the speaker application ofa perspective early step in thinking for speaking ' is irrelevant here. The decision is s the ; speaker linguistic perspective system not on a semantic basedon a visual or imagistic representation , (lexical-conceptual) ) . This is, probably , the same level of representation representation (seefigure 3.11 where linearization decisionsare taken. When we describe2-D or 3-D spatial patterns (such as the patterns in figure 3.14 or the layout of our living quarters), we must . decideon someorder of description because speechis a linear medium of expression 1989 are nonlinguistic The principles governing these linearization strategies(Levelt 1981 , ) ) in character; they relate exclusively to the image (and in fact nonsemantic itself. But these very clear results on ellipsis create a paradox. If ellipsis runs on a -free spatial representation , spatial representations are apparently not perspective . But this contradicts the convincing experimental findings reported by perspectivized Brown and Levinson ( 1993 ) and by Levinson (chapter 4, this volume), which show that when a languageusesabsoluteperspective , its speakersuseoriented(i.e., perspective -dependent in ) spatial representations nonlinguistic spatial matching tasks. For instance , the subject is shown an array of two objects A and B on a table, where A is (deictica11y ) left of B (henceAB ) . Then the subject is turned around 1800to another table with two arrays of the sameobjects, namely, A -B and BA , and then asked to

102

Willem J. M . Levelt

indicate whic~ of the two arrays is identical to the one the subject saw before. The " " absolute subjectinvariably choosesthe BA array, where A is deictically to the right -=+ is the absolutedirection of the vector AB . of B. What the subjectapparently preserves A native English or Dutch subject, however, typically produces the deictic response that they (A -B) . Hence spatial representationsare perspectivizedalready, in the sense in tasks even follow the dominant perspectiveof the language , that is, nonlinguistic " " 12 where there is no thinking for speaking taking place. How to solve this paradox? One point to note is that the above ellipsis data and Brown and Levinson' s ( 1993 ) data on oriented spatial representationsinvolve different . As and the , perspectives ellipsis predictions are different for different perspectives can be seenfrom table 3.1, columns 1 and 4, the samepredictions result from the deep . The two models can only be distinguished and the surface model under deictic perspective 's when the speaker is intrinsic (cf. columns 2 and 5); violations perspective could show that neither model is correct. In this respect deictic under only perspective 's . If a speaker , absolute perspectivebehaveslike deictic perspective perspective is absolute, the deep and surface models of ellipsis make the samepredictions; if two arcs have the samespatial direction or orientation , the corresponding lexical concepts will be the sameas well (e.g., both north , or both east ). In other words, ellipsis data of the kind analyzedherecan only distinguish between 's the deep and surface models if the speaker perspectiveis intrinsic . One could then ' s show that absolute and deictic perspective that Brown and Levinson findings argue are " Whorfian ," that is, a property of the spatial representationitself. If , in addition , the intrinsic systemis not Whorfian in the samesense , the above ellipsis data would be explained as well. The problem is, of course, why intrinsic perspectiveshould be non-Whorfian . After all , speakersof Mopan , exclusiveusersof intrinsic perspective , will profit from registering the position of foregrounded objects relative to background objects that have intrinsic orientation . If at some later time the sceneis talked about from memory, that information about intrinsic position will be crucial for an intrinsic spatial description. But if we discard the option of excluding intrinsic perspective from " Whorfianness" the , paradox remains. to me, is the fact noted in the introduction that perspective More important , it seems " is linguistically free. There is no " hard-wired mapping from spatial to semantic . What we pick out from a scene in terms of entities and spatial representations relations to be expressedin language is not subject to fixed laws. There are preferences , human interest, and so on, , for sure, following Gestalt properties of the scene . Similarly, we can go for one perspectiveor but they are no more than preferences another if our culture leaves us the choice, and this chapter has discussedvarious reasonsfor choosing one perspectiverather than another, dependingon communica-

in Spatial Descriptions Perspective Taking and Ellipsis

103

tive intention and situation. It is correct to say that Guugu Yimithirr speakerscan choosefrom only one, absolute perspective , but that doesnot obliterate their freedom in expressingspatial configurations in language . The choice of referents , relata, spatial relations to be expressed , the pattern of linearization chosen when the sceneis complex, and even the decision to expressabsolute perspectiveat all (e.g., A is north of B, rather than A is in B' s neighborhood) are prerogatives of the speakerthat are not thwarted by the limited choice of perspective . As all other speakers , the Guugu Yimithirr can attend to various aspects of their spatial representations ; they can in what deem relevant and in that are express language they ways communicatively effective. This would be impossible if the spatial representationdictated its own semantics . Hence, Brown and Levinson' s ( 1993 ) important Whorflan findings cannot mean that spatial and semantic representationshave a " hard-wired" isomorphia. A more likely state of affairs is this. A culture' s dominant perspectivemakes a speaker attend to spatial properties that are relevant to that perspectivebecauseit will facilitate . In particular , theseattentional blasesmake the (later) discourseabout the scene -specific , such as the speakerregister in memory spatial features that are perspective absolute orientation of the scene . This does not mean, however, that an ellipsis decision must make referenceto such features . That one arc in figure 3.12 is acontinuation of another arc is a spatial feature in its own right that is available to a .speaker of any culture. Any speakercan attend to it and make it the ground for ellipsis. In -relevant spatial featuresdoes not preempt or other words, the addition of perspective suppressthe registration of other spatial properties that can be referred to or used in discourse .

3.5 Conclusion
This chapter openedby recalling, from Levelt ( 1989 ), the distinction betweenmacroplanning and microplanning . In macroplanning we elaborate our communicative intention , selecting information whose expressioncan be effective in revealing our intentions to a partner in speech . We decide on what to say. And we linearize the information to be expressed that is, we decideon what to say first , what to say next, , " and so forth . In microplanning, or " thinking for speaking , we translate the information to be expressedin some kind of " propositional " format , creating a semantic must , or message , that can be formulated. In particular , this message representation consist of lexical concepts that is for which there are in words the , , concepts target . When we apply thesenotions to spatial discourse , we can say that macroplanning language involves selectingreferents . , relata, and their spatial relations for expression Microplanning involves, among other things, applying some perspectivesystemthat will map spatial directions/ relations onto lexical concepts .

104

WillemJ. M. Levelt

The chapter has been largely about microplanning, in particular about the pragmatics of different perspective systems . It has considered the advantagesand disadvantages of deictic, intrinsic , and absolute systems for spatial reasoning and for speaker . It has also considered how a / hearer coordination in spatial discourse speakerdeals with situations in which perspectivesystemsare not aligned. " " Thinking for speaking led, as a matter of course, to the question whether this perspectival thinking is just for speaking or more generally permeatesour spatial thinking , that is, in some Whorfian way. The discussedrecent findings by Levinson and Brown strongly suggestthat such is indeed the case . I then presentedexperimental ' s decision data on spatial ellipsis showing that perspectiveis irrelevant for a speaker to elide a spatial direction term. Having speculatedthat the underlying spatial -free, contrary to the Whorfian findings, I argued representationmight be perspective that this is paradoxical only if the mapping from spatial representationsonto semantic " " representationsis hard-wired. But this is not so; speakershave great freedom in both macro- and microplanning . There are no strict laws that govern the choice of relatum and referent, that dictate how to linearize information , and so forth . In particular , there is no law that the speaker must acknowledge orientednessof a spatial representation (if it exists ) when deciding on what to expressexplicitly and what implicitly . There are only (often strong) preferenceshere that derive from Gestalt factors, cultural agreementon perspectivesystems , easeof coordination between interlocutors, requirementsof the communicative task at hand, and so on. Still , it is not my intention to imply that anything goes in thinking for speaking . " " Perspectivesystemsare interfaces between our spatial and semantic modules (in Jackendoff' s sense , chapter I , this volume), performing well-defined restricted map. The ping operations interfacing requirements are too specific for these perspective systemsto be totally arbitrary . But much more challenging is the dawning insight from anthropological work that there are only a few such systemsaround. What is it in our biological roots that makesthe choice so limited?
Notes I . I am in full agreementwith Levinson' s taxonomy of frames of reference(here called " perspective " systems ) in chapter 4 of this volume. The maiQdistinction is betweenrelative, intrinsic , and absolute systems , and each has an egocentric and an allocentric variant. The three perspectivesystemsdiscussedhere are relative egocentric ( = deictic), intrinsic allocentric, and absolute allocentric. The relative systemsare three-place relations betweenreferent, relatum, and baseentity (" me" in the deictic system ); the intrinsic and absolute systemsare two-place relations betweenreferent and relatum. 2. Brown and Levinson ( 1993 ) present the caseof Tenejapan , where the traverse direction in the absolute systemis not polarized, that is, spannedby two converseterms; there is just one

PerspectiveTaking and Ellipsis in Spatial Descriptions

105

tenD meaning " traverse ." Obviously, the notion of converseness is not applicable. The notion of transitivity , however, is applicable and holds for this system(seebelow in text). 3. Barbara Tversky ( personal communication) has correctly pointed out that Buhler ( 1934 ) " would treat this caseas a derived fonD of deixis, " Deixis am Phantasma , where the speaker imaginesbeing somewhere(for instancein the network) . There would be two speakersthen, a real one and imaginary one, each fonning a base for a (different) deictic system . This is . But Buhler' s caseis not strong unobjectionable as long as we do not confound the two systems for this network. It is not essential in the route-type description that " I " (the speaker in his imagination) make the moves and turns. If there were a ball rolling through the pattern, ' t have deictic the directional tenDSwould be just the same . But a ball doesn . What perspective the speaker in fact does in this description is to use the last directed path as the relatum for the subsequentpath. The new path is straight, right, or left from the current one. Hence it is the intrinsic orientation of the current path that is taken as the relatum. 4. I am ignoring a further variable, the listener' s viewpoint/ orientation . Speakerscan and often do expressspatial relations from the interlocutors perspective , as in for you, the ball is to the left of the chair. Conditions for this usagehave been studied by Herrmann and his colleagues(cf. Herrmann and Grabowski 1994 ). 5. Here I am considering only one case of nonalignment, namely, a 900 angle between the relevant bases . Another case studied by Carlson- Radvansky and Irwin ( 1993 ) is 1800 nonalignment. 6. Carlson- Radvanskyand Irwin do not discussitem-specificeffects , although it is likely that the type of relatum used is not irrelevant. It is the case , though, that their statistical findings . Another point to keep in mind is that the always agree between subject and item analyses " " on the experimental procedure may invite the development of perspectivestrategies part of " " , and occasionally the employment of an unusual perspective subjects . 7. Carison-Radvansky and Irwin included severalscenes that were fonnally of the sametype as scene(g) in figure 3.8, among them the one in figure 3.9 with fly 2. 8. There is, however, no reasonwhy this should also hold in other cultures. StephenLevinson ( personal communication), for instance , has presentedevidence that the principle does not hold for speakersof Tzeltal, who can use their intrinsic system when the relatum' s critical dimension is not in canonical orientation . But the Tzeltal intrinsic systemdiffers substantially from the standard average European (SAE) intrinsic system (see Levinson 1 992a ) . What is intrinsic top/ bottom in SAE is " longestdimension" or the " modal axis" of an object in Tzeltal; the fonner , but not the latter , has a connotation of verticality . 9. These numbers differ from those reported in Levelt ( 1982b ) becausethe present selection criterion is a different one. 10. My criterion for ellipsis was a strict one. There should, of course , be no directional tenD, but there also should be no coordination that can be interpreted as one directional tenD having scopeover two constituents, as in From pink right successively yellow and blue or A road turns right from pink and meets first yellow and then blue. I have excluded all caseswhere subjects mention a line on which the nodes are located.

106

Willem J. M. Levelt

II . The case occurs in a deictic description of the fourth pattern down the first column in . Andfrom there to a green node . figure 3.14. It goesas follows. From there left to a pink node This obviously violates both models of ellipsis. I prefer to seeit as a mistake or omission. 12. The discussion that follows in the text is much inspired by discussions with Stephen Levinson.
References

Brown : , P., and Levinson , S. C. ( 1993 ). Linguisticand nonlinguisticcodingof spatialarrays in Mayancognition . Working paperno. 24, CognitiveAnthropologyResearch Explorations . , Nijmegen Group, Max PlanckInstitutefor Psycholinguistics Buhler : Die Darstel /ungsfunktion . Jena . A major derSprache : Fischer , K. ( 1934 ). Sprachtheorie in translationin R. J. Jarvellaand W. Klein (Eds .), part on deixisfrom this work appeared : Studies in deixisandrelated : Wiley, 1982 . , place , andaction , 9- 30. Chichester Speech topics -Laird, P. N. ( 1989 . Journalof Memoryand , R. M. J., and Johnson ). Spatialreasoning Byrne . , 28, 564 575 Language -Radvansky Carlson in vision and , L. A., and Irwin, DE . ( 1993 ). Framesof reference - 244 ? Cognition : Whereis above . , 46, 223 language -Radvansky Carlson frameactivationduringspatial , L. A., and Irwin, DE . ( 1994 ). Reference - 671 . Journalof MemoryandLanguage termassignment . , 33, 646 Ehrich . In R. J. Jarvellaand W. Klein , V. ( 1982 ). The structureof living space descriptions - 249 .), Speech : Studies in deixisand relatedtopics . Chichester : , place , and action , 219 (Eds . Wiley Friederici in weightlessness : Perceptual , A. D., and Levelt , W. J. M. ( 1990 ). Spatialreference - 266 factorsandmentalrepresentations . Perception andPsychophysics . , 47, 253 . Garnham , A. ( 1989 ). A unified theory of the meaningof somespatial relational terms

- 60 . Cognition , 31 . 45 Hankamer andsurface , J., andSag , I. ( 1976 anaphora., Linguistic Inquiry , 7, 391- 426. ). Deep

Hill , A. ( 1982 . , front/ back , left/right: A contrastive ). Up/down study of Hausaand English In J. Weissenborn .), Hereand there : Cross on deixisand and W. Klein (Eds linguisticstudies - 42. Amsterdam demonstration : Benjamins . , 13 ' . Philosophical Levelt s linearization Transaction , W. J. M. ( 1981 ). The speaker problem of the - 315 . , London , B95 , 305 RoyalSociety Levelt . In S. Petersand , W. J. M. ( 1982a ). Linearizationin describingspatial networks - 220 E. Saarinen .), Process es . Dordrecht : Reidel . , beliefs , andquestions , 199 (Eds Levelt in theuseof spatialdirectionterms . In R. J. Jarvella , W. J. M. ( 1982b ). Cognitive styles - 268 and W. Klein (Eds .), Speech in deixisandrelatedtopics : Studies . , place , andaction , 251 Chichester : Wiley.

Perspective Taking and Ellipsis in Spatial Descriptions

107

. In A. vanDoom, limitationson talkingaboutspace Levelt , W. J. M. ( 1984 ). Some perceptual in honour : Essays .), Limits of perception W. vandeGrind, andJ. Koenderink of Maarten (Eds - 358 . Press . Utrecht : VNU Science A. Bouman , 323 . . Cambridge to articulation : Fromintention Levelt , MA .: MIT Press , W. J. M. ( 1989 ). Speaking : Tzeltalbody part tenninol Levinson , shape , and linguisticdescription , S. C. ( 1992a ). Vision . Workingpaperno. 12 , CognitiveAnthropologyResearch Group, ogyand objectdescription . Max PlanckInstitutefor Psycholinguistics , Nijmegen
of spatial Levinson, S. C. ( I 992b) . Language and cognition : The cognitive consequences description in Guugu Yimithirr . Working paper no. 13, Cognitive Anthropology Research , Nijmegen. Group , Max Planck Institute for Psycholinguistics . Cambridge, MA : Miller , G. A ., and Johnson-Laird , P. N . ( 1976 ) . Languageand perception . Harvard University Press , R. R., and Hurwitz , S. ( 1984 ) . Upward direction , mental rotation , and discrimination Shepard - 193 . of left and right turns in maps. Cognition, 18, 161 . In J. AskeN . Beery, L . Michaelis, and H . Filip Slobin, D . ( 1987 ) . Thinking for speaking of the Thirteenth Annual Meeting, 435- 444. (Eds.), Berkeley Linguistics Society: Proceedings . Berkeley: Berkeley Linguistics Society . In H . Pick and L . Acredolo (Eds.), Spatial ) . How languagestructures space Talmy , L . ( 1983 . orientation: Theory, research , and application. New York : Plenum Press ). Perspectivein spatial descriptions. Journal of Memory Taylor , H . A ., and Tversky, B. ( 1996 and Language(in press ). ) . Spatial mental models. In G. H . Bower (Ed.), The psychologyof learning Tversky, B. ( 1991 - 146 . . New York : Academic Press in researchand theory, vol . 27, 109 and motivation: Advances

4 Chapter
-

: Cross Framesof Reference and Molyneux' s Question UnguisticEvidence C. Levinson Stephen

4.1 WhatThisis AUAbout


The title of this chapter invokes a vast intellectual panorama; yet instead of vistas, I will offer only a twisting trail . The trail begins with some quite surprising cross cultural and cross linguistic data, which leads inevitably on into intellectual swamps " conversewith one another - issuesabout how our " inner languages and minefields , . information exchangingspatial To preview the data, first , languagesmake use of different frames of referencefor spatial description. This is not merely a matter of different use of the same set of frames of reference(although that also occurs); it is also a question of which frames of referencethey employ. For example , some languagesdo not employ our apparently fundamental spatial notions of left/right/front / back at all ; instead they may, for , specifying locations in terms of north/ , employ a cardinal direction system example south/ east / westor the like. There is a secondsurprising finding . The choice of a frame of referencein linguistic ) correlates with preferencesfor the same frame coding (as required by the language of referencein nonlinguistic coding over a whole range of nonverbal tasks. In short, -modal tendency for the same frame of referenceto be employed in there is a cross language tasks, recall and recognition memory tasks, inference tasks, imagistic reasoning that the underlying representation . This suggests tasks, and even unconsciousgesture have adopted the same and modalities all these capacities systemsthat drive . frame of reference Thesefindings, describedin section 4.2, prompt a seriesof theoretical ruminations " " in section 4.3. First , we must ask whether it even makes senseto talk of the same ! . Second frame of referenceacross modalities or inner representation systems , we " " reformation a and must clarify the notion frame of reference in language , suggest slight of the existing distinctions. Then we can, it seems , bring some of the distinctions made in other modalities into line with the distinctions made in the study of

110

Stephell C. Levinson

" " , so that some sensecan be made of the idea of sameframe of reference language acrosslanguage , nonverbal memory, mental imagery, and so on. Finally , we turn to the question Why does the same frame of reference tend to get employed across ? It turns out that modalities or at least across distinct inner representation systems information in one frame of referencecannot easily be converted into another, distinct . This has interesting implications for what is known as frame of reference " the 's " question about how and to what extent there is cross Molyneux question, modal transfer of spatial information .

T7 ...lt. 1 : Evidence 4.1 . Cross - ModalTra Mferof Frame of Reference from Tenejapan
To describewhere something (let us dub it the " figure" ) is with respectto something else (let us call it the " ground" ) we need some way of specifying angles on the horizontal . In English we achieve this either by utilizing features or axes of the " " ground (as in the boy is at the front of the truck ) or by utilizing anglesderived from ' " the viewer s body coordinates (as in the boy is to the left of the tree" ) . The first " the second a " relative frame of solution I shall call an " intrinsic frame of reference , ; " reference (becausethe description is relative to the viewpoint - from the other side of the tree the boy will be seento be to the right of the tree) . The notion " frame of " will be reference explicated in section 4.3 but can be thought of as labeling distinct . kinds of coordinate systems At first sight, and indeed on close consideration (see ; , for example , Clark 1973 Miller and Johnson-Laird 197 ( , these solutions seem inevitable, the only natural solutions for a bipedal creature with particular bodily asymmetrieson our planet. But they are not. Somc languagesusejust the first solution. Some languagesuse neither of thesesolutions; instead, they solve the problem of finding angleson the horizontal , something like our cardinal directions north , south, plane by utilizing fixed bearings east, and west. Spatial descriptions utilizing such a solution can be said to be in an " absolute" frame of reference becausethe ( anglesare not relative to a point of view, i.e., are not relative, and are also independentof properties of the ground object, i.e., are not intrinsic ) . A tentative typology of the three major frames of reference in , will be found in section 4.3. , with someindication of the range of subtypes language in a Mayan language . as found Here I wish to introduce one such absolute system , Tzeltal is a Mayan languagewidely spoken in Chiapas, Mexico, but the particular dialect described is spoken by at least 15,000 people in the Indian community of . The results ; I will therefore refer to the relevant population as Tenejapans Tenejapa with of an conducted here are a Penelope Brown part ongoing project, reported ,b; Levinson and Brown 1994 ). ( Brown and Levinson 1993a

Frames of Referenceand Molyneux ' s Question 4.2.1 Tzeltal Absolute Linguistic Frame of Reference

Tzeltal has an elaborate intrinsic system(seeBrown 1991 ; Levinson 1994 ), but it is of limited utility for spatial description becauseit is usually only employed to describe objects in strict contiguity . Thus for objects separatedin space , another system of a cardinal direction system , although spatial description is required. This is in essence it has certain peculiarities. First , it is transparently derived from a topographic feature : Tenejapa is a large mountainous tract , with many ridges and crosscut ting valleys , which neverthelessexhibits an overall tendency to fall in altitude toward the north -northwest. Hencedownhill hascome to mean (approximately) north , and uphill , the coordinate system is deficient, in that the orthogonal designatessouth. Second acrossis labeled identically in both directions (eastand west); the particular direction can be specifiedperiphrastically, by referring to landmarks. Third , there are therefore certain ambiguities in the interpretation of the relevant words. Despite this, however, the systemis a true fixed-bearing system . It applies to objectson the horizontal as well as on slopes . And speakersof the languagepoint to a specificdirection for down, and they will continue to point to the same compassbearing when transported outside their territory . Figure 4.1 may help to make the systemclear. The three-way semanticdistinction betweenup, down, and acrossrecurs in a number of distinct lexical systemsin the language . Thus there are relevant abstract nominals that describe directions, specializedconcrete nominals of different roots that describe , for example , edgesalong the relevant directions, and motion verbs that designate ascending (i.egoing south), descending (going north ), and traversing (going east or west) . This linguistic ramification , together with its insistent use in spatial description, make the three-way distinction an important feature of language use. There are many other interesting features of this system ( Brown and Levinson 1993a ), but the essentialpoints to grasp are the following . First , this is the basic way to describethe relative locations of all objects separatedin spaceon whatever scale . Thus if one wanted to pick out one of two cups on a table, one might ask for , say, the uphill one; if one wanted to describewhere a boy was hiding behind a tree, one might , say, the north (downhill ) side of the tree; if one wanted to ask where designate " someonewas going, the answer might be " ascending (going south); and so forth . Second , linguistic specificationslike our to the left, to the right, infront , behindare not " available in the language ; thus there is no way to encodeEnglish locutions like pass " " " " the cup to the left , the boy is in front of the tree, or take the first right Turn.,,2 Third , the useof the systempresupposes a good sense of direction ; testsof this ability to keep track of directions (in effect, to dead reckon), show that Tenejapans , even

112

StephenC. Levinson

" "The bottleis uphill of thechair.


w . . -I - 1 ..-Ji'oI .dIG ,. IiI8 * Ir , . . ~ ~ ~ at II..8p6IlI dIG

4.1 Flame . Tzeltal /downhill system Tenejapan uphill

Frames of Referenceand Molyneux ' s Question

'~ fo
z

1 Table STIMU r
113

2 Table

r , cae

: TASK Choose arrow same as stimulus


z

1 r RELATIVE ABSOLUTE

~ ~
. Figure 4.2 . Underlying design of the experiments

without visual access to the environment, do indeed maintain the correct bearingsof various locations as they move in the environment. In short, the Tzeltal linguistic system does not provide familiar viewer-centered locutions like " turn to the left " or " in front of the tree." All such directions and locations can be adequatelycoded in terms of antecedentlyfixed, absolute bearings . work on an Australian language(Haviland 1993 ; Levinson 1992b Following ) where such a linguistic system demonstrably has far -reaching cognitive consequences , a seriesof experimentswere run in Tenejapa to ascertainwhether nonlinguistic coding might follow the pattern of the linguistic coding of spatial arrays. 4.2.2 Use of an Absolute Frame of Reference in NonverbalTasks 4.2.2.1 Memory and Inference As part of a larger comparative project, my colleagues and I have devised experimental means for revealing the underlying nonlinguistic coding of spatial arrays for memory (seeBaayen and Danziger 1994 ) . The ' aim is to find tasks where subjects will reveal which frame of reference , responses intrinsic , absolute, or relative, has been employed during the task. Here we concentrate on the absolute versus relative coding of arrays. The simple underlying design behind all the experimentsreported herecan be illustrated as follows. A male subject, say, seesan array on a table (table I ) : an arrow pointing to his right , or objectively to the north (seefigure 4.2) . The array is then removed, and after a delay, the subject

Left

114

StephenC. Levinson

is rotated 180degreesto face another table (table 2) . Here there are, say, two arrows, one pointing to his right and one to his left - that is, one to the north and one to the south. He is then askedto identify the arrow like the one he saw before. Ifhe chooses the one pointing to his right (and incidentally to the south), it is clear that he coded the first arrow in terms of his own bodily coordinates, which have rotated with him. If he choosesthe other arrow , pointing north (and to his left ), then it is clear that he coded the original array without regard to his bodily coordinates, but with respectto somefixed bearing or environmental feature. Using the samemethod, we can explore a range of different psychological faculties: recognition memory (as just sketched ), recall memory (by, for example , asking the subject to place an arrow so that it is the sameas the one on table I ) and various kinds of inference(as sketchedbelow) . We will describeherejust three such experimentsin outline form (seeBrown and Levinson 1993b for further details and further experiments ) . They were run on at least twenty five Tenejapansubjects(dependingon the experiment) of mixed age and sex / sex , and a Dutch comparison group of at least thirty -nine subjectsof similar age relative between absolute and . As far as the distinction coding linguistic composition , Dutch like English relies heavily of course on a right/left/front / back system goes -centered coordinates for the description of most spatial arrays. So the of speaker hypothesisentertainedin all the experimentsis the following simple Whorfian conjecture : the coding of spatial arrays- that is, the conceptual representationsinvolvedin a range of nonverbal tasks should employ the same frame of reference that is dominant in the languageused in verbal tasks for the same sort of arrays. Because Dutch , like English, provides a dominant relative frame of reference , we expect Dutch subjectsto solve all the nonlinguistic tasks utilizing a relative frame of reference . On the other hand, becauseTzeltal offers only an absolute frame of reference for the relevant arrays, we expectTenejapan subjectsto solve the nonlinguistic tasks . Clearly it is crucial that the instructions utilizing an absolute frame of reference for the experiments , do not suggestone , or the wording used in training sessions . Instructions (in Dutch or Tzeltal) were of the or another of the frames of reference " kind " Point to the pattern you saw before," " Remake the array just as it was, ." " Rememberjust how it is, that is, as much devoid of spatial information as possible , . and as closely matched in content as could be achievedacrosslanguages
Recall Memory

Method The design was intended to deflect attention from memorizing direction towards memorizing order of objects in an array, although the prime motive was to 3 tap recall memory for direction. The stimuli consistedof two identical sets of four model animals (pig, cow, horse, sheep ) familiar in both cultures. From the set of four ,

' Frames of Reference and Molyneux s Question

115

three were aligned in random order , all heading in ( a randomly assigned) lateral direction on table I . Subjects were trained to memorize the array before it was removed " " , then after a three - quarters of a minute delay to rebuild it exactly as it was , first with correction for misorders on table I , then without correction under rotation on table 2. Five main trials then proceeded , with the stimulus always presented on

table I , and the responserequired under rotation , and with delay, on table 2. Responses " were coded as " absolute if the direction of the recalled line of animals " " preservedthe fixed bearingsof the stimulus array , and as relative if the recalledline preservedegocentricleft or right direction. Results Ninety -five percent of Dutch subjectswere consistent relative coders on at least four out of five trials , while 75% of Tzeltal subjects were consistent absolute . The remainder failed to recall direction so consistently. coders by the samemeasure For the purposes of comparison across tasks, the data have been analyzed in the ' following way. Each subject s performance was assignedan index on a scalefrom 0 to 100 , where 0 representsa consistentrelative responsepattern and 100a consistent absolute pattern; inconsistenciesbetween codings over trials were representedby indices in the interval. The data are displayed in the graph of figure 4.3, where subjectsfrom each population have beengrouped by 20-point intervals on the index. As the graph makes clear, the curves for the two populations are approximately mirror images , except that Tenejapan subjectsare lessconsistent than Dutch ones. This may be due to various factors: the unfamiliarity of the situation and the tasks, , or to the " school" -like nature of task performed by largely unschooled subjects less dominant. but is available that of reference frame interferencefrom an egocentric Only two Tenejapan subjects were consistent relative coders (on 4 out of 5 trials) . . The result appears to This pattern is essentially repeated across the experiments confirm the hypothesis that the frame of referencedominant in the language is the frame of referencemost available to solve nonlinguistic tasks, like this simple recall task. RecognitionMemory Method Five identical cards were prepared; on each there was a small green circle and a large yellow circle.4 The trials wereconducted as follows. One card was usedas a stimulus in a particular orientation ; the subject saw this card on table I . The other four were arrayed on table 2 in a number of patterns so that each card was distinct by orientation (seefigure 4.4) . The subject saw the stimulus on table I , which was then removed , and after a delay the subject was rotated and led over to table 2. The subject was asked to identify the card most similar to the stimulus. The eight trials

116

StephenC. Levinson

4.3 Figure

2 0
20 40

Dutch (n- 37 )

..... Tenejapan (n- 27)

60

80

100

Estimated absolute tendency (%)

Animals recall task: direction.

were coded as indicated in figure 4.3: if the card which maintained orientation from an egocentricpoint of view (e.g., " small circle toward me" ) was selected , the response was coded as a relative response , while the card which maintained the fixed bearings of the circles (" small circle north " ) was coded as an absolute response . The other two cards servedas controls, to indicate a basic comprehensionof the task. Training was conducted first on table I , where it was made clear that sameness of type rather than token identity was being requested .

Results We find the same basicpatternof resultsasin the previoustask , as shown in figure4.5. Onceagain the Dutch are relativecoders , , whilethe subjects consistently

Frames of Referenceand Molyneux ' s Question

117

Figure 4.4 " " " " Chips recognitionl task: absolute versus relative solutions.

. Nevertheless , of the Tenejapan subjects who perTenejapans are less consistent fonned consistently over 6 or more of 8 trials, over 80% were absolute coders. The greater inconsistency of Tenejapan subjects may be due to the same factors mentioned above, but there is also here an additional factor becausethis experiment -west) testedfor memory on both the transverseand sagittal (or north -south and east . As mentioned above, the linguistic absolute axesare asymmetric: one axis has axes distinct labels for the two half lines north and south, while the other codesboth east " . If there was some effect of this and west identically (" across ) linguistic coding on the conceptual coding for this nonlinguistic task, one might expect more errors or -west axis. This was indeed the case . inconsistencyon the east Trasiti , e Il Jference Levelt ( 1984 ) observed that relative, as opposed to intrinsic , transitive and converseinferences relations ; Levinson ( 1992a ) noted support spatial that absolute spatial relations also support transitive and converse inferences(see also Levelt, chapter 3, this volume) . This makes it possible to devise a task where, " from two spatial arrays or nonverbal " premises , a third spatial array, or nonverbal " conclusion" can be drawn by transitive inference utilizing either an absolute or a . The following task was designed by Eric Pedersonand relative frame of reference BernadetteSchmitt, and piloted in Tamilnadu by Pederson( 1994 ).

table

table 1

E3

ADS

REL

(;

~ ca ~

118

4.5 Figure task . Chips recognition

100 \ 80 60 40 20 0 10 60 80 20 40 0
StephenC. Levinson "'-""-h Dutch (n- 39) ..... Tenejapan (n- 24)

Estimated absolute tendency (%)

" " , a blue Design Subjects seethe first nonverbal premise on table 1, for example cone A and a yellow cube B arranged in a predetermined orientation . The top diagram in figure 4.6 illustrates one such array from the perspectiveof the viewer. Then " " , a red cylinder C and the yellow subjectsare rotated and seethe second premise cube B in a predeterminedorientation on table 2 (the array appearing from an egocentric , in the seconddiagram in figure 4.6) . Finally , point of view as, for example subjectsare rotated again and led back to table 1, where they are given just the blue cone A and asked to place the red cylinder C in a location consistent with the previous " " nonverbal " premises ." For example , if a female subject, say, sees( premise 1 )

Frames

of Reference

and Molyneux

'

s Question

119

Table 1

. blue A

yellow B '

First premise

'

2 Table yellow B ' ' Second premise EJ red C


1 J Table

~ red C Absolute Solution

) blue A

4.6 Figure - the visual arrays. inference Transitive

: red

Solution

) blue A

( Relative

Table 1

120

StephenC. Levinson

" " the yellow cube to the right of the blue cone, then ( premise 2 ) the red cylinder to the right of the yellow cube, when given the blue cone, she may be expectedto place the red cylinder C to the right of the blue cone A . It should be self-evident from the top two diagrams in figure 4.6, representing the arrays seen sequentially, why the " " " " third array (labeled the relative solution ) is one natural nonverbal conclusion from the first two visual arrays. However, this result can only be expectedif the subjectcodesthe arrays in terms of egocentricor relative coordinates which rotate with her. If instead the subject utilizes " " fixed bearings or absolute coordinates, we can expect a different conclusion - in fact the reversearrangement , with the red cylinder to the left of the blue cone (seethe " last diagram labeled " absolute solution in figure 4.6) ! To seewhy this is the case , ' If the situation. view of the consider figure 4.7, which givesa bird s eye experimental subjectdoesnot usebodily coordinates that rotate with her, the blue cone will be, say, south of the yellow cube on table I , and the red cylinder farther south of the yellow cube on table 2; thus the conclusion must be that the red cylinder is south of the blue cone. As the diagram makesclear, this amounts to the reversearrangementfrom that , and in half the trials , produced under a coding using relative coordinates. In this case the absolute inference is somewhat more complex than a simple transitive inference ), but in the other half of the trials the relative (involving notions of relative distance one in just the sameway. than the absolute more solution was complex
Method Three objects distinct in shape and color were employed . Training was conducted on table I , where it was made clear that the positions of each object relative to the other object - rather than exact locations on a particular table - was the relevant thing to remember . When transitive inferences were achieved on table I , subjects were introduced to the rotation between the first and second premises ; no correction was given unless the placement of the conclusion was on the orthogonal axis to the stimulus arrays . There were then ten trials , randomized across the transverse and sagittal axes ( i .e., the arrays were either in a line across or along the line of vision ) .

Results The resultsare given in the graph in figure 4.8 Essentially , we have the same Dutch : subjectsare consistently pattern of resultsas in the prior memory experiments relative coders , and Tenejapan subjects strongly tend to absolute coding, but more inconsistently. Of the Tenejapanswho produced consistentresultson at least 7 out of 10trials , 90% were absolutecoders(just two out of25 subjectsbeing relative coders ). are of for the The reasons presumably greater inconsistency Tenejapan performance the sameas in the previous experiment: unfamiliarity with any such procedure or test -west axis lacking situation and the possibleeffectsof the weak Absolute axis (the east

' Frames of Reference and Molyneux s Question

1 Table TASK : PllCeC " ~ A ,--ca '~ ~ ~ . ~ ' ; O / M ~ ' 1 ~I ~ ~ 1 Table 1 Table C . , '" A A ( c3 C(:---r~~
RELATIVE Response
ABSOLUTE Response
Figure 4.7 - bird ' s- eyeview of experimental situation. Transitive inference

~ III I

1 ~ Table Sub -~ " ' ~ " ~ / ~ 1 B IJ ~ -ca {:: A


~.. N

100 80 ~ 60 : c ' i 5 t c 40 ~ 20 0 10 80 40 60 20 0
122
StephenC . Levinson
--....... Dutch ( n - 39 )

..... Tenejapan (n- 25)

Estimated absolute (%) tendency

4.8 FiIUre Transitive inferencetask

distinct linguistic labels for the half lines) . Once again, Tenejapansmade most errors -west axis. or performed most inconsistently, on the east DiSC IIS S ;OIl The results from these three experiments , together with others unreported here (seeBrown and Levinson 1993b ), all tend in the samedirection. While Dutch subjectsutilize a relative conceptual coding ( presumably in terms of notions like left , right , in front , behind ) to solve these nonverbal tasks, Tenejapan subjects use an absolute . This is of coursein line with the coding predominantly coding system . The samepattern built into the semanticsof spatial description in the two languages holds across different psychological faculties: the ability to recall spatial arrays, to

' Frames of Referenceand Molyneux s Question

123

recognize those one has seen before, and to make inferences from spatial arrays. Further experiments of different kinds, exploring recall over different arrays and inferencesof different kinds, all seemto show that this is a robust pattern of results. The relative inconsistencyof Tenejapanperformance might simply be due to unfamiliar materials and proceduresin this largely illiterate , peasantcommunity . But as accumulatedon one absoluteaxis inparticular above, errors or inconsistencies suggested . However, becausethe experimentswere all run on one set of fixed bearings , the error pattern could have been due equally to a strong versus weak egocentric axis -west (and in fact it is known that the left -right axis- here coinciding with the east half the subjects than the front back axis . Therefore axis is less robust conceptually ) . were recalled and the experiments rerun on the orthogonal absolute bearings do indeed accumulate The results showed unequivocally that errors and inconsistencies -west absolute axis (although there also appears to be some interference on the east from egocentric axes ) . This is interesting becauseit shows that Tenejapan subjectsare not simply using an ad hoc system of local landmarks, or some fixed; rather, the conceptual primitives bearing systemtotally independentof the language used to code the nonverbal arrays seem to inherit the particular properties of the semanticsof the relevant linguistic distinctions. This raises the skeptical thought that perhaps subjectsare simply using linguistic mnemonics to solve the nonverbal tasks. However, an effective delay of at least three-quarters of a minute betweenlosing sight of the stimulus and responding on table 2 would have required constant subvocal rehearsalfor the mnemonic to remain available in short-term memory. Moreover, there is no particular reasonwhy subjects should converge on a linguistic rather than a nonlinguistic mnemonic (like crossing - which the fingers on the relevant hand, or using a kinesthetic memory of a gesture would yield uniform relative results ) . But above all , two other experimental results suggest the inadequacy of an account in terms of a conscious strategy of direct linguistic coding. 4.2.2.2 Visual Recall and Gesture The first of these further experimentsconcerns the recall of complex arrays. Subjectssaw an array of betweentwo and five objects on table I , and had to rebuild the array under rotation on table 2. Up to five of these , a model of a chair, a truck , a tree, a , for example objects had complex asymmetries . The majority of Tenejapan subjectsrebuilt the horse leaning to one side, or a shoe arrays preserving the absolute bearings of the axes of the objects. This amounts to mental rotation of the visual array (or of the viewer) on table I so that it is reconstructed on table 2 as it would look like from the other side. Tenejapansprove to be exceptionally good at this, preservingthe metric distancesand preciseanglesbetween objects. It is far from clear that this could be achievedeven in principle by a linguistic

124

StephenC. Levinson

coding: the precise angular orientation of each object and the metric distances between objects must surely be coded visually and must be rebuilt under visual control of the hands. This ability argues for a complex interaction between visual memory and a conceptual coding in terms of fixed bearings : an array that is visually distinct may be conceptually identical, and an array visually identical may be conceptually distinct (unlike with a systemof relative coding, where what is to the left side of the " that an visual field can be describedas to the left) . Thus being able to " see array is conceptually identical to another in absolute terms may routinely involve mental rotation of the visual image. That a particular conceptual or linguistic system may exerciseand thus enhanceabilities of mental rotation has already beendemonstrated for American Sign Language(ASL ) by Emmorey (chapter 5, this volume) . Tenejapans appear to be able to memorize a visual image of an array tagged , as it were, with the relevant fixed bearings . There is another line of evidencethat suggests that the Tenejapan absolute coding of spatial arrays is not achievedby conscious artificial use of linguistic mnemonics . , To show this, one would wish for some repetitive, unconscious nonverbal spatial behavior that can be inspected for the underlying frame of referencethat drives it . There is indeedjust such a form of behavior, namely, unreflective spontaneousgesture . Natural Tenejapan conversation can be inspectedto see accompanying speech whether, when places or directions are referred to , gesturespreservethe egocentric coordinates appropriate to the protagonist whose actions are being described , or whether the fixed bearings of those locations are preservedin the gestures . Preliminary work by PenelopeBrown showsthat such fixed bearingsare indeed preservedin spontaneousTenejapan gestures A pilot experiment seemsto confirm this. In the experiment, a male subject, say, facing north , seesa cartoon on a small portable monitor with lateral action from east to west. The subject is then moved to another room where he retells the story as best he can to another native speakerwho has not seenthe cartoon. In one condition , the subject retells the story facing north ; in another condition the subject retells the story facing south. Preliminary results show that at least somesubjectsunder rotation systematicallypreservethe fixed bearing of the observed action (from east to west) in their gestures , rather than the direction coded in terms of left or right . (Incidentally, the reversefinding has been established for American English by McCullough 1993 ) . Becausesubjects had no idea that the experimenter was interested in gesture , we can be sure that the gestures record unreflective conceptualization of the directions. Although the gesturesof course accompany , gesturespreserving the fixed bearings of the stimulus often occur speech without explicit mention of the cardinal directions, suggestingthat the gesturesreflect an underlying spatial model, at least partially independentof language .

Frames of Referenceand Molyneux ' s Question

125

4.2.3 Conclusion from theTenejapan Studies


Putting all these results together, we are led to the conclusion that the frame of referencedominant in the language , whether relative or absolute, comes to bias the choice of frame of referencein various kinds of nonlinguistic conceptual representations . This correlation holds across a number of " modalities" or distinct mental : over codings for recall and recognition memory, over representations representations for spatial inference , over recall apparently involving manipulations of visual and over whatever kind of kinesthetic representation systemdrives gesture . , images These findings look robust and general similar observations have been ; previously made for an Aboriginal Australian community that usesabsolute linguistic spatial -cultural survey over a ; Levinson 1992b description (Haviland 1993 ), and a cross dozen non-Western communities shows a strong correlation of the dominant frame of referencein the linguistic systemand frames of referenceutilized in nonlinguistic tasks (seeBaayenand Danziger 1994 ).
4.3 Frames of Reference aerna Modalities

Thus far , we have seenthat ( I ) not all languagesuse the samepredominant frame of referenceand (2) there is a tendency for the frame of referencepredominant in a particular languageto remain the predomina~t frame of referenceacrossmodalities, as displayed by its use in nonverbal tasks of various kinds, unconsciousgesture , and so on. The results seemfirm ; they appear to be replicable acrossspeechcommunities, but the more one thinks about the implications of thesefindings, the more peculiar they seem to be. First , the trend of current theory hardly prepares us for such Whorfian results: the general assumption is rather of a universal set of semantic primes (conceptual primitives involved in language ), on the one hand, and the identity or homomorphism of universal conceptual structure and semantic structure, on the other. Second , ideas about modularity of mind make it seemunlikely that such -modal effectscould occur. Third , the very idea of the sameframe of reference cross acrossdifferent modalities, or different internal representationsystemsspecializedto different sensorymodalities, seems incoherent. In order to make sense of the results, I shall in this section attempt to show that the notion " same frame of referenceacross modalities" is, after all , perfectly coherent, and indeed already adumbrated across the disciplines that study the various mod" across alities. This requires a lightning review of the notion " frame of reference the relevant disciplines (section 4.3.1 and 4.3.2); it also requires a reformation of the linguistic distinctions normally made (section 4.3.3). With that under our belts, we can then face up to the peculiarity , from the point of view of ideas about the

126

StephenC. Levinson

modularity of mind , of this cross modal adoption of the same frame of reference some intrinsic 4.4 . Here properties of the different frames of reference may ) ( section offer the decisive clue: if there is to be any cross- modal transfer of spatial information , we may have no choice but to fixate predominantly on just one frame of reference.

" of Reference 4.3.1 " SpatialFrames


" The notion of " frames of reference is crucial to the study of spatial cognition across all the modalities and all the disciplinesthat study them. The idea is as old as the hills: medieval theoriesof space , were deeply preoccupiedby the puzzle raised , for example in the river. If we think about the location moored the boat case of by Aristotle , the of an object as the place that it occupies , and the place as containing the object, then the puzzle is that if we adopt the river as frame of reference , the boat is moving, but - 201 for a if we adopt the bank as frame, then it is stationary (seeSorabji 1988 , 187 discussionof this problem, which dominated medieval discussionsof space ). " " But the phrase frame of reference and its modern interpretation originate, like . How , for so much elseworthwhile , from Gestalt theories of perception in the 1920s acrossthe the moon skims as when of motion illusions account for do we , , example clouds, except by invoking a notion of a constant perceptual window against which motion (or the perceived vertical, say) is to be judged? The Gestalt notion can be summarized as " a unit or organization of units that collectively serve to identify a coordinatesystemwith respect to which certain properties of objects, including the " 6 , 404; emphasismine) . phenomenalself, are gauged (Rock 1992 In what follows , I will emphasizethat distinctions betweenframes of referenceare , essentiallydistinctions betweenunderlying coordinate systemsand not , for example .7 In a recent review, between the objects that may invoke them. Not all will agree ) ranging over the philosophical and psychologiphilosophersBrewer and Pears( 1993 cal literature , conclude that frames of referencecome down to the selectionof reference - when I go from one room to another, do on my nose objects. Take the glasses "" they change their location or not? It dependson the frame of reference nose or s room. This emphasison the ground or relatum or referenceobject9 severelyunderplays the importance of coordinate systemsin distinguishing frames of reference , as I 10 of I : can of reference shall show below. Humans usemultiple frames happily say the ' " ' sameassemblage (ego looking at car from side, car s front to ego s left ): the ball is " in front of the car" and " the ball is to the left of the car, without thinking that the ball has changed its place. In fact, much of the psychological literature is concerned with ambiguities of this kind . I will therefore insist on the emphasison coordinate " " systemsrather than on the objects or units on which such coordinates may have their origin .

Frames of Referenceand Molyneux ' s Question

127

" acroa Modalities andthe Disciplines that StudyThem 4.3.2 " Frames of Reference
" across different If we are to make senseof the notion " same frame of reference modalities, or inner representationsystems , it will be essentialto seehow the various distinctions betweenthe frames of referenceproposed by different disciplines can be ultimately brought into line. This is no trivial undertaking, becausethere are a host of such distinctions, and eachof them has beenvariously construed, both within and acrossthe many disciplines (such as philosophy, the brain sciences , psychology, and " frames of reference " A serious review . that the notion linguistics) explicitly employ of thesedifferent conceptionswould take us very far afield. On the other hand, some sketch is essential , and I will briefly survey the various distinctions in table 4.1, with . ll somedifferent construals distinguished by the letters a, b, C " " ' " " . Newton s distinction betweenabsolute First , then, relative versus absolute space and relative spacehas played an important role in ideas about frames of refer-

4.1 Table Frames of Reference : Some Distinctions in theLiterature Spatial


" ven8 " absolute ": " Relative , brain sciences , linguistics ) (philosophy a. Spaceas relations betweenobjects versusabstract void b. Egocentric versusallocentric c. Directions: Relations betweenobjects versusfixed bearings " " " " Egocentric ven8 aUocentric and behavioralpsychology , brain sciences ) (developmental a. Body-centeredversusenvironment-centered(Note many ego centers : retina, shoulder, etc.) b. Subjective(subject- centered ) versusobjective " versus" " " " " Viewer- centered " " object- centered or 2} -0 sketch ven8 3- D models ) ( vision theory, imagery debatein psychology " ven8 " orientation " Orientation- bound -free" , imagery debatein psychology ) ( visualperception " " Deictic" ven8 " intril Ltic ) (linguistics a. Speaker - centric versusnon-speaker - centric versusthing b. Centered on speakeror addressee c. Ternary versusbinary spatial relations " versus" " " " Viewer-centered " -centered object-centered vers18 environment (psycholinguistics ) = " gazetour " versus" body tour " perspectives " " versus" route = ?" survey perspective perspective

128

StephenC. Levinson

ence , in part through the celebrated correspondencebetween his champion Clarke and Leibniz, who held a strictly relative view. 12 For Newton , absolute spaceis an abstract, infinite , immovable, three-dimensional box with origin at the center of the universe , while relative spaceis conceivedof as specifiedby relations betweenobjects. " , Newton claimed, we are inclined to relative notions: Relative space Psychologically is some moveable dimension or measure of the absolute spaces , which our senses instead of absolute . . and so determine by its position to bodies. placesand motions, " we use relative ones (quoted in Jammer 1954 , 97 98) . Despite fundamental differences thinkers in philosophy and psychology in philosophical position , most succeeding - spaceanchored to have assumedthe psychological primacy of relative space the places occupied by physical objects and their relations to one another- in our mental life. A notable exception is Kant , who cameto believethat notions of absolute , space are a fundamental intuition , although grounded in our physical experience that is, in the useof our body to define the egocentriccoordinates through which we ' deal with space(Kant 1768 ) . O Keefe and ; seealso Van Cleve and Frederick 1991 ' Nadel ( 1978 ; seealso O Keefe 1993and chapter 7, this volume) have tried to preserve this Kantian view as essentialto the understanding of the neural implementation of our spatial capacities , but by and large psychologists have considered notions of " absolute" irrelevant to theories of the naive spatial reasoning underlying language space , 380) . (Absolute notions of ; Miller and Johnson- Laird 1976 (seeClark 1973 space may, however, be related to cognitive maps of the environment discussed " " under the rubric of allocentric frames of referencebelow.) Early on, the distinction betweenrelative and absolute spaceacquired certain additional associations , relative space became associatedwith egocentric ; for example coordinate systems , and absolute space with non-egocentric ones (despite Kant 13 1768 ), so that this distinction is often confused with the egocentric versus allo centric distinction (discussedbelow) . Another interpretation of the relative versus absolute distinction , in relating relativistic spaceto egocentric space , goeson to emphasize in the different ways coordinate systemsare constructed relative versusabsolute : " Ordinary languagesare designedto deal with relativistic spatial conceptions ; with space relative to the objects that occupy it . Relativistic space provides space three orthogonal coordinates, just as Newtonian space does , but no fixed units of to extend without coordinates need , nor is there any for angle or distanceare involved " limit in any direction (Miller and Johnson Laird 1976 , 380; emphasismine) . Thus a to the relativistic " space is directions or cardinal of fixed , , opposed bearings system " whether , which Miller and Johnson-Laird egocentric or object-centered concept, 1973 like Clark other authors and ) and Svorou , ), Herskovits ( 1986 , 395) ( ( 1976 many , 213), have assumedto constitute the conceptual core of human spatial thinking ( 1994 . But because , some languagesuse as a conceptual basis coordi , as we have seen

Frames of Referenceand Molyneux ' s Question

129

nate systemswith fixed angles (and coordinates of indefinite extent), we need to " " recognizethat thesesystemsmay be appropriately called absolute coordinate systems . Hence I have opposed relative and absolute frames of referencein language (seesection 4.3.3). " versus Let us turn to the next distinction in table 4.1, namely, " egocentric " allocentric." The distinction is of course between coordinate systemswith origins within the subjectivebody frame of the organism, versuscoordinate systems centered elsewhere(often unspecified , ) . The distinction is often invoked in the brain sciences where there is a large literature concerning frames of reference(see , for example , the ) . This emphasizesthe plethora of different egocentric compendium in Paillard 1991 coordinate systemsrequired to drive all the different motor systemsfrom saccades to arm movements(see for Stein 1992 or the control of the head a as platform , , ), example for our inertial guidanceand visual systems(again seepapersin Paillard 1991 ). In addition , there is a general acceptance(Paillard 1991 , 471) of the need for a ' distinction (following Tolman 1948 ; O Keefe and Nadel 1978 ) between egocentric ' and allocentric systems . O Keefe and Nadel' s demonstration that something like Tolman ' s mental maps are to be found in the hippocampal cells is well known. 14 O' Keefe' s recent ( 1993 ) work is an attempt to relate a particular mapping systemto the neuronal structures and process es. The claim is that the rat can use egocentric measurementsof distance and direction toward a set of landmarks to compute a non-egocentric abstract central origo (the " centroid" ) and a fixed angle or " slope." Then it can keep track of its position in terms of distancefrom centroid and direction from slope. This is a " mental map" constructed through the rat ' s exploration of the environment, which gives it fixed bearings (the slope), but just for this environment. Whether this strictly meetsthe criteria for an objective, " absolute," allocentric system 15 has been questioned (Campbell 1993 , 76- 82). We certainly need to be able to " " distinguish mental maps of different sorts: egocentric strip maps (Tolman 1948 ), allocentric landmark-based maps with relative angles and distances between landmarks (more Leibnizian), and allocentric maps basedon fixed bearings (more Newtonian 16 , this is the sort of thing neurophysiologistshave in mind ) . But in any case " and " allocentric" frames of reference when they oppose" egocentric .17 Another area of work where the opposition has beenusedis in the study of human . For example , Acredolo ( 1988 ) showsthat , as Piagetargued, conceptualdevelopment infants have indeed only egocentric frames of referencein which to record spatial memories ; but contrary to Piaget (Piaget and Inhelder 1956 ), this phase lasts only for perhaps the first six months. Thereafter, they acquire the ability to compensate for their own rotation , so that by sixteen months they can identify , say, a window in one wall as the relevant stimulus even when entering the room (with two identical windows) from the other side. This can be thought of as the acquisition of a

130

StephenC. Levinson

" " " " non-egocentric . ls Pick , absolute or geographic orientation or frame of reference , 35) points out , however, that such apparently allocentric behavior can be mimicked ( 1993 ' by egocentric mental operations, and indeed this is suggestedby Acredolo s , 165 ( 1988 ) observation that children learn to do such tasks by adopting the visual " if " strategy you want to find it , keep your eyeson it (as you move) . These lines of work identify the egocentric versusallocentric distinction with the . opposition between body-centered and environment-centered frames of reference But as philosophers point out (see , for example , Campbell 1993 ), ego is not just any old body, and there is indeed another way to construe the distinction as one between . The egocentricframe of referencewould subjectiveand objective frames of reference then bind together various body centeredcoordinate systems with an agentivesubjective , distinct zones of spatial interaction (reach, being, complete with body schema " " peripheral vs. central vision, etc.). For example , phenomenalike phantom limbs or proprioceptive illusions argue for the essentiallysubjectivenature of egocentriccoordinate . systems " versus" " The next distinction on our list , " viewer-centered , comes object-centered ' from the theory of vision, as reconstructed by Marr ( 1982 ) . In Marr s well-known a of vision should take us from retinal image to visual conceptualization, theory object recognition, and that , he claimed, entails a transfer from a viewer-centered " frame of reference , with incremental processing up to what he called the 2! -D " sketch, to an object centered frame of reference , a true 3-D model or structural 19Because . we can an even when foreshortenedor viewed description recognize object in differing lighting conditions, we must extract someabstract representationof it in terms of its volumetric properties to match this token to our mental inventory of such types. Although recent developments have challenged the role of the 3-D model within a modular theory of vision,2O there can be little doubt that at someconceptual level such an object-centeredframe of referenceexists. This is further demonstrated by work on visual imagery, which seemsto show that , presented with aviewer centered perspective view of a novel object, we can mentally rotate it to obtain different perspectival " views" of it , for example , to compare it to a prototype and Metzler 1971 1980 1991 ; Kosslyn (Shepard ; Tye , 83- 86). Thus at somelevel, the visual or ancillary systemsseem to employ two distinct reference frames, viewercenteredand object-centered . This distinction between viewer-centeredand object-centeredframes of reference relatesrather clearly to the linguistic distinction betweendeictic and intrinsic perspectives discussedbelow. The deictic perspectiveis viewer-centered , whereasthe intrinsic perspectiveseemsto use (at least in part) the same axial extraction that would be neededto compute the volumetric properties of objects for visual recognition (see Landau and Jackendoff 1993 ; Jackendoff, chapter 1, this volume; Landau, chapter 8,

's and Molyneux of Reference Frames Question

131

this volume; Levinson 1994 ) . This parallel will be further reinforced by the reformation in section 4.3.3. distinctions the of suggested linguistic " " " " This brings us to the orientation bound versus orientation -free frames of reference .21The visual imagery and mental rotation literature might be thought to have . After all , visual imagery would seem to be little to say about frames of reference thus 2 D and at most necessarilyin a viewer-centeredframe of reference necessarily ! to a 3-D description) . But recently there have (evenif mental rotations indicate access beenattempts to understandthe relation betweentwo kinds of shaperecognition: one where shapesare recognized without regard to orientation (thus with no response of orientation from a familiar related stimulus), curve latency associatedwith degrees are where and another shapes recognizedby apparent analog rotation to the familiar related stimulus. The Shepard and Metzler ( 1971 ) paradigm suggestedthat only where handednessinformation is present (as where enantiomorphs have to be discriminated ) would mental rotation be involved, which implicitly amounts to some distinction betweenobject-centered and viewer-centeredframes of reference ; that is discrimination of enantiomorphs dependson an orientation bound perspective , while 22 the recognition of simpler shapesmay be orientation free. But some recent controversies seemto show that things are not as simple as this (Tarr and Pinker 1989 ; in fact tasks that rotation 1985 . and Just Cohen and Kubovy 1993 ) argue ) Carpenter ( can be solved using four different strategies , someorientation -bound and someorientation -free.23 Similarly , Takano ( 1989 ) suggeststhat there are four types of spatial information involved, classifiable by crossing elementary(simple) versusconjunctive (partitionable) forms with the distinction betweenorientation boundand orientation for rotation mental forms should bound require free . He insists that only orientation ) claim that such a view makes the recognition. However, Cohen and Kubovy ( 1993 wrong predictions becausehandednessidentification can be achieved without the . In fact, I believe that despite these mental rotation latency curves in special cases the recent controversies , original assumption that only objects lacking handedness can be recognized without mental rotation - must be basically correct for logical .24In any case reasonsthat have beenclear for centuries , it is clear from this literature that the study of visual recognition and mental rotation utilizes distinctions in frames of referencethat can be put into correspondencewith those that emerge from , for . Absolute and relative frames of referencein language , the study of language example (to be firmed up below) are both orientation -bound, while the intrinsic frame is orientation -free (Danziger 1994 ). " " " " have , long distinguished deictic versus intrinsic frames of reference Linguists " of the is in front like the sentence of a of the rather obvious ambiguities because boy " see for house ; Clark 1973 ) . It has also , 168 ; Fillmore 1971 , Leech 1969 ( , example beenknown for a while that linguistic acquisition of thesetwo readingsof terms like

132

StephenC. Levinson

in front , behind , to the side of is in the reverse direction from the developmental sequenceegocentric to allocentric (Pick 1993 ) : intrinsic notions come resolutely earlier than deictic ones (Johnston and Slobin 1978 ) . Sometimesa third term, extrinsic , is opposed , to denote, for example , the contribution of gravity to the interpretation of words like aboveor on. But unfortunately the term deictic breedsconfusions. In fact there have been at least three distinct interpretations of the deictic versus intrinsic contrast, as listed in table 4.1: ( 1) speaker -centric versusnon-speaker -centric (Levelt 1989 ); (2) centered on any of the speechparticipants versus not so centered ( Levinson 1983 ); (3) ternary versus binary spatial relations (implicit in Levelt 1984 and chapter 3, this volume; to be adopted here ) . These issueswill be taken up in section4.3.3, wherewe will ask what distinctions in framesof referenceare grammaticalized or lexicalized in different languages . Let us turn now to the various distinctions suggestedin the psychology of language . Miller and Johnson- Laird ( 1976 ), drawing on earlier linguistic work , explored the opposition betweendeictic and intrinsic interpretations of such utterancesas " the cat is in front of the truck " ; the logical properties of thesetwo frames of reference , and their interaction , have beenfurther clarified by Levelt ( 1984 , 1989 , and chapter 3, this volume) . Carlson- Radvansky and Irwin ( 1993 , 224) summarize the general assumption in psycholinguisticsas follows: Threedistinctclasses of reference frames existfor representing the spatialrelationships among -centered -centered in theworld. . . viewer centered , object , andenvironment objects frames frames . In a viewer framesof reference -centered frame in a retinocentric , objectsare represented , 's -centriccoordinatesystem head -centricor body basedon the perceiver of the perspective world. In an object -centered frame arecodedwith respect to their intrinsicaxes . In an , objects -centered environment frame with respect to salientfeatures of the , objectsare represented environment . In order to talk about space , suchas gravity or prominentvisual landmarks , verticalandhorizontalcoordinate axes mustbeoriented with respect to oneof these reference " and " to the left of " can be framesso that linguisticspatialtermssuchas " above . assigned added ) (Emphasis Notice that in this formulation frames of referenceinhere in spatial perception and : above may simply be semantically general over cognition rather than in language the different frames of reference , not ambiguous (Carlson- Radvansky and Irwin 25 , 242) . Thus deictic, intrinsic , and extrinsic are merely alternative labels for ( 1993 the linguistic interpretations corresponding, respectively , to viewer-centered , objectcentered . , and environment-centeredframes of reference There are other oppositions that psycholinguists employ, although in most cases they map onto the sametriadic distinction . One particular set of distinctions, between different kinds of surveyor route description, is worth unraveling becauseit has - 144 causedconfusion. Levelt ( 1989 , 139 ) points out that when a subject describesa

's and Molyneux Frames of Reference Question

133

" " complex visual pattern, the linearization of speech requires that we chunk the . Typically , we seemto pattern into units that can be describedin a linear sequence window small a D 2 D or 3 , as it were, traversing configurations through represent into a description is converted static of the that is the array; arrays , description complex " " . Levelt of motion through units or chunks of the array (chapter 3, this volume ) has examinedthe description of 2 D arrays, and found two strategies( I ) : a gaze tour perspective , effectively the adoption of a fixed deictic or viewer-centeredperspective 2 a and , where a pathway ; ( ) body or driving tour, effectively an intrinsic perspective to used of the the direction and is found through the array , assign front , left, path and so on from anyone point (or location of the window in describing time) . Because can be thought of as egocentric both perspectives ; seealso Taylor and , Tversky ( 1991 Levelt' s intrinsic to call 12 this volume in and , , ) opts Tversky chapter Tversky press " " " " his deictic perspective perspectivea deictic frame of reference or route description and ' s " intrinsic " or " ' " " " 26 is Levelt . Thus Tversky s deictic a surveyperspective ! This confusion is, I believe nondeictic perspective , not merely terminological but results from the failure in the literature to distinguish coordinate systemsfrom their origins or centers(seesection 4.3.3) . , there seemsto Finally , in psycholinguistic discussionsabout frames of reference be someunclarity , or sometimesovert disagreement , at which level perceptual, conceptual or linguistic- such frames of referenceapply . Thus Carlson- Radvansky and Irwin ( 1993 , 224) make the assumption that a frame of referencemust be adopted within somespatial representationsystem , as a precondition for coordinating perception and language ; but seeLevelt, chapter 3, this volume) has , whereasLevelt ( 1989 chosenin the very processof mapping from is of reference that a frame freely argued to or perception spatial representation language(seealso Logan and Sadier, chapter 13, this volume) . On the latter conception, frames of referencein languageare peculiar to the nature of the linear, propositional representation system that underlies , that is, they are different ways of conceiving the samepercept in linguistic semantics order to talk about it .27 The view that frames of reference in linguistic descriptions are adopted in the mapping from spatial representationor perception to languageseemsto suggestthat make no useof frames of reference the perceptionsor spatial representationsthemselves : there has to be some coordinate system . But this of course is not the case involved in any spatial representationof any intricacy , whether at a peripheral (sensory ' ) level or at a central (conceptual) level. What Levelt s results (chapter 3, this ' volume) or Friederici and Levelt s ( 1990 , is that framesof reference ) seemto establish at the perceptual or spatial conceptual level do not necessarilydetermine frames of referenceat the linguistic level. This is exactly what one might expect. Language is flexible and it is an instrument of communication- thus it naturally allows us, for

134

StephenC. Levinson

' . Further , the ability to cast a description , to take the other person s perspective example in one frame or another implies an underlying conceptual ability to handle multiple frames, and within strict limits (seebelow) to convert betweenthem. In any case , we need to distinguish in discussionsof frames of referencebetween at least three levels: ( I ) perceptual, (2) conceptual, and (3) linguistic; and we needto consider the possibility that we may utilize distinct frames of referenceat each level (but see section 4.4) . There is much further pertinent literature in all the branches of psychology and brain science , but we must leave off here. It should already be clear that there are , and different construals of the sameterms, many, confusingly different classifications not to mention many unclarities and many deep confusions in the thinking behind them. Nevertheless , there are someobvious common basesto the distinctions we have " " reviewed . It is clear for example , that on the appropriate construals, egocentric " " " " " " " correspondsto viewer-centered and 2; -0 sketch to deictic frame, while intrinsic " " " " " " ; absolute maps onto object-centered or 3-D model frames of reference " and so forth . We should seizeon these " is related to " environment-centered ; in this chapter we are concernedwith making sense commonalities, especiallybecause " across modalities and . of the " same frame of reference representational systems However, before proposing an alignment of thesedistinctions acrossthe board, it is essentialto deal with linguistic framesof reference , whosetroubling flexibility has led to various confusions. in Croalinguistic Perspective 4.3.3 Linguistic Framesof Reference will give the impression that the linguists literature of the linguistic inspection Cursory have their house in order. They talk happily of topological vs. projective spatial relators (e.g., prepositions like in vs. behind), deictic versus intrinsic usages of projective prepositions, and so on (see ; , Bierwisch 1967 ; Lyons 1977 , for example Herskovits 1986 ; Miller and ; and psycholinguists Clark 1973 ; Vandeloise 1991 Johnson- Laird 1976 ) . But the truth is lesscomforting . The analysis of spatial terms 28 in familiar European languages remains deeply confused , and those in other languagesalmost entirely unexplored. Thus the various alleged universals should be taken with a great pinch of salt (in fact, many of them can be directly jettisoned) . " " One major upset is the recent finding that many languagesuse an absolute frame of reference(as illustrated in the caseof Tzeltal) where European languageswould " use a " relative or viewpoint-centered one (see , b; , Levinson I 992a , for example Haviland 1993 , usesuch , like many Australian ones ). Another is that somelanguages frames of referenceto replace so-called topological notions like in, on, or under. A third is that familiar spatial notions like left and right and even sometimes front and . Confident of all a third back are missing from many, perhaps predictions languages

Frames of Referenceand Molyneux ' s Question

135

and assumptionscan be found in the literature that no such languagescould occur , for example , Clark 1973 ; Miller and Johnson- Laird 1976 (see ; Lyons 1977 , 690) . Thesedevelopmentscall for some preliminary typology of the frames of reference that are systematicallydistinguished in the grammar or lexicon of different languages (with the caveat that we still know little about only a few of them) . In particular, we shall focus on what we seemto needin the way of coordinate systemsand associated referencepoints to set up a cross linguistic typology of the relevant frames of reference . In what follows I shall confine myself to linguistic descriptions of static arrays, and I shall excludethe so-called topological notions, for which a new partial typology concerning the coding of concepts related to in and on is available (Bowerman and Pedersonin prep.) .29Moreover, I shall focus on distinctions on the horizontal plane. This is not whimsy: the perceptual cuesfor the vertical may not always coincide, but , giving us a good universal solution to one axis. But they overwhelmingly converge the two horizontal coordinates are up for grabs: there simply is no corresponding force like gravity on the horizontal .3oConsequentlythere is no simple solution to the description of horizontal spatial patterns, and languagesdiverge widely in their solutions to the basic problem of how to specify anglesor directions on the horizontal . , three main frames of referenceemergefrom thesenew findings as solutions Essentially to the problem of description of horizontal spatial oppositions. They are appropriately named " intrinsic ," " relative" and " absolute," even though theseterms may have a somewhatdifferent interpretation from someof the construals reviewedin the section above. Indeed, the linguistic frames of referencepotentially crosscutmany of the distinctions in the philosophical, neurophysiological, linguistic, and psychological literatures, for one very good reason . Linguistic framesof referencecannot be defined with respect to the origin of the coordinate system (in contrast to , for example , egocentricvs. allocentric) . It will follow that the traditional distinction deictic versus - theseare not opposedterms. All this requires some intrinsic collapses explanation. We may start by noting the difficulties we get into by trying to make the distinction between deictic and intrinsic . Levelt ( 1989 , 48- 55) organizes and summarizes the standard assumptionsin a useful way: we can cross - classify linguistic usesaccording to (a) whether they presumethat the coordinates are centeredon the speaker(deictic) or not (intrinsic); and (b) whether the relatum (ground) is the speakeror not. Suppose then we call the usage " deictic" just in case the coordinates are centered on the " " , intrinsic otherwise. This yields, for example , the following classification of speaker : examples ( I ) The ball is in front of me. Coordinates: Deictic (i.e., origin on speaker ) Relatum: Speaker

136
(2) The ball is in front of the tree. Coordinates: Deictic (i.e., origin on speaker ) Relatum: Tree ' (3) The ball is in front of the chair (at the chair s front ) . Coordinates: Intri . ic (i.e., origin not on speaker ) Relatum: Chair

C. Levinson Stephen

Clearly, it is the locus of the origin of the coordinates that is relevant to the traditional opposition deictic versusintrinsic , otherwise we would group (2) and (3) as both sharing a nondeictic relatum. The problem comeswhen we pursue this classi fication further : (4) The ball is in front of you . Coordinates: Intri . . ic (origin on addressee,, not speaker) Relaturn: Addressee (5) The ball is to the right of the lamp, from your point of view. Coordinates: Intri _ ic (origin on addressee ) Relatum: Lamp Here the distinction deictic versusintrinsic is self-evidently not the right classification, as far as frames of referenceare concerned . Clearly, ( I ) and (4) belong together: the is the same , with the samecoordinate systems ; there interpretation of the expressions arejust different origins- speakerand addressee , respectively(moreover, in a normal " " construal of " deictic," inclusive of first and secondpersons , both are deictic origins ) . Similarly, in another grouping, (2) and (5) should be classedtogether: they have the sameconceptual structure, with a viewpoint (acting as the origin of the coordinate ), a relatum distinct from the viewpoint , and a referent- again the origin system alternatesover speakeror addressee . We might therefore be tempted simply to alter the designations , and label ( I ), (2), " " " " (4), and (5) all deictic as opposed to (3) intrinsic . But this would produce a further confusion. First , it would conftate the distinct conceptual structures of our groupings ( I ) and 4 , the conceptual structure of the coordinate systemsin ( ) versus(2) and (5) . Second " " ( I ) and (4) is in fact shared with (3) . The ball is in front of the chair presumes(on the relevant reading) an intrinsic front and usesthat facet to define a searchdomain for the ball ; but just the sameholds for " the ball is in front of me/you." 31Thus the : the notion " in front of " is here a logical structure of ( I ), (3), and (4) is the same binary spatial relation , with arguments constituted by the figure (referent) and the ground (relatum), where the projected angle is found by referenceto an intrinsic or inherent facet of the ground object. In contrast, (2) and (5) have a different logical

Frames of Referenceand Molyneux ' s Question

137

structure: " in front of " is here a ternary relation , presuming a viewpoint V (the origin 32 of the coordinate system ), a figure, and ground, all distinct. In fact, thesetwo kinds of spatial relation have quite different logical properties, as demonstrated elsewhere , and chapter 3, this volume), but only when distinguished and by Levelt ( 1984 " " in this grouped way. Let us dub the binary relations intrinsic , but the ternary " " relations relative (becausethe descriptions are always relative to a viewpoint, in contradistinction to " absolute" and " intrinsic " descriptions ). To summarize then, the proposed classification is ' ( I ) The ball is in front of me Coordinates: Intri . . ic Origin : Speaker Relatum: Speaker ' ' (3 ) The ball is in front of the chair (at the chair s front ) Coordinates: Inm. . ic Origin : Chair Relatum: Chair ' (4 ) The ball is in front of you Coordinates: Inm. . ic Origin : Addressee Relatum: Addressee ' (2 ) The ball is in front of the tree Coordinates: Relative Origin : Speaker Relatum: Tree ' (5 ) The ball is to the right of the lamp, from your point of view Coordinates: Relative Origin : Addressee Relatum: Lamp ' (6 ) John noticed the ball to the right of the lamp For John, the ball is in front of the tree. Coordinates: Reladve Origin : Third person (John) Relatum: Lamp (or Tree) Note that useof the intrinsic systemof coordinates entails that relatum (ground) and origin are constituted by the sameobject (the spatial relation is binary , betweenFand G ), while use of the relative system entails that they are distinct (the relation is

138

StephenC. Levinson

ternary, betweenF, G, and viewpoint V ) . Note , too , that whether the center is deictic, that is, whether the origin is speaker(or addressee ), is simply irrelevant to this classifi' ' ' cation. This is obvious in the caseof the grouping of ( I ), (3 ), and (4 ) together. It is -centric, it also clear that although the viewpoint in relative usesis normally speaker in (6' ) . illustrated as third on a centered or even - centric party may easily be addressee Hence deictic and intrinsic are not opposed ; instead, we need to oppose coordinate systemsas intrinsic versusrelative, on the one hand, and origins as deictic and non deictic (or , alternatively, egocentric vs. allocentric), on the other. Becauseframes of reference are coordinate systems , frames of reference , it follows that in language but variable , origins. cannot be distinguished according to their characteristic, I expect a measure of resistanceto this reformation of the distinctions, if only " " because the malapropism deictic frame of reference has become a well-worn phrase. How , the critic will argue, can you define the frames of referenceif you no longer employ the feature of deicticity to distinguish them? I will expendconsiderable effort in that direction in section4.3.3.2. But first we must compare thesetwo systems with the third systemof coordinates in natural language , namely, absolute frames of . Let us review them together. reference 4.3.3.1 The Three Linguistic Framesof Reference As far as we know , and according to a suitably catholic construal, there are exactly three frames of referencegrammaticalized or lexicalized in language (often, lexemesare ambiguous over two of 33 these frames of reference , sometimes expressionswill combine two frames, but 34 often each frame will have distinct lexemesassociatedwith it ) . Each of thesethree es a whole family of related but distinct semantic frames of reference encompass 3SIt is true to . say that eventhe most closely related languages(and probably systems in the details of the underlying coordinate systems will differ them within even dialects ) , the and their geometry, the preferential interpretation of ambiguous lexemes presumptive origins of the coordinates, and so on. Thus the student of languagecan expect that expressionsglossed as, say, intrinsic side in two languageswill differ considerably in the way in which sideis in fact determined, how wide and how distant . a searchdomain it specifies , and so on. With that caveat, let us proceed .36 for the description of all systems Let us first define a set of primitives necessary The application of some of the primitives is sketchedin figure 4.9, which illustrates . Minimally , three canonical exemplarsfrom each of our three main types of system . in passing will illustrate which we of we need the primitives in table 4.2, the use clas Combinations of theseprimitives yield a large family of systemswhich may be : ( I ) intrinsic frame of reference ; (2) relative sified in the following tripartite scheme . 3 absolute and ; frame of reference ( ) frame of reference

Frames of Referenceand Molyneux ' s Question

INTRINSIC
"He's In front of the house."

X . X G ~ F

139

RELATIVE

"He 's to theleftofthehouse ."

L .1 ~ : : B i~ E G

ABSOLUTE
- He's north of the house."

~
~

4.9 Figure Canonical . examplesof the three linguistic frames of reference

140 Table 4.2 Inventory of Primitives

StephenC. Levinson

1. Systemof labeled angles -specific Labeled arcs are specifiedby coordinates around origin (language ); such labels may or may not form a fixed armature or template of oppositions. 2. Coordinates a. Coordinates may be polar, by rotation from a fixed x -axis, or rectangular, by specification of two or more axes ; b. One primary coordinate systemC can be mapped from origin X to secondaryorigin X2, by the following transformations: . translation , . rotation . reflection . (and possibly a combination ) to yield a secondarycoordinate systemC2. 3. Points F = figure or referent with center point at volumetric center Fc. G = ground or relatum, with volumetric center Gc, and with a surrounding region R V = viewpoint X = origin of the coordinate system , X2 = secondaryorigin A = anchor point , to fix labeled coordinates L = designatedlandmark 4. Anchoring system A = Anchor point , for example , with G or V; in landmark systemsA = L . " " , yielding parallel lines acrossenvironment in eachdirection Slope = fixed-bearing system

Intrinsic Frame of Reference Informally , this frame of referenceinvolves an object" centeredcoordinate system , where the coordinates are determined by the inherent " features or facets of the object to be used as the ground or relatum. The , sidedness " inherent features" , though widely used in the literature , is misleading: such phrase " facets " as we shall call them have to be , , conceptually assignedaccording to some or learned on a case -case basis . , , or more often a combination of these algorithm by The procedure varies fundamentally across languages . In English, it is (apart from top and bottom, and specialarrangementsfor humans and animals) largely functional , for example , the sketch in Miller and Johnson- Laird 1976 , 403), so that thefront (see of a TV is the side we attend to , while thefront of a car is the facet that canonically lies in the direction of motion , and so forth . But in some languages , it is much more . For example , in Tzeltal the assignment of sides utilizes a closely based on shape volumetric analysis very similar to the object-centeredanalysis proposed by Marr

Frames of Referenceand Molyneux ' s Question

( 1982 ) in the theory of vision, and function and canonical orientation is largely 37 irrelevant (seeLevinson 1994 ) . In many languagesthe morphology makes it clear that human or animal body (and occasionally plant) parts provide a prototype for " " " " " " : hence we talk about the " front , " backs, sides the opposed sides , lefts, and " and in " heads" " feet " " horns " " roots " etc. of other " , , , , ) many languages rights 38 , it relies primarily on objects. But whatever the procedure in a particular language the conceptual properties of the object: its shape , canonical orientation , characteristic motion and use , and so on. The attribution of such facets provides the basis for a coordinate systemin one of two ways. Having found , for example , thefront , this may be used to anchor a readymade 39 , and so forth . Alternatively , in other systemof oppositionsfront / back, sides , there may be no such fixed armature, as it were, each object having parts languages determined, for example, by specific shapes , finding front does not predict ; in that case from the volumetric determines a direction the locus of back, but nevertheless center of the object through thefront , which can be used for spatial description.4OIn either case , we can use the designatedfacet to extract an angle, or line, radiating out " from the ground object, within or on which the figure object can be found (as in the " statue in front of the town hall ) . The geometrical properties of such intrinsic coordinate systemsvary crosslinguis tically . Systemswith fixed armatures of contrastive expressionsgenerally require the angles projected to be mutually exclusive (nonoverlapping), so that in the intrinsic to say, " The cat is to the frame of reference(unlike the relative one) it makesno sense " front and to the left of the truck . Systemsutilizing single parts make no such constraints " " (cf. The cat is in front of , and at the foot of , the chair ) . In addition , the metric extent of the searchdomain designated(e.g., how far the cat is from the truck ) can vary greatly. Some languages require figure and ground to be in contact, or " visually continuous, others allow the projection of enormous search domains ( in " front of the church lie the mountains, running far off to the horizon ) . More often ' , the notion of a region, an object s penumbra, as it were, is relevant, related perhaps .41 to its scale More exactly An intrinsic spatial relation R is a binary spatial relation , with arguments F and G, where R typically namesa part of G. The origin X of the coordinate system C is always on (the volumetric center of ) G. An intrinsic relation R (F, G ) assertsthat F lies in a searchdomain extending from G on the basis of an angle or line projected from the center of G, through an anchor point A (usually the named facet R), outwards for a determined distance. F and G may be any objects whatsoever (including ego), and F may be a part of G. The relation R does not support transitive inferences , nor converseinferences(seebelow) .

142

StephenC. Levinson

Coordinates mayor may not come in fixed armatures. When they do , they tend to be polar; for example , given that facet A is thefront of a building , clockwise rotation in 900stepswill yield side, back, side. Here there is a set of four labeled oppositions, with one privileged facet, A. Given A , we know which facet back is. Because A fixes the coordinates, we call it the " anchor point ." But coordinates need not be polar , or indeed part of a fixed set of oppositions; for example , given that facet B is the entranceof a church and Gcits volumetric center, we may derive a line BGc(or an arc with angle determined by the width of B)- thus " at the entrance to the church" designatesa searcharea on that line (or in that arc), with no necessary implications about the locations of other intrinsic parts, front , back, and so on. Because A determines the line, we call A once again the " anchor point ." Relati, e Frame of Reference This is roughly equivalent to the various notions of viewer-centeredframe of referencementioned above (e.g., Marr ' s " 21-0 sketch," or the psycholinguists " deictic" ), but it is not quite the same . The relative frame of referencepresupposes a " viewpoint " V (given by the location of a perceiver in any sensorymodality), and a figure and ground distinct from V; it thus offers a triangulation of three points and utilizes coordinates fixed on V to assigndirections to figure and ground . English " The ball is to the left of the tree" is of this kind of course. Becausethe perceptual basis is not necessarilyvisual, calling this frame of reference " is " viewer-centered potentially misleading, but perhaps innocent enough. Calling it " deictic " however is " " , , potentially pernicious becausethe viewer need not be ego " and neednot be a participant in the speech event- take, for example , Bill kicked the ball to the left of the goal." Nevertheless , there can be little doubt that the deictic uses of this systemare basic (prototypical ), conceptually prior , and so on. The coordinate system , centered on viewer V, seemsgenerally to be basedon the planesthrough the human body, giving us an up/ down, back/front and left/right set of half lines. Such a system of coordinates can be thought of as centered on the main axis of the body and anchored by one of the body parts (e.g., chest ) . In that casewe have polar coordinates, with quadrants counted clockwise from front to right , back, and left (Herskovits 1986 ) . Although the position of the body of viewer V may be one criterion for anchoring the coordinates, the direction of gaze may be another, and there is no doubt that relative systemsare closely hooked into visual criteria. Languages , the extent to may differ in the weight given to the two criteria , for example which occlusion plays a role in the definition of behind . But this set of coordinates on V is only the basis for a full relative system ; in addition , a secondary set of coordinates is usually derived by mapping (all or some of ) the coordinates on V onto the relatum (ground object) G. The mapping involves a transformation which may be 1800rotation , translation (movement without rota -

' Frames of Referenceand Molyneux s Question

143

tion or reflection), or arguably reflection across the frontal transverseplane. Thus " the cat is in front of the tree" in English entails that the cat F is between V and G on V appear to have been rotated in the coordinates the because the tree , primary ( ) " front " before which the cat sits. Hausa Hill 1982 a that G has G so onto ) ( , mapping so that a the coordinates than rotate rather , and many other languages translate " sentenceglossing " The cat is in front of the tree will mean what we would mean in " " English by The cat is behind the tree. But English is also not so simple, for rotation " " will get left and right wrong . In English, The cat is to the left of the tree has left on ' the sameside as V s left , not rotated. In Tamil , the rotation is complete; thus just as front and back are reversed glossed , so are left and right , so that the Tamil sentence " The cat is on the left side of the tree" would on the relevant interpretation) mean ( " " The cat is on V ' s right of the tree. To get the English system right , we might supposethat the coordinates on V should be reflectedover the transverseplane, as if we wrote the coordinates of Von a sheetof acetate , flipped it over in front of V, and placed it on G. This will get front , back, left , and right at least in the correct polar around the secondaryorigin . But it may not be the correct solution because sequence 42 other interpretations are possible , and indeed more plausible. But the point to establish here is that a large variation of systemsis definable, constituting a broad . family of relative systems Not all languageshave terms glossing left/right , front / back. Nor does the possession . of such a system of oppositions guarantee the possessionof a relative system when intrinsic even in more or less terms a use such they way ( purely Many languages are primarily used with deictic centers ); that is, they are used as binary relations " (as in to my specifying the location of Fwithin a domain projected from a part of G " ' " " ' " " " " left , in front of you, at the animal s front , at the houses front , etc.) . The test for a relative systemis ( I ) whether it can be usedwith what is culturally construed as a ground object without intrinsic parts,43 and (2) whether there is a ternary relation with viewpoint V distinct from G, such that when V is rotated around the array, the description changes(seebelow) . Now , languagesthat do indeed have a relative system of this kind also tend to havean intrinsic systemsharing at least someof the same terms.44This typo logical implication , apart from showing the derivative and secondary nature of relative systems , also more or lessguaranteesthe potential ambiguity of back systems(although they may be disambiguated syntactically, as left/right , front / " ' in " to the left of the chair " vs. " at the chair s left ) . Some languages that lack have encoded the odd isolated any such systematicrelative systemmay nevertheless " " relative notion , as in F is in my line of sight toward G. That somerelative systemsclearly use secondarycoordinates mapped from V to G that thesemappings are by origin a meansof extending the intrinsic frame suggests of referenceto caseswhere it would not otherwise apply . (And this may suggestthat

144

StephenC. Levinson

the intrinsic systemis rather fundamental in human linguistic spatial description.4S ) Through projection of coordinates from the viewpoint V, we assign pseudointrinsic facets to G, as if trees had inherent fronts, backs, and sides .46 For some languages , this is undoubtedly the correct analysis ; the facets are thus named and regions projected with the samelimitations that hold for intrinsic regions.47Thus many relative - systemsthat utilize relative systemscan be thought of as derivedintrinsic ones relations to extend and conceptual supplement intrinsic ones. One particular reason to so extend intrinsic systemsis their extreme limitations as regards logical inference of spatial relations from linguistic descriptions. Intrinsic descriptions support neither transitive nor converseinferences , but relative ones do (Levelt 1984 , chapter 3, this volume; and seebelow).48

More exactly A relative relator R express es a ternary spatial relation, with arguments V, F, and G, where F and G are unrestricted as to type, except that V and G must be distinct.so The primary coordinate systemalways has its origin on V; there may be a secondarycoordinate systemwith origin on G. Such coordinate systemsare ,front , right, back, and left may be assignedby clockwise normally polar; for example rotation fromfront . Coordinate systemsbuilt primarily on visual criteria may not be polar , but be defined, for example , by rectangular coordinates on the two-dimensional visual field (the retinal projection) so that left and right are defined on the horizontal or x -axis, and front and bac on the vertical or y -axis (back has (the base ~ of ) F higher than G and/ or occluded by G ) . Terms that may be glossed left and right may involve no secondary coordinates , although they sometimesdo (as when they have reversed application from the English usage ). Terms glossedfront and back normally do involve secondary coordinates (but compare the analysis in terms of vectors by O' Keefe, chapter 7, this volume) . Secondarycoordinates may be mapped from primary origin on V to secondary origin on G under the following transformations: rotation , translation, and (arguably) reflection.51Typo logical variations of such systemsinclude degreeto

' Frames of Reference and Molyneux s Question

145

which a systematicpolar systemof coordinates is available, degreeof use of secondary coordinates, type of mapping function (rotation , translation , reflection) for secondary coordinates, differing anchoring systemsfor the coordinates (e.g., body axis vs. gaze ), and differing degreesto which visual criteria (like occlusion, or place in retinal field) are definitional of the terms. " " AbsoluteFrame of Reference Among the many usesof the notion absolute frame of reference , one refers to the fixed direction provided by gravity (or the visual horizon under canonical orientation ) . Lessobviously of psychologicalrelevance , the same idea of fixed directions can be applied to the horizontal . In fact, many languages make extensive , useof such an absolute frame of referenceon , somealmost exclusive " " the horizontal . They do so by fixing arbitrary fixed bearings , cardinal directions, corresponding one way or another to directions or arcs that can be related by the . Speakersof such languagescan then describean array analyst to compassbearings " of , for example , a spoon in front of a cup, as spoon to north / south/ east / (etc.) of ' " cup without any referenceto the viewer/speakers location. Such a systemrequires that personsmaintain their orientation with respectto the fixed bearings at all times. People who speak such languagescan be shown to do so- for example , they can dead reckon current location in unfamiliar territory with , and thus point to any named location from any other ( Lewis extraordinary accuracy 1976 ; Levinson 1992b ) . How they do so is simply not known at the present time, but we may presume that a heightened senseof inertial navigation is regularly cross 52 such are Indeed checked with many environmental clues. , many systems clearly abstractions and refinements from environmental gradients (mountain slopes , prevailing " 53 wind directions, river drainages , celestial azimuths, etc.) . These cardinal directions" may therefore occur with fixed bearings skewedat various degreesfrom , " and in effect unrelated to , our " north ," ' south," " east," and " west. It perhapsneeds emphasizingthat this keeping track of fixed directions is, with appropriate socializa tion , not a feat restricted to certain ethnicities, races , environments, or culture types, ?) as shown by its widespreadoccurrence(in perhaps a third of all human languages from Meso-America, to New Guinea, to Australia , to Nepal . No simple ecological determinism will explain the occurrenceof such systems , which can be found alternating with , for example , across neighboring ethnic groups in similar , relative systems environments, and which occur in environmentsof contrastive kinds (e.g., wide open desertsand closedjungle terrain) . The conceptual ingredients for such systems are simple: the relevant linguistic expressionsare binary relators, with figure and ground as arguments and a system of coordinates anchored to fixed bearings , which always have their origin on the In are the . fact these , only systemswith conceptual simplicity and systems ground

146

StephenC. Levinson

. For example, they are the only systemsthat fully support transitive inferences elegance acrossspatial descriptions. Intrinsic descriptionsdo not do so, and relative ones do so only if viewpoint V is held constant (Levelt 1984 ) . Intrinsic systemsare dogged by the multiplicity of object types, the differing degreesto which the asymmetriesof " " , and the problem of unfeatured objects. Relative objectsallow the naming of facets systemsare dogged by the psychological difficulties involved in learning left/right distinctions, and the complexities involved in mapping secondarycoordinates; often developedfrom intrinsic systemsthey display ambiguities acrossframes of reference " " (like English in front of ) . The liabilities of absolute systemsare not , on the other hand, logical but psychological; they require a cognitive overhead , namely the constant calculation of cardinal directions with a systemof dead , together background will that for P which direction P is from ego' s reckoning specify any arbitrary point current locus (so that ego may refer to the location of P) . Absolute systemsmay also show ambiguities of various kinds. First , places of particular sociocultural importance may come to be designatedby a cardinal direction term, like a quasi-proper name, regardlessof their location with respect to G. Second , where the systemis abstractedout of landscapefeatures , the relevant expressions " " " " (e.g., uphill or upstream ) may either refer to placesindicated by relevant local features(e.g., local hill , local stream), or to the abstractedfixed bearings , where these do not coincide. Third , some such systemsmay even have relative interpretations " " (e.g., uphill may imply further away in my field of vision; cf. our interpretation of " north " as top of a map) . One crucial question with respect to absolute systemsis how, conceptually, the coordinate system is thought of . It may be a polar system , as in our north/south/ east west where north is the anchor and east south , , , west , found by clockwise / designated 54 rotation from north. Other systemsmay have a primary and a secondaryaxis, so that , for example , a north-south axis is primary , but it is not clear which direction , north or south, is itself the anchor.55Yet other systemsfavor no particular primary referencepoint , each half axis having its own clear anchor or fixed central bearing.56 " Some systemslike Tzeltal are " degenerate , in that they offer two labeled half lines " north " " south" but label both ends of the , (roughly, ), orthogonal with the same terms. Even more confusing, some systemsmay employ true abstracted cardinal directions on one axis, but landmark designationson the other, guaranteeingthat the two axes do not remain orthogonal when arrays are described in widely different . Thus on Bali , and similarly for many Austronesian systems , one axis is determined places and monsoons is a fixed abstracted axis but the other is determined , , by by the location of the central mountain and thus varies continuously when one circumnavigates the island. Even where systematiccardinal systemsexist, the geometry of the designatedangles is variable. Thus, if we have four half lines based on orthogonal

Framesof Referenceand Molyneux ' s Question

147

axes , the labels may describe quadrants (as in Guugu Yimithirr ), or they may have narrower arcs of application on one axis than the other (as appearsto be the casein Wik Mungan S7 ) . Even in English, though we may think of north as a point on the horizon , we also usearcs of variable extent for informal description. More exactly An absolute relator R express es a binary relation betweenF and G, that F can be in found a search domain at the fixed bearing R from G. The asserting X of the coordinate is origin system always centered on G. G may be any object whatsoever , including ego or another deictic center; F may be a part of G. The geometry of the coordinate systemis linguistically/culturally variable, so that in some systemsequal quadrants of 90 degreesmay be projected from G, while in others something more like 45 degreesmay hold for arcs on one axis, and perhaps 135 on the other. The literature also reports abstract systemsbasedon star-setting degrees points, which will then have unevendistribution around the horizon. Just as relative relators can be understood to map designatedfacets onto ground " " objects (thus on the front of the tree assignsa named part to the tree), so absolute relators may also do so. Many Australian languageshave cardinal edge roots, then " affixes indicating , for example ." Some of these stems can only be , northern edge analyzed as an interaction between the intrinsic facets of an object and absolute directions. 4.3.3.2 " Logical Structure" of the Three Frames of Reference We have argued that , as far as language is concerned , we must distinguish frame of referencequa coordinate systemfrom , say, deictic center qua origin of the coordinate system . Still , the skeptical may doubt that this is either necessary or possible . First , to underline the necessity , each of our three frames of referencemay occur with or without a deictic center (or egocentricorigin) . Thus for the intrinsic frame, we can say, " The ball is in front of me" (deictic center); for the absolute frame we can " " " say, The ball is north of me ; and of course in the relative frame, we can say, The " ' ball is in front of the tree (from ego s point of view) . Conversely , none of the three frames needhave a deictic center. Thus in the intrinsic frame one can say " in front of the chair" ; in the absolute frame, " north of the chair" ; and in the relative frame, " in front of the tree from Bill ' s point of view." This is just what we should expect given - it follows from Hockett ' s ( 1960 the flexible nature of linguistic reference ) design ' feature of displacement , or Buhler s ( 1934 ) concept of transposeddeictic center. Second , we need to show that we can in fact define the three frames of reference adequately without reference to the opposition deictic versus nondeictic center or origin . We have already hinted at plenty of distinguishing characteristics for each of the three frames. But to collect them together, let us first consider the logical

148

StephenC. Levinson

properties. The absolute and intrinsic relators sharethe property that they are binary relations whereasrelative relators are ternary. But absolute and intrinsic are distinguished in that absolute relators define asymmetric transitive relations (if F1 is north of G, and F2 is north ofF l ' then F2 is north of G ), whereconverses can be inferred (if Fis north of G, G is south ofF ) . The samedoes not hold for intrinsic relators, which hardly support any spatial inferencesat all without further assumptions(seeLevelt 1984and chapter 3, this volume) . In this case , absolute and relative relators share logical features becauserelative relators support transitive and converseinferences provided that viewpoint V is held constant. Although this is already sufficient to distinguish the three frames, we may add further distinguishing factors. Certain important properties follow from the nature of the anchoring system in each case . In the intrinsic casewe can think of the named facet of the object as providing the anchor; in the relative casewe can think of the , with the anchor being constituted by, say, the direction viewpoint Von an observer ' s front or of the observer , while in the absolute caseone or more of the labeled gaze fixed bearings establish es a conceptual " slope" across the environment, thus fixing the coordinate system . From this, certain distinct properties under rotation emergeas illustrated in figure 4.10.58Theseproperties have a special importance for the study of nonlinguistic conceptual coding of spatial arrays becausethey allow systematic ; Brown and experimentation (as illustrated in section 4.1; seealso Levinson 1992b Levinson 1993b ; Pederson1993 , 1994 ; Danziger 1993 ). Altogether then, we may summarize the distinctive features of each frame of reference as in table 4.3; these features are jointly certainly sufficient to establish the nature of the three frames of referenceindependently of referenceto the nature of the origin of the coordinate system . We may conclude this discussionof the linguistic frames of referencewith the following observations : I . Languages use : absolute, intrinsic , and , it seems , just three frames of reference relative; 2. Not all languagesuse all frames of reference ; some use predominantly one only (absolute or intrinsic ; relative seemsto require intrinsic ); some usetwo (intrinsic and relative, or intrinsic and absolute ), while some useall three; 3. Linguistic expressionsmay be specializedto a frame of reference , so we cannot assumethat choice of frame of referencelies entirely outside language , for example , in spatial thinking , as some have suggested . But spatial relators may be ambiguous (or semantically general ) acrossframes, and often are. 4.3.3.3 Realigning Frames of Referenceacroa Disciplines and Modalities Weare now at last in a position to seehow our three linguistic frames of referencealign with

no

no

yes no

description ? ?

same

array whole

object

Fiaure 4.10 Properties of the frames of referenceunder rotation .


NZ ~ yes yes " chair of north to ball " Absolute Z ~ A yes no " chair of left to ball " Relative 0 5 ! l o JJ yes fj " chair of front in ball " description ? description Intrinsic same same ground viewer

Frames of Referenceand Molyneux ' s Question

Rotation of: 149

150

StephenC. Levinson

Intrinsic Relation is Origin on Anchored by Transitive? Constant under rotation of whole array? viewer? ground? Yes Yes No

Absolute

Relative ternary viewpoint V A within V Yes if V constant No No Yes

binary ground A within No

binary ground " " slope Yes No Yes Yes

the other distinctions in the literature arising from the consideration of other modalities (as listed in table 4.1). The motive, let us remember , is to try to make senseof " acrossmodalities and in the very idea of " sameframe of reference , particular from various kinds of nonlinguistic thinking to linguistic conceptualization. An immediate difficulty is that , by establishingthat frames of referencein language should be consideredindependently of the origin of the coordinate systems , we have a between and the various modalities where the , openedup gulf language perceptual of the coordinate is so often fixed on some -center. But this mismatch origin system ego is in fact just as it should be. Languageis a flexible instrument of communication ' , designed(as it were) so that one may expressother persons points of view, take other perspectives , and so on. At the level of perception, origin and coordinate system come , and perhaps presumably prepackagedas a whole, but at the level of language more generally at the level of conception, they can vary freely and combine. So to realign the linguistic distinctions with distinctions made across other modalities, we need to fix the origin of the coordinate systemso that it coincides , or fails to coincide, with ego in each frame of reference . We may do so as follows. First , we , though not necessarilyegocentric , may concedethat the relative frame of reference is prototypically so. Second , we may note that the intrinsic systemis typically , but not . Third , and perhaps most arbitrarily , we may assigna definitionally , non-egocentric non-egocentric origin to the absolute system . These assignmentsshould be understood as special subcases of the usesof the linguistic frames of reference . If we make these restrictions, then we can align the linguistic frames of reference with the other distinctions from the literature as in table 4.4.59 Notice then that , under the restriction concerning the nature of the origin :

Frames of Reference and Molyneux ' s Question

4.4 Table Classifications of Frames of Reference Aligning


Intrinsic Origin ~ ego Object-centered Intrinsic perspective 3-D model Allocentric Orientation -free Allocentric Orientation -bound Absolute Origin ~ ego Environment-centered Relative Origin = ego Viewer-centered Deictic perspective 21-D sketch Egocentric Orientation -bound

I . Intrinsic and absolute are grouped as allocentric frames of reference , as opposed to the egocentricrelative system ; 2. Absolute and relative are grouped as orientation -bound, as opposed to intrinsic , which is orientation -free. This correctly captures our theoretical intuitions . In certain respects , absolute and intrinsic viewpoints are fundamentally similar- they are binary relations that are , where the origin may happen to be ego but neednot be; they viewpoint -independent are allocentric systemsthat yield an ego-invariant picture of the " world out there." On the other hand, absolute and relative frameworks are fundamentally similar on another dimension because they both imposea larger spatial framework on an assemblage , specifyingits orientation with respectto external coordinates; thus in an intrinsic framework it is impossible to distinguish enantiomorphic pairs, while in either of the orientation -bound systemsit is inevitable.6O Absolute and relative frameworks a Newtonian or Kantian spatial envelope presuppose , while the intrinsic framework is Leibnizian. The object-centerednature of the intrinsic systemhooks it up to Marr ' s ( 1982 ) 3-D model in the theory of vision, and the nature of the linguistic expressionsinvolved that the intrinsic framework is a generalization from the analysis of objects suggests into their parts. A whole configuration can be seenas a singlecomplex object, so that we can talk of the leading car in a convoy as " the head of the line." On the other hand, the viewer-centerednature of the relative framework connectsit directly to the of 2-D representationsin the theory of vision. Thus the spatial frameworks sequence in the perceptual systems can indeed be correlated with the linguistic frames of reference . To summarize , I have sought to establish that there is nothing incoherent in the " acrossmodalities notion " sameframe of reference or inner representationsystems . Indeed, even the existing distinctions that have been proposed can be seenin many

152

StephenC. Levinson

detailed ways to correlate with the revised linguistic ones, once the special flexibility of the linguistic systemswith respectto origin is taken into account. Thus it should be possible , and intellectually profitable , to formulate the distinct frames of reference -modal application . Notice that this view conflicts in such a way that they have cross with the views of some that frames of referencein languageare imposedjust in the . On the contrary, I mapping from perception to languagevia the encoding process shall presumethat any and every spatial representation , whether perceptual or conceptual , must involve a frame of reference ; for example , retinotopic imagesjust are, . willy nilly , in a viewer-centeredframe of reference But at least one major problem remains. It turns out that the three distinct frames of referenceare " untranslatable" from one to the other, throwing further doubt on the idea of correlations and correspondences acrosssensoryand conceptual represen ' tationallevels . Which brings us to Molyneux s question.

's 4.4 Molyneux Question


In 1690William Molyneux wrote John Locke a letter posing the following celebrated question: If a blind man, who knew by touch the difference between a cube and a , had his sight restored, would he recognizethe selfsameobjects under his new sphere 61 perceptual modality or not? The question whether our spatial perception and conception is modality -specificis as alive now as then. Is there one central spatial model, to which all our input senses report, and from which instructions can be generated appropriate to the various , gaze , and so on)? output systems(touch, movement, language There have of course been attempts to answer Molyneux directly, but the results are conflicting . On the one hand, sight-restored individuals take a while to adjust , 94- 96; Valvo 1971 (Gregory 1987 ), monkeys reared with their own limbs masked from sight have trouble relating touch to vision when the mask is finally removed , 730- 731), and touch and vision are attuned to different properties (Howard 1987 is more attuned to weight and texture than shape ; Klatsky and (e.g., the tactile sense on the other hand human neonates Lederman 1993 , ); immediately extrapolate from touch to vision (Meltzoff 1993 and the ), neurophysiology suggestsdirect cross , 81; but seealso Stein 1992 wirings ( Berthoz 1991 ), so that some feel that the answer ' " Eilan 1993 237 . More to the question is a " resounding ' yes , ( ) soberly, it seemsthat there is some innate supramodal system observable in monkeys and infants, but it -modal thinking may even be dependent may be very restricted, and sophisticatedcross 62 on language . Here I want to suggestanother way to think about this old question. Put simply, we may ask whether the sameframes of referencecan in principle operate acrossall

Framesof Referenceand Molyneux ' s Question

153

the modalities, and if not , whether at least they can be translated into one another. " What we should mean by " modality here is an important question. In what follows I shall assumethat corresponding to (some of ) the different senses , and more generally " " to input/output systems , , there are specialized central representationalsystems for example, an imagistic systemrelated to vision, a propositional systemrelated to , Levelt , and so on (see , for example , a kinaestheticsystemrelated to gesture language ' two related then becomes of s 1989 ; Jackendoff 1991 ) . Our version Molyneux question : questions I . Do the different representationalsystemsnatively and necessarilyemploy certain frame~ nf reference ? 2. If so, can representationsin one frame of referencebe translated (converted) into ? another frame of reference Let us discount here the self-evident fact that certain kinds of information may , spatial representationsin an ; for example , in principle , be modality -specific perhaps , while those in a , be determinate with respectto shape imagistic mode must, it seems 63 SO . cannot be mode need not and , , Similarly , the hapticperhaps propositional kinesthetic modality will have available direct information about weight, texture, tactile warmth , and three-dimensional shapewe can only guessat from visual information ), while the directional and inertial information (Klatsky and Lederman 1993 from the vestibular systemis of a different kind again. All this would seemto rule out . What hybrid monster would a a single supramodal spatial representation system such have to be to record disparate information ? All that representation system concernsus here is the compatibility of frames of referenceacrossmodalities. . This is First , let us consider question 2, translatability across frames of reference the easierquestion, and the answerto it offers an indirect answerto question I . There is a striking , but on a moment' s reflection, self-evident fact: you cannot freely convert information from one framework to another. Consider, for example , an array, with a bottle on the ground at the (intrinsic ) front side of a chair. Suppose , too , that you view the array from a viewpoint such that the bottle is to the right of the chair; as it , the bottle is also north of the chair (see figure 4.11 ) . Now I ask you to happens " " : remember it , and supposeyou code the scenein an intrinsic frame of reference " bottle in front of chair " , discarding other information . It is immediately obvious that , from this intrinsic description, you cannot later generatea relative descriptionif you were viewing the array so that you faced one side of the chair , then the bottle would be to the left of or to the right of the chair - dependingon your viewpoint . So without a " coding" or specification of the locus of the viewpoint V, you cannot generate a relative description from an intrinsic description. The same holds for an absolute description. Knowing that the bottle is at the front of the chair will

154

ABSOLUTE

k- -~ ---~ -"""

.Y

L R ~ Lft ~ of ch bot to rig


StephenC. Levinson

C '~ IOJ Z oS ~ c ~

RELATIVE

bottle in front of chair

INTRINSIC

' Framesof Reference and Molyneux s Question

155

not tell you whether it is north or south or east or west of the chair- for that , you will need ancillary infonnation . In short, you cannot get from an intrinsic description- an orientation -free representation to either of the orientation bound . representations What about conversionsbetween the two orientation -bound frameworks? Again , . From the relative description or coding " The it is clear that no conversion is possible " bottle is to the left of the chair , you do not know what cardinal direction the bottle " lies in , nor from " the bottle is north of the chair can you derive a viewpoint -relative " " description like to the left of the chair. Indeed, the only directions in which you can convert frames of referenceare, in principle , from the two orientation -bound frames(relative and absolute) to the orientation -free one (intrinsic ) .64 For if the orientation of the ground object is fully spe cified, then you can derive an intrinsic description. For example , from the relative " The chair is the the bottle is to to and right of the chair facing my right description " in the same plane," and likewise from the absolute description The chair is facing " north and the bottle to the north of the chair, you can, in principle , arrive at the " ' intrinsic specification " The bottle is at the chair s front . Nonnally , though, because the orientation of the ground object is irrelevant to the orientation -bound descriptions , translations , this remainsa translation only in principle . By the samereasoning " " . in all other directions are in principle out , that is, impossible This simple fact about translatability across frames of reference may have far. Consider, for example , the following syllogism: reaching consequences I . Framesof referenceare incommensurable(i.e., a representationin one framework is not freely convertible into a representationin another); 2. Each senseutilizes its own frame(s) of reference(e.g., while vision primarily uses a viewer-centered frame, touch arguably uses primarily an object-centered frame, basedon the appreciation of form through three-dimensional grasping) ; 3. Representationsfrom one modality (e.g., haptic) cannot be freely translated into representationsin another (e.g., visual) . ' The syllogism suggest , then, that the answer to Molyneux s question is no- the blind man upon seeing for the first time will not recognize by sight what he knew before by touch. More generally, we will not be able to exchangeinfonnation across any internal representationsystemsthat are not basedon one and the sameframe of reference . I take this to be a counterintuitive result, a clearly false conclusion, in fact a reductio ad absurdum. We can indeed fonD mental images of contour shapesexplored , we can talk about, by touch alone, we can gestureabout what we have seen

156

StephenC. Levinson

or draw, what we have felt with our fingers, and so on. Becausepremise I seems self-evidently true, we must then reject premise 2, the assumption that each sensory modality or representational systemoperatesexclusively in its own primary , proprietary frame of reference . In short, either the frame of referencemust be the same -modal sharing of information or each acrossall sensorymodalities to allow the cross . modality must allow more than one frame of reference Intuitively , this seemsthe correct conclusion. On the one hand, peripheral sensory ; for example , low-level vision systemsmay operate in proprietary frames of reference 2 D know of while otoliths are restricted to a gravitational may only retinotopic arrays, . But, on the other hand, at a higher level, visual processing frame of reference seems to deliver 3-D analysesof objects as well as 2-D ones . Thus when we (presum ably) use the visual system to imagine rotations of objects, we project from 3-D models (intrinsic ) to 2! -D (relative) ones , showing that both are available. Thus more central, more conceptual, levels of representation seemcapable of adopting more than one frame of reference . Here, then, is the first part of the answer to our puzzle. Representationalsystems of different kinds, specializedto different sensorymodalities (like visual memory) or output systems(like gesture and language ), may be capable of adopting different frames of reference . This would explain how it is that Tenejapans , or indeed Dutch , can adopt the same frame of referencewhen utilizing different representational subjects - those involved in generating gesture , those involved in tasks requiring systems visual memory, those involved in making spatial inferences , as well as those in involved speaking . But to account for the facts described in section 4.2, it will not be sufficient to establish that the sameframe of referencecan, in principle, be used acrossdifferent kinds of internal representationsystems , those involved in nonverbal memory, gesture and language and so on. To account for those facts, it will be necessaryto assume , that individual subjectsdo indeed actually utilize the sameframe of referenceacross modalities. But now we have an explanation for this apparent fact: the untranslatability across frames of referencerequires individuals to stabilize their representational . For example , if a Tenejapan systemswithin a limited set of frames of reference man sees an array and remembersit only in terms of a viewer-centeredframework, he will not later be able to describeit - his languagesimply fails to provide a systematic viewer-centered frame of description. Thus the facts that (a) frameworks are not freely convertible, (b) languagesmay offer restricted frameworks as output , and (c) it may be desirable to describeany spatial experiencewhatsoever at some later point , theseconspire to require that a speakercode spatial perceptionsat the time of experience ' s dominant in whatever output frameworks the speaker languageoffers.

Frames of Reference and Molyneux ' s Question


4.5 Conclusions

157

This chapter began with some quite unexpectedfindings: languagescan differ in the set of frames of referencethey employ for spatial description. Moreover, the options in a particular languageseemto dictate the useof frames of referencein nonlinguistic -modal tendency to fix on a dominant frame of tasks- there seemsthus to be a cross reference . This raisesa number of fundamental puzzles : What sensedoes it make to " across talk of " same frame of reference modalities, or psychological faculties of quite different kinds? If it does make sense , why should it be so? What light does the phenomenon throw on how spatial information is shared across the senses , across the various " input " and " output " devices ? I have tried to sketch answersto thesepuzzles . The answersconvergein two kinds of responses to Molyneux ' s question " do the senses talk to one another?" The first kind of responseis an empirical argument: 1. The frame of reference dominant in a given language " infiltrates " other modalities, presumably to ensurethat speakerscan talk about what they see , feel, and so on; 2. Therefore, other modalities have the capacity to adopt, or adapt to , other frames of reference a yes answer to Mr . Molyneux . , which suggests The secondkind of responseis an a priori argument: I . Frames of referencecannot freely " translate" into one another; 2. Therefore, if the modality most adaptive to external influences , namely, language , , the others must follow suit; adopts one frame of reference 3. To do this, all modalities must have different frames of referenceavailable, or be able to " annotate" experiences with the necessary ancillary information , which suggests a yes answer to Mr . Molyneux . ' Actually , an affirmative answer to Molyneux s question is evidently requiredotherwise we could not talk about what we see . What is deeply mysterious is how -modal transfer is achieved this cross . The untranslatability across frames of reference greatly increasesthe puzzle. It is in this light that the findings with which we - the standardization of frames of referenceacrossmodalities in line with the began - now seemnot only less local language surprising, but actually inevitable. Dts Ackaowledgme Thischapter is based on results of joint research Brownon Tzeltal , in particularwith Penelope , but also with many colleagues in the CognitiveAnthropologyResearch Group, who have the research collaboratively developed programoutlined here(seealso Senft 1994 ; Wilkins

158 1993 ; Hill 1994 ) . I am also indebted to colleaguesin the wider ; Danziger 1994 ; Pederson1994 different researchprograms challenged premature who have Institute , through Psycholinguistics conclusions and emboldened others (see , in this volume Bierwisch, Levelt, , for example debt to Levelt' s pioneering work on the the and Bowerman, chapters2, 3, and 1O ; , respectively typology and logic of spatial relations will be particularly evident) . In addition , John Lucy, ; and Bernadette SuzanneGaskins, and Dan Slobin have beenimportant intellectual influences Schmitt and Laszlo Nagy have contributed to experimental design and analysis. The contributions , ideas , and criticisms of other participants at the conferenceat which this paper was given . have been woven into the text; my thanks to them and the organizers of the conference Finally , I receivedvery helpful comments on the manuscript from Sotaro Kita , Lynn Nadel, , and David Wilkins , not all of which I have beenable to adequatelyrespond to. Mary Peterson Notes 1. I shall usethe tenn modality in a slightly special , but I think motivated, way. When psychologists -modal" effects in mind transfer of infonnation acrosssensory have talk of " cross , they " " modalities (vision, touch, etc.) . Assuming that these sensory input systemsare modules in the Fodorean sense , we are then interestedin how the output of one module, in someparticular inner representation system , is related to the output of some other module, most likely in -modal another inner representationsystemappropriate to another sensoryfaculty . Thus cross -specific , effectscan be assumedto occur through communication betweencentral, but still sense , not through peripheral representationsystemsspecializedto modular representationsystems . But seesection 4.4. es process 2. Although there are phrasesdesignatingleft -hand and right -hand, theseare body-part tenns with no spatial uses , while body-part tenns for face and back are used for spatial description and then on the basisof an intrinsic assignment , not nearly exclusivelyfor objects in contiguity ' a relative one basedon the speakers viewpoint (seeLevinson 1994 ). 3. The design of this experiment was much improved by BernadetteSchmitt. 4. The design of this experiment is by Eric Pedersonand Bernadette Schmitt, building on an . earlier design describedin Levinson 1992b 5. The phenomenonof fixed bearingsin gesturewas first noticed for an Australian Aboriginal ), who subsequentlydemonstratedthe existenceof the samephenomenon group by Haviland ( 1993 in Zinacantan, a neighboring community to Tenejapa. 6. Rock ( 1992 , which built directly on the ) is here commenting on Asch and Witkin 1948 . 1990 Gestalt notions. Seealso Rock ( ) " is voiced 7. One kind of disagreement , 471) : Spatial frameworks are incorporated by Paillard ( 1991 . They are not however to be confused with in our perceptual and motor experiences " the system of coordinateswhich abstractly represent them (emphasis ) . But this is terminol , oglcal; for our purposeswe wish preciselyto abstract out the properties of frames of reference . so that we can consider how they apply acrossdifferent perceptual or conceptual systems 8. " When placesare individuated by their spatial relation to certain objects, a crucial part of ' ' what we need to know is what those objects are. As the tenn frame of reference is commonly " , 25) . used , theseobjectswould be said to provide the frame of reference ( Brewerand Pears1993

's Frames of Reference andMolyneux Question

159

9. I shall use the opposition figure versusground for the object to be located versusthe object with respect to which it is to be located, respectively . This opposition is , after Talmy 1983 identical to themeversusre/atum, referent versusre/atum, trajector versuslandmark, and various other terminologies. 10. Brewer and Pears( 1993 , 26) consider the role of coordinate systems , but what they have to say only increases our puzzlement : " Two eventsare representedas being in the samespatial position if and only if they are assignedthe sameco-ordinates. Specifying a frame of reference would have to do with specifying how co-ordinates are to be assignedto eventsin the world on the basis of their spatial relations to certain objects. Theseobjects provide the frame of reference ." This fails to recognizethat two distinct systemsof coordinates over the sameobjectscan describethe sameplace. II . There are many good sketch es of parts of this intellectual terrain (see , for example , Miller ' and Johnson- Laird 1976 ; Jammer 1954 ; O Keefe and Nadel 1978 ), but none of it all. ' s introduction of 12. Some notion of absolute spacewas already presupposedby Descartes coordinate systems , as Einstein ( 1954 , xiv ) pointed out . 13. This association was in part due to the British empiricists like Berkeley whose solipsism made egocentric relative spacethe basis for all our spatial ideas . SeeO' Keefe and Nadel 1978 , 14- 16. 14. Much behavioral experimentation on rats in mazeshas led to classifications of behavior . O' Keefe and Nadel' s 1978 classification, for parallel to the notions of frame of reference , is in terms of body position responses(cf. egocentric frames of reference example ), cue responses (a kind of allocentric responseto an environmental gradient), and place responses (involving allocentric mental maps) . Work on infant behavior similarly relates behavioral - see , usually egocentricversusallocentric (or geographic response types to frames of reference Pick 1988 147 156 . , ) 15. Seealso Brewer and Pears( 1993 , 29), who argue that allocentric behavior can always be mimicked through egocentric computations: " Perhapslanguage . . . provides the only conclusive " macroscopicevidencefor genuine allocentricity . 16. Thesedistinctions are seldom properly made in the literature on mental maps in humans. Students of animal behavior, though, have noted that maps consisting of relative angles and distancesbetweenlandmarks have quite different computational properties to maps with fixed : in the former , but not the latter , each time landmarks are added to the map, the bearings databaseincreasesexponentially (see , for example , Mc Naughton, Chen, and Markus 1990 ). Despite that , most rat studies fail to distinguish between these two kinds of allocentricity, relative and absolute. " " 17. Paillard ( 1991 , 471- 472) has a broader notion of frames of reference than most brain scientists (and closer to psychological ideas ); he proposes that there are four such frames subserving visually guided action, all organized around the geocentric vertical: ( I ) a body frame, presuming upright posture for action; (2) an object frame, presumably similar to Marr ' s ; (3) a world frame, a Euclidean spaceinclusive of both body and ( 1982 ) object-centeredsystem object; and (4) a retinal frame, feeding the object and world frames. He even provides a rough neural " wiring diagram" (p . 473) .

160

StePhenC. Levinson

18. The age at which this switch to the non- egocentric takes place seemshighly task- dependent . SeeAcredolo ( 1988 ), for ), who gives sixteenmonths as an end point ; seealso Pick ( 1993 a route-finding task, where the processhas hardly begun by sixteenmonths. 19. This leap from a perspectiveimage, or worse, a silhouette, is possible (Marr argued) only by assumingthat objects can be analyzedinto geometrical volumes of a specifickind (generalized cones ); hence3-D models must be of this kind , where principal axesare identified. 20. Others have suggestedthat what we store is a 2! -D image coupled with the ability to ), thus giving our apparent ability to rotate mental mentally rotate it (Tarr and Pinker 1989 ' ) some evolutionary raison d etre. Yet others suggestthat images (Shepard and Metzler 1971 object recognition is achieved via a set of 2! D images from different orientations (Bulthoff 1991 ) suggestwe have none of thesepowers. ), while some( Rock, Wheeler, and Tudor 1989 21. SeeDanziger 1994for possible connections to linguistic distinctions; I am grateful to Eve Danziger for putting me in touch with this work . " 22. AsKant 1768made clear, objects differing in handedness (enantiomorphs or incongruent " in Kant ' s terminology), cannot be distinguished in an object-centered(or intrinsic counterparts . SeeVan Cleve and Frederick , but only in an external coordinate system ) frame of reference . Brown 1994 to Tzeltal Levinson and and for the relevance 1991 , , , 23. For example , the cube comparisonstest can be solved by ( 1) rotation using viewer-centered coordinates; (2) rotation around an object-centeredaxis imaged with viewer-centeredcoordinates ; (3) rotation of the perspectivepoint around the object; or (4) purely object-centered . compansons : 24. Thus Cohen and Kubovy ( 1993 , 379 ) display deep confusion about frames of reference in information of handedness that one can have orientation free representations they suggest " an orientation -free frame of referenceby utilizing the notion " clockwise. But asKant ( 1768 ) showed , and generations of philosophers since have agreed (see Van Cleve and Frederick " " an external orientation . 1991 ), the notion clockwise presupposes ' ' 25. Carlson- Radvansky and Irwin s view would seem to be subtly different from Levelt s ); seebelow in text. ( 1989 26. The equation is Tversky' s; actually, her survey perspectivein somecases(e.g., outside the " " context of maps) may also relate to a more abstract absolute spatial framework where both . viewer and landmarks are embeddedin a larger frame of reference 27. The conceptual systemis abstract over different perceptualclues , as shown by the fact that " astronauts can happily talk about, say, " above and to the left where one perceptual clue for - 155 the vertical (namely gravity) is missing (Friederici and Levelt 1990 , 154 ) ) . Levelt ( 1989 concludes that the spatial representation itself does not determine the linguistic description: " There is . . . substantial freedom in putting the perceivedstructure, which is spatially represented " into one or another , propositional format . " " 28. For example , there is no convincing explanation of the English deictic use of front , " : we " The cat in front of the tree " as if the tree was an interlocutor " back " " left " " , , , right say, " " , in facing us, but when we say, The cat is to the left of the tree, we do not (as, for example

' Frames of Reference and Molyneux s Question

' s left therefore for this explanatory Tamil) meanthecat is to the tree to our right. The reason , not , the requisitecoordinatesystems gap is that the factshavealwaysbeenunderdescribed . out evenin the mostrecent works beingproperlyspelled or relatorshavea complexrelation to framesof 29. The so -calledtopologicalprepositions in termsof coordinate . First, notethat frames of reference areheredefined reference , systems " relators no or coordinate information and many" topological , , for example express angular and often intrinsic at or near . However , othersdo involve the vertical absolutedimension " . Thus properanalysis of the " topological features , or axial properties , of landmarkobjects notionsinvolvespartitioning their features between noncoordinate spatialinformation and the framesof reference mentioned belowin the features of informationdistributedbetween " is an intrinsic notion basedon text. Thus Englishin as in " the moneyin the piggy bank " " of the ground object intrinsic ; underas in the dust under the rug compounds properties , andso forth. , bottom (vertical ) information (undersurface ) andabsolute 30. Exceptin someplaces , like the TorresStraits , wherethe tradewindsroar throughwestward " and " windward can be in termsof " leeward ." Or wherethe and spatialdescriptions earthdropsawayin onedirection of mountainranges , ason theedges , gravitycanbenaturally . importedinto the horizontalplane 31. The readermay feelthat the notion of " front" is differentfor chairsand persons (and so " is somehow moreabstract than" in front of course it is), andin particularthat " in front of me " or " at the foot of the chair" of the chair." But noticethat wecould havesaid" at my feet " " " " here feet or foot clearlymeans differentin eachcase the notion of , but shares something . an intrinsicpart of the relatumobject
32. The importance of the distinction betweenbinary and ternary spatial relators was pointed . out by Herrmann 1990 33. For example , the Australian language Guugu Yimithirr has (derived) lexemesmeaning " north side of " " south side of " and so on which combine both intrinsic and absolute frames , , , of referencein a single word. Less exotically, English on as in " the cup on the table" would seemto combine absolute (vertical) information with topological information (contact) and intrinsic information (supporting planar surface ). 34. This point is important . Somepsychologistshave beentempted to presume of the , because " " ambiguity of English spatial expressionssuch as in front , that frames of referenceare imposed on language by a spatial interpretation , rather than being distinguished semantically see for , Carlson- Radvansky and Irwin 1993 ( , ). example 35. We know one way in which this tripartite typology may be incomplete: somelanguagesuse conventionalized landmark systems that in practice grade into absolute systems , although there are reasonsfor thinking that landmark systemsand fixed-bearing systemsare distinct conceptual types. 36. I am indebted to many discussions with colleagues (especially Balthasar Bickel, Eric Pederson , although they would not , and David Wilkins ) over the details of this scheme necessarily agreewith this particular version. 37. Thus the " face" of a stone may be the bottom surfacehidden in the soil, as long as it meets the necessary axial and shapeconditions.

162

StephenC. Levinson

38. We tend to think of human prototypes as inevitably the sourceof such prototype parts, but such anthropomorphism may be ethnocentric; for example , in Mayan languagesplant parts figure in human body-part descriptions (seeLaughlin 1975 ; Levinson 1994 ). 39. Thus Miller and Johnson- Laird ( 1976 : " Peopletend to , 401), thinking of English speakers treat objects as six-sided. If an object has both an intrinsic top and bottom , and an intrinsic front and back, the remaining two sides are intrinsically left and right ." Incidentally, the of " intrinsic left /right " is perhaps an indication that such systemsare not exclusively possession object-centered(becauseleft and right cannot ultimately be distinguished without an external frame of reference ). 40. For a nice contrast betweentwo apparently similar Meso-American systems , one of which is armature-based and the other based on the location of individual facets , see MacLaury ( 1989 ) on Zapotec, and Levinson ( 1994 ) on Tzeltal. 41. Miller and Johnson- Laird ( 1976 ) suggestthat the notion of intrinsic region may be linked to perceptualcontiguity within 10degrees of visual arc (p . 91), but that the conceptualcounterpart to this perceptual notion of region combines perceptual information with functional information about the region drawn from social or physical interaction (pp . 387- 388) . 42. It may be that left and right are centeredon V, whilefront and back are indeed rotated and have their origin on G. Evidence for that analysis comes from various quarters. First , some allow both the English- and Hausa-style interpretations offront , while languageslike Japanese , suggestingthat there are two distinct subsystems maintaining left and right always the same " " " " involved. Second , English left and right are not clearly centeredon G becausesomething can be to the left of G but not in the same plane at all (e.g., " the mountain to the left of the tree" ), while English " front " and " back" can be centeredon G, so that it is odd to say of a cat near me that it is " in front ofa distant tree." Above all , there is no contradiction in " the cat is to the front and to the left of the tree." An alternative analysis of English would have the coordinates fixed firmly on V, and give " F is in front of the tree" an interpretation along the lines " F is between V and G " (" behind" glossing " G is between V and F " ) . My own guessis that English is semantically general over thesealternative interpretations. 43. Note that , for example , we think of a tree as unfeatured on the horizontal dimension, so that it lacks an intrinsic front , while someNilotic cultures make the assumption that a tree has a front , away from the way it leans . 44. But some languagesencode relative concepts based directly on visual occlusion or the absence of it ; thesedo not have intrinsic counterparts (as S. Kita has pointed out to me) . 's 45. As shown by the intrinsic system priority in acquisition (Johnston and Slobin 1978 ) . On the other hand, some languageshardly utilize an intrinsic frame of referenceat all (see , for , Levinson 1992bon an Australian language example ). 46. I owe the germ of this idea to Eric Pederson . 47. This does not seem , once again, the right analysis for English left/right, becauseF and G need not be in the same plane at all (as in " the tree to the left of the rising moon" ), and " " intuitively , to the left of the ball does not ascribe a left facet to the ball.

' Framesof Referenceand Molyneux s Question

163

in relative descriptions hold only on the presumption 48. Although transitivity and converseness that V is constant. 49. Conversely , other languageslike Tamil useit in more far -reaching ways. " " 50. Fmay be a part of G, as in the bark on the left (side) of the tree. 51. Rotation will havefront toward V, and clockwise (looking down on G ) fromfront : right , back, left (as in Tamil ) . Translation will have back toward V, and clockwise from back: left , front , right (as in Hausa) . Reflection will havefront toward V, but clockwise from front : left , back, right (as in English, on one analysis ) . The rotation and translation casesclearly involve secondarypolar coordinates on G. The reflection casescan be reanalyzedas defined by horizontal correct and vertical coordinates on the retinal projection , or can be thought of (as seems for English) as the superimposition of two systems , the left/right terms involving only primary coordinates on V, and thefront / back terms involving rotated secondarycoordinates on G. 52. Environmental clues will not explain how some people can exercisesuch heighteneddead reckoning abilities outside familiar territory . I presumethat such people have been socialized to constantly compute direction as a background task, by inertial navigation with constant checks with visual information and other sensory information (e.g., sensingwind direction ) . But seeBaker ( 1989 ), who believesin faint human magnetoreception. 53. Note that none of these environmental gradients can provide the cognitive basis of . Once the community has fixed a direction , it remains in that direction abstracted systems in local landfall , drainage, wind source fluctuations , equinox, and so on , or even regardlessof removal of the subject from the local environment. Thus the environmental sourcesof such not generally explain how they are used , or how the systemsmay explain their origins but do " " cardinal directions are psychologically fixed. 54. Our current polar systemis due no doubt to the introduction of the compassin medieval " " , showing times. Before, maps typically had east at the top , hencethe expression orient oneself . that our use of polar coordinates is older than the compass , 55. Warlpiri may be a casein point . Although such a systemmay be basedon a solar compass solstitial variation makes it necessaryto abstract an equinoctial bisection of the seasonal movement of the sun along the horizon ; it is therefore less confusing to fix the system by referenceto a mentally constituted orthogonal . 56. Guugu Yimithirr would be a casein points becausethere are no elicitable associationsof or priority betweencardinal directions. sequence ' 57. See Peter Sutton s ( 1992 ) description of the Wik Mungan system (another Aboriginal languageof Cape York ) . , for helping me to systematizethese 58. I am grateful to David Wilkins , and other colleagues . observations ). 59. Table 4.4 owesmuch to the work of Eve Danziger (seeespeciallyDanziger 1994 60. SeeVan Cleve and Frederick 1991 for discussion of this Kantian point . For the cross cultural implications and a working out of the place of absolute systems in all this, see . Danziger 1994

164

StephenC. Levinson

61. First discussedin Locke, Essay on Human Understanding(book 2, ix , 8), Molyneux ' s : Ch. 13 question was brought back into philosophical discussionby Gareth Evans ( 1985 ), and many of the papers in Eilan , McCarthy , and Brewer 1993explicitly addressit . -modal bridge" ; Dennett 62. See : " languageservesas a cross , for example , Ettlinger 1987 , 174 1991 . , 194 199 63. The issuemay be lessclear than it at first seems ; seeTye 1991 , 5- 9. 64. The possibility of getting from a relative representation to an intrinsic one may help to ' explain the apparent inconsistency between our findings here and Levelt s (chapter 3, this ' volume) . In Levelt s task, subjectswho made ellipses always presupposedan underlying uniform , even when their spatial descriptions varied between relative spatial frame of reference and intrinsic , thus suggestingthat frames of referencemight residein the mapping from spatial representation to language rather than in the spatial representation itself. But , as Levelt acknowledges , the data are compatible with an analysis whereby the spatial representation is itself in a relative frame of referenceand the mapping is optionally to an intrinsic or relative description. The mapping from relative to intrinsic is one of the two mappings, in principle possible between frames of reference , as here described , whereas a mapping from intrinsic spatial representation to linguistic relative representation would be in principle impossible. This would seemto explain all the data that we currently have in hand.
References

-Davis, M . Krit Acredolo , L . ( 1988 . In J. Stiles ) . Infant mobility and spatial development - 166 and U . Eds. : Brain bases and development . , ), Spatial cognition chevsky Bellugi ( , 157 Hinsdale, NJ: Erlbaum. Asch, S. E., and Witkin , H . A . ( 1948 ) . Studiesin spaceorientation 2. Perception of the upright with displaced visual fields and with body tilted . Journal of Experimental Psychology , 38, 455- 477. Reprinted in Journal of Experimental Psychology , General , 121 ( 1992 , 4), 407- 418. , H ., and Danziger, E. (Eds.) . ( 1994 Baayen ) . Annual Report of the Max Planck Institute for . Nijmegen. , 1993 Psycholinguistics Baker, M . ( 1989 . Manchester : University of Manchester ) . Human navigation and magneto reception Press . Berthoz, A . ( 1991 framesfor the perceptionand control of movement . In J. Painard ) . Reference . Oxford : Oxford Science . , 81- 111 (Ed.), Brain and space Bickel, B. ( 1994 ) . Spatial operations in deixis, cognition , and culture: Where to orient oneself in Belhare . Working paper no. 28, Cognitive Anthropology ResearchGroup, Max Planck Institute for Psycholinguistics , Nijmegen. Bierwisch, M . ( 1967 ) . Some semantic universals of German adjectivals. Foundationsof Language , 3, 1- 36. Bowerman, M ., and Pederson , E. ( 1992 ) . Cross-linguistic perspectiveson topological spatial relations. Talk given at the American Anthropological Association, San Francisco, December .

Frames of Reference and Molyneux ' s Question

165

Brewer, B., and Pears . In N . Eilan , R. McCarthy , and B. , J. ( 1993 ) . Frames of reference Brewer (Eds.) , Spatial representation : Problemsin philosophy and psychology , 25- 30. Oxford : Blackwell. Brown, P. ( 1991 ) . Spatial conceptualization in Tzeltal. Working paper no. 6, Cognitive , Nijmegen. Anthropology ResearchGroup , Max Planck Institute for Psycholinguistics " " " " Brown, P., and Levinson, S. C. ( 1993a ) . Uphill and downhill in Tzeltal. Journal of Linguistic Anthropology, 3( 1), 46- 74. Brown , P., and Levinson, S. C. ( 1993b ) . Explorations in Mayan cognition . Working paper no. 24, Cognitive Anthropology Research Group , Max Planck Institute for Psycholinguistics , Nijmegen. Buhler, K . ( 1934 ) . The deictic field of languageand deictic words. Reprinted in R. Jarvella and W. Klein ( Eds.), Speech . , place and action, 9- 30. New York : Wiley, 1982 Bulthoff , H . H . ( 1991 ) . Shape from X : Psychophysicsand computation . In MS . Landy and J. A . Movshon (Eds.), Computational modelsof visualprocessing , 305- 330. Cambridge, MA : MIT Press . Campbell, J. ( 1993 ) . The role of physical objects in spatial thinking . In N . Eilan , R. McCarthy , and B. Brewer ( Eds.), Spatial representation : Problems in philosophy and psychology , 65- 95. Oxford : Blackwell.

- 244 : Where is above ?Cognition . , 46 , 223 guage

Carlson- Radvansky, L . A ., and Irwin , D . A . ( 1993 ) . Frames of referencein vision and lan-

Clark , H . H . ( 1973 , time, semantics , and the child. In TE . Moore (Ed.), Cognitive ) . Space and the acquisition of language . , 28- 64. New York : Academic Press development Cohen, D ., and Kubovy , M . ( 1993 . , and fiat slopes ) . Mental rotation , mental representation , 25, 351- 382. Cognitive Psychology

, E. (Ed.). ( 1993 Danziger ). Cognition and spacekit version 1.0. CognitiveAnthropology Research . , Nijmegen Group, Max PlanckInstitutefor Psycholinguistics in Mopan , E. ( 1994 ). As freshmeatlovessalt: The logic of possessive Danziger relationships , Cognitive Maya. Workingpaperno. 30 AnthropologyResearch Group, Max PlanckInstitute for Psycholinguistics . , Nijmegen Dennett . Boston : Little, Brown . , D. ( 1991 ). Consciousness explained 's Eilan, N. ( 1993 . ) Molyneux questionand the idea of an externalworld. In N. Eilan, R. .), Spatialrepresentation : Problems in philosophy andpsychology , and B. Brewer(Eds McCarthy - 255 . Oxford: Blackwell . , 236 Eilan, N., McCarthy : Problems in philosophy , R., and Brewer , B. ( 1993 ). Spatialrepresentation andpsychology . Oxford: Blackwell . Einstein : Thehistoryof theories , A. ( 1954 ). Introductionto M. Jammer , Concepts of space of in physics . Cambridge . , MA: HarvardUniversityPress space -modelsensory . In R. Gregory(Ed.), TheOxfordcompanion , G. ( 1987 Ettlinger ). Cross integration - 174 to themind . Oxford: Oxford UniversityPress . , 173

166 Evans, G. ( 1985 . Oxford : Clarendon Press . ). Collectedpapers

StephenC. Levinson

Fillmore , C. ( 1971 ) . Toward a theory of deixis. Paper presented at Pacific Conference on Contrastive Linguistics and LanguageUniversals, University of Hawaii , Honolulu , January. Friederici, A ., and Levelt, W . J. M . ( 1990 : Perceptual ) . Spatial reference in weightlessness factors and mental representations . Perceptionand Psychophysics , 47(3), 253- 266. . Gregory, R. L . ( 1987 ) . Oxford companionto the mind. Oxford : Oxford University Press Haviland , J. B. ( 1993 . and . Journal ) Anchoring iconicity in Guugu Yimithirr pointing gestures of Linguistic Anthropology, 3( 1), 3- 45. Hemnann , T . ( 1990 ). Vor , hinter , rechts, und links: Das 6H -Modell . Zeitschrift fUr Liter - 140 und Linguistik , 78, 117 . aturwissenschaft Herskovits, A . ( 1986 ) . Languageand spatial cognition : An interdisciplinary study of the prepositions in English. In Studies in natural languageprocessing , 208 p . Cambridge: Cambridge . University Press Hill , C. ( 1982 ) . Up/ down, front / back, left/right : A contrastive study of Hausa and English. In J. Weissenborn and W . Klein (Eds.), Here and there: Cross linguistic studies on deixis and demonstration , 11- 42. Amsterdam: Benjamins. Hill , D . ( 1994 ). Spatial configurations and evidential propositions. Working paper no. 25, Cognitive Anthropology Research Group , Max Planck Institute for Psycholinguistics , Nijmegen. Hockett , C. F. ( 1960 . Scientific American ). The origin of speech , 203, 89- 96. Howard , I . P. ( 1987 . In R. L . Gregory ( Ed.), The Oxford ) . Spatial coordination of the senses . companionto the mind, 727- 732. Oxford : Oxford University Press Jackendoff, R. ( 1991 ). Parts and boundaries. Cognition, 4/ , 9 45. Jammer, M . ( 1954 : The history of theoriesof spacein physics . Cambridge, ) . Conceptsof space MA : Harvard University Press . Johnston, J. R., and Slobin, D . ( 1978 ) . The development of locative expressionsin English, Italian , Serbo-Croatian , and Turkish . Journal of Child Language , 6, 529- 545. Just, M ., and Carpenter, P. ( 1985 : Accounts of mental rotation ) . Cognitive coordinate systems - 172 and individual differencesin spatial ability . PsychologicalReview . , 92(2), 137 Kant , E. ( 1768 ) . Von Dernersten Grunde des Unterschiedesder Gegendenim Raume. Translated as On the first ground of the distinction of regions in spacein J. Van Cleve and RE . Frederick (Eds.) The philosophy of right and left : Incongruent counterpartsand the nature of . , 27- 34. Dordrecht: Kluwer , 1991 space Klatsky , R. L ., and Lederman, S. J. ( 1993 ) . Spatial and nonspatial avenuesto object recognition . In N . Eilan, R. McCarthy and B. Brewer ( Eds.), Spatial by the human haptic system - 205. Oxford : Blackwell. : Problemsin philosophyand psychology representation , 191 . Kosslyn, S. M . ( 1980 ) . Image and mind. Cambridge, MA : Harvard University Press

' Frames of Referenceand Molyneux s Question

167

" " " " and spatial Landau , R. ( 1993 ). What and where in spatiallanguage , B., and Jackendoff . 217 16 265 Brain Sciences and . Behavioral , , cognition . Washington Zinacantan Tzotzil . The , DC: R. 1975 dictionary of SanLorenzo , ( ) great Laughlin . Smithsonian . . London : Longmans a semantic Leech of English description , G. ( 1969 ). Towards . In A. J. van limitationson talking about space Levelt , W. J. M. ( 1984 ). Someperceptual - 358 . .), Limits in perception Doorn, W. A. van der Grind, and J. J. Koenderink(Eds , 323 . Press : VNU Science Utrecht . . Cambridge to articulation : Fromintention Levelt , MA: MIT Press , W. J. M. ( 1989 ). Speaking . : Cambridge . Cambridge Levinson UniversityPress , S. C. ( 1983 ). Pragmatics and conception of spatialdescription Levinson , S. C. ( 1992a ). Primerfor the field investigation 2 I 5 47. . Pragmatics , ( ), of spatial : The cognitiveconsequences and cognition S. C. Levinson , ). Language ( 1992b , CognitiveAnthropologyResearch descriptionin Guugu Yimithirr. Working paperno. 13 . , Nijmegen Group, Max PlanckInstitutefor Psycholinguistics -part terminology : Tzeltalbody Levinson , andlinguisticdescription , shape , S. C. ( 1994 ). Vision . 4 791 856 32 . Special volumeof Linguistics and objectdescription , ( ), : Anthropology . Immanuel Kant P. 1994 Brown S. C. and Levinson , ( , ) amongthe Tenejapans , . Ethos asappliedphilosophy , 22( I ), 3- 41. . Journalof Navigation in Australia , 29, Lewis , D. ( 1976 ). Routefinding by desertaborigines 21- 38 . . : Cambridge . Vols. I and2. Cambridge UniversityPress , J. ( 1977 ). Semantics Lyons . extensions and metaphoric : Prototypes , R. ( 1989 ). Zapotecbody part locatives MacLaury - 154 . Journalof American International , 55(2), 119 Linguistics . . NewYork: Freeman Marr, D. ( 1982 ). Vision of English in the gesticulation , K. E. ( 1993 ). Spatialinformationand cohesion McCullough American of the the Annual Convention at . andChinese Psychological presented Paper speakers . Society " . " Deadreckoning , and on, B., Chen , landmarklearning McNaught , E. 1990 , L., and Markus . Journalof Cognitive andcomputational : A neurophysiological of direction the sense hypothesis - 202 . Neuroscience , 3(2), 191 ' -modalperception : Cross Meltzoff , and the mind , imitation , A. N. ( 1993 ). Molyneuxs babies : .), Spatialrepresentation infant. In N. Eilan, R. McCarthy of thepreverbal , andB. Brewer (Eds - 235 . . Oxford: Blackwell andpsychology in philosophy Problems , 219 -Laird, P. N. ( 1976 . Cambridge andperception , MA: Miller, G. A., and Johnson ). Language . HarvardUniversityPress -horse . In N. of space in the neurophilosophy : An essay O' Keefe , J. ( 1993 ). Kant and the sea and : in Problems . B. Eds and Brewer Eilan, R. McCarthy philosophy , ), Spatialrepresentation ( . Blackwell 64. Oxford: 43 , psychology

168

StephenC. Levinson

O' Keefe , J., and Nadel, L . ( 1978 ) . The hippocampusas a cognitive map. Oxford : Clarendon Press .
Paillard . Oxford: Oxford Science . , J. (Ed.). ( 1991 ). Brainandspace Pederson in two Tamil linguisticsystems and manipulable . In , E. ( 1993 ). Geographic space - 311 .), Spatialinformation A. U. Frank and I. Campari(Eds . Berlin : Springer . , 294 theory Pederson ascontext asmeans : Spatialcognitionand habitual , E. ( 1995 , language ). Language use . Cognitive , 6( 1), 33- 62. language Linguistics ' . London : Routledge and , J., and Inhelder , B. ( 1956 Piaget ). Thechilds conception of space . KeganPaul -Davis Pick, H. L., Jr. ( 1988 of spatialcognitivedevelopment . In J. Stiles ). Perceptual , aspects

M. Kritchevsky .), Spatial : Brainbases anddevelopment , andU. Bellugi (Eds , 145 cognition . Hinsdale 156 . . NJ: Erlbaum

Pick in children . In N.. Eilan, R. , H. L., Jr. ( 1993 ). Organizationof spatial knowledge .), Spatialrepresentation : Problems in philosophy andpsychology , and B. Brewer(Eds McCarthy - 42. Oxford: Blackwell . , 31 Pinker andcognition . Cambridge . , S. ( 1989 , MA: MIT Press ). Learnability Rock . In I. Rock (Ed.), Thelegacyof Soloman Asch , I. ( 1990 ). The frameof reference , 243 268 . Hillsdale . NJ: Erlbaum , I. Rock 1992 on Aschand Witkin' s " Studies in space . 2." Journalof orientation , ( ). Comment - 406 : General . , 121 Experimental (4), 404 Psychology Rock , I., Wheeler , D., and Tudor, L. ( 1989 ). Canwe imaginehow objectslook from other - 210 ? Cognitive . , 21, 185 viewpoints Psychology - A case Senft in Kilivila: The Tinker . , G. ( 1994 ). Spatialreference toy matching games study and in , 25, 98 99. Language linguistics Melanesia -dimensional . Science , R. N., and Metzler , J. ( 1971 Shepard ). Mentalrotationof three , objects - 703 171 . , 701

;on : Theories in antiquity and their sequel . London : , space , andmot , R. ( 1988 Sorabji ). Matter
Duckworth . Stein, J. F. ( 1992 ) . The representation of egocentric space in the posterior parietal cortex. Behavior a/ and Brain Sciences , 15(4), 691- 700.

Sutton in Wik Mungan . Talk givenat the 1stAustralianLinguistic , P. ( 1992 ). Cardinaldirections Institute , Sydney , July. Svorou . Amsterdam : Benjamins . , S. ( 1994 ). Thegrammar of space Takano of rotated forms : A theory of information types . Cognitive , Y. ( 1989 ). Perception 21 1 59. , , Psychology structures . In H. Pick and L. Acredolo(Eds .), Spatial , L. ( 1983 ). How language Talmy space - 282 orientation : Theory . NewYork: Plenum . Press , research , andapplication , 225

Frames of Referenceand Molyneux ' s Question

169

in shape Tarr, M., andPinkerS. ( 1989 ). Mentalrotationandorientationdependence recognition . Cognitive 21 233 282 . , , Psychology in spatialdescriptions . Journalof Memory , B. (in press Taylor, H. A., andTversky ). Perspective & Language , 35. - 208 . Psychological . Tolman Review , E. C. ( 1948 , 55(4), 189 ). Cognitivemapsin rats and men . Psychology andMotivation , B. ( 1991 , 27, 109 Tversky ). Spatialmentalmodels of Learning 145 . : Representation debate andmind . Cambridge . , MA: MIT Press ). Theimagery Tye, M. ( 1991
Valvo , A . ( 1971 : The problems and behavior ) . Sight restoration after long-tenD blindness . York . of visual rehabilitation New patterns Van Cleve, J., and Frederick, RE . (Eds.) . ( 1991 ) . Thephilosophyof right and left : Incongruent . Dordrecht : Kluwer . counterpartsand the nature of space Vandeloise : A casestudyfrom French. Chicago University of , C. ( 1991 ) . Spatial prepositions . Chicago Press Wilkins , D . ( 1993 of semanticchangeand the search ) . From part to person: Natural tendencies for cognates . Working paper no. 23, Cognitive Anthropology ResearchGroup , Max Planck Institute for Psycholinguistics , Nijmegen.

Karen Emmorey

Expressedby hands and face rather than by voice, and perceivedby eye rather than by ear, signed languageshave evolved in a completely different biological medium . Used primarily by deaf people throughout the world , they from spoken languages have arisen as autonomous languagesnot derived from spoken language and are ; passeddown from one generation of deaf people to the next ( Klima and Bellugi 1979 Wilbur 1987 ) . Deaf children with deaf parents acquire sign language in much the sameway that hearing children acquire spoken language(Newport and Meier 1985 ; Meier 1991 ) . Sign languagesare rich and complex linguistic systemsthat manifest the universal properties found in all human languages( Lillo -Martin 1991 ). : the linguistic In this chapter, I will explore a unique aspect of sign languages . Because use of physical space they directly use spaceto linguistically expressspatial locations, object orientation , and point of view, sign languagescan provide important . Four insight into the relation between linguistic and spatial representations of a will be examined: how functions as part linguistic system space major topics ; the relative efficiency of ) at various grammatical levels (American Sign Language signed and spoken languages for overt spatial description tasks; the impact of a visually basedlinguistic systemon performance with nonlinguistic tasks; and finally , . aspectsof the neurolinguistics of sign language

of Space in Signed 5.1 Multifunctionality Languages


In this section, I describe several linguistic functions of space in American Sign , I do not discussthe use of Language(ASL ) . The list is not exhaustive (for example ), but the discussionshould illustrate spaceto create discourseframes; seeWinston 1995 how spatial contrasts permeate the linguistic structure of sign languageAl though the discussion is limited to ASL , other signed languagesare likely to share most of the spatial properties discussedhere.

172

Karen Emmorey

Figure 5.1 Example of a phonological contrast in ASL . These signs differ only in the location of their articulation .

5.1.1 PhonologicalContrasts Spatial distinctions function at the sublexical level in signed languagesto indicate phonological contrasts. Sign phonology does not involve sound patternings or vocally based features , but linguists have recently broadened the term phonology to mean the " patterning of the formational units of the expressionsystemof a natural " , 5). Location is one of the formational units language (Coulter and Anderson 1993 of sign language phonology, claimed to be somewhat analogous to consonants in spokenlanguage(seeSandier 1989 , the ASL signsSUMMER , UGLY , ) . For example and D Ry1 differ only in where they are articulated on the body, as shown in figure 5.1. At the purely phonological level, the location of a sign is articulatory and does not carry any specific meaning. Where a sign is articulated is stored in the lexicon as .2 Sign languagesdiffer with respect to the part of its phonological representation phonotactic constraints they place on possible sign locations or combinations of locations. For example, in ASL no one-handed signs are articulated by contacting the contralateral side of the face ( Battison 1978 ) . For all signedlanguages , whether a sign is made with the right or left hand is not distinctive (left -handers and righthanders - what is distinctive is a contrast betweena dominant produce the samesigns and nondominant hand) . Furthermore, I have found no phonological contrasts in ASL that involve left -right in signing space . That is, there are no phonological minimal pairs that are distinguished solely on the basis of whether the signs are articulated on the right or left side of signing space . Such left -right distinctions appear to be reservedfor the referential and topographic functions of spacewithin the discoursestructure, syntax, and morphology of ASL (seebelow) . For a recent and comprehensivereview of the nature of phonological structure in sign language , see Corina and Sandier ( 1993 ).

DRY

'

OJ "

Ia UGLY SUMMER

The Confluenceof Spaceand Languagein Signed Languages

173

~ ~ ~ GIVE base form

;= ;:::::-~ ~ GIVE GIVE continuative habitual GIVE reciprocal

5.1.2 Morphological Inflection In many spoken languages , morphologically complex words are formed by adding es or suffix es to a word stem. In ASL and other signed languages prefix , complex forms are most often created by nesting a sign stem within dynamic movement contours and planes in space . Figure 5.2 illustrates the base form GIVE along with severalinflected forms. ASL has many verbal inflections that convey temporal information about the action denoted by the verb, for example , whether the action was habitual , iterative, or continual. Generally, thesedistinctions are marked by different movement patterns overlaid onto a sign stem. This type of morphological encoding contrasts with the primarily linear affixation found in spoken languages . For spoken simultaneous affixation es such as , languages process templatic morphology (e.g., in the Semitic languages ), infixation , or reduplication are relatively rare. Signed languages es such as reduplication; and , by contrast, prefer nonconcatenative process ' prefixation and suffixation are rare. Sign languages preference for simultaneously producing affixes and stemsmay have its origin in the visual-manual modality . For example , the articulators for speech(the tongue, lips, jaw) can move quite rapidly , producing easily perceiveddistinctions on the order of every 50- 200 milliseconds . In contrast, the major articulators for sign (the hands) move relatively such that the duration of an isolated sign is about 1,000 milliseconds slowly ; the duration of an average spoken word is more like 500 milliseconds. If language processingin real time has equal timing constraints for spoken and signedlanguages , then there is strong pressurefor signedlanguagesto expressmore distinctions simultaneously . The articulatory pressuresseem to work in concert with the differing capacities of the visual and auditory systems for expressing simultaneous versus sequential information . That is, the visual system is well suited for simultaneously perceiving a large amount of information , whereasthe auditory systemseems particularly adept at perceiving fast temporal distinctions. Thus both sign and speechhave exploited the advantagesof their respectivemodalities.

174

Karen Emmorey

8Thedog bites the cat.8 Figure 5.3 Example of the sentential use of spacein ASL . Nominals (cat, dog) are first associatedwith spatial loci through indexation. The direction of the movementof the verb (BITE ) indicates the grammatical role of subject and object.

5.1.3 Coreferenceand Anapllora Another hypothesizeduniversal use of spacewithin sign languagesis for referential functions. In ASL and other sign languages , nominals can be associatedwith locations . This associationcan be establishedby " indexing" or pointing in signing space to a location in spaceafter producing a lexical sign, as shown in figure 5.3. Another device for establishing the nominal-locus association is to articulate the nominal sign(s) at a particular location or by eye gazetoward that location. In figure 5.3, the nominal DOG is associated with a spatial locus on the signer' s left and CAT is associatedwith a locus on the signer' s right . The verb BITE moves between these " " locations identifying the subject and object of the sentence [ Thedog] bites [the cat] . BITE belongs to a subset of ASL verbs termed agreeing verbs3 whose movement and/ or orientation signal grammatical role. ASL pronouns also make use of established associationsbetween nominals and spatial loci. A pronominal sign directed toward a specificlocus refers back to the nominal associatedwith that locus. Further description of coreferenceand anaphora in ASL can be found in Lillo -Martin ( 1991 ) and Padden( 1988 ). Recently, there has been some controversy within sign linguistics concerning whether spaceitself performs a syntactic function in ASL . Liddell ( 1993 , 1994 , 1995 ) has argued that spatial loci are not morphemic. He proposesthat spacein sentences like those illustrated in figure 5.3 is being useddeictically rather than anaphorically. That is, the signer deictically points to a locus in the same way he would point to a physically present person. In contrast, other researchershave argued that these spatial loci are agreementmorphemes or clitics that are attached to pronouns and verbs (e.g., Janis 1995 ; Padden 1990 , ) . As evidence for his position, Liddell ( 1993 1995 ) arguesthat just as there is an unlimited number of spatial positions in which a

in Signed of Space and Language TheConfluence Languages

175

physically present referent could be located, there also appears to be an unlimited number of potential locations within signing space(both vertically and horizontally ) toward which a verb or pronominal form can be directed (seealso Lillo - Martin and Klima 1990 , then location specificationsare not listable or categorizable ) . If this is the case and therefore cannot be agreementmorphemesor clitics. The syntactic role of subject or object is assigned , not by the spatial loci , but either by word order or by the orientation or the temporal end points of the verb itself.4 According to this view, the particular location at which a verb begins or ends servesto identify the referent of the subject and object roles. The spaceitself, Liddell has argued, is not part of a syntactic representation ; rather, space is used nonmorphemically and deictically is much as deictic ) . This hypothesisis quite ( gesture usedwhen accompanyingspeech For radical, and many of the details have not beenworked out. , evenif space example itself does not perform a syntactic function , it does perform both a referential and a locative function within the language(seeEmmorey, Corina , and Bellugi 1995 ) . The association of a nominal with a particular location in spaceneedsto be part of the linguistic representation at some level in order to expresscoreferencerelations between . If this association is not part of the linguistic a proform and its antecedent then there must be an , extremely intimate mixing of linguistic structure representation . and nonlinguistic representationsof space 5.1.4 Locative Expressio The spatial positions associatedwith referentscan also convey locative infonnation about the referent. For example , the phraseDOG INDEX . shown in figure 5.3 could " be interpreted as " the dog is there on my left , but such an interpretation is not es required by the grammar. Under the nonlocative reading, INDEX simply establish a referencerelation between DOG and a spatial locus that happens to be on the ' signer s left. To ensurea locative reading, signersmay add a specificfacial expression (e.g., spread tight lips with eye gaze to the locus), produced simultaneously with the INDEX sign. Furthennore , ASL has a set of classifier fonDS for conveying specific locative infonnation , which can be embedded in locative and motion predicates ; for these predicates , signing space is most often interpreted as corresponding to a . The use of spaceto directly represent physical location in real (or imagined) space in contrast to spoken languages , in which spatial spatial relations stands marked infonnation must be recovered from an acoustic signal that does not map onto . In locative expressionsin the infonnation content in a one-to-one correspondence ASL , the identity of each object is provided by a lexical sign (e.g., TABLE , T -V , CHAIR ); the location of the objects, their orientation , and their spatial relation vis-a-vis one another are indicated by where the appropriate accompanyingclassifier sign is articulated in the space in front of the signer. The flat B handshape is

176

Room of classifie const layout Description layout using spatlallze - - Karen Emmorey

5.4 Figure
Example of an ASL spatial description using classifierconstructions.

-prorninent objects like the classifier handshapefor rectangular, fiat -topped, surface tables or sheetsof paper. The C handshape is the classifier handshape for bulky boxlike objects like televisionsor microwaves . The bent V is the classifier handshape " " for squat, legged objects like chairs, srnall anirnals, and seatedpeople. Flat B handshape : ~ C handshape : ~ Bent V handshape :~ These handshapesoccur in verbs that expressthe spatial relation of one object to another and the rnanner and direction of rnotion (for rnoving objects/people) . Figure 5.4 illustrates an ASL description of the roorn that is sketched at the far left. An " English translation of the ASL description would be I enter the roorn; there is a table to rny left , a TV on the far side, and a chair to rny right ." Where English uses separatewords to expresssuch spatial relations, ASL usesthe actual visual layout displayed by the array of classifiersignsto expressthe spatial relations of the objects. Landau and Jackendoff ( 1993 ) have recently argued that languages universally encodevery little information about object shapein their locative closed -classvocabulary (e.g., prepositions) cornpared to the arnount of spatial detail they encode in object narnes(seealso Landau, chapter 8, this volume) . As one can surmisefrorn our discussionand frorn figure 5.4, ASL appears to have a rich representation of shape in its locative expressions . Like the locational predicates in Tzeltal ( Brown 1991 ; Levinson 1992a ), ASL verbs of location incorporate detailed information about the shape of objects. It is unclear whether these languagesare counterexarnplesto Landau and Jackendoff' s clairns for two reasons . First , both Tzeltal and ASL express locative information through verbal predicates that form an open-class category, unlike prepositions (although the rnorphernesthat rnake up these verbal predicates belong to a closed class ) . The distinction rnay hinge on whether theseforms are con-

in Signed andLanguage TheConfluence of Space Languages

177

5.5 Figure Finalclassifier of either (2a) or (2b) . configurati 011


-classelementsor not (seealso Talmy 1988 sideredgrammaticizedclosed , in ) . Second . For ASL the degreeof shape detail is less in classifierforms than in object names , the flat B handshapeclassifier is used for both TABLE and for PAPER example the count nouns encodemore detailed shapeinformation about theseobjects than the classifier form . Thus, although the contrast is much less striking in ASL than in English, it still appearsto hold. ) has proposed several universal features that are associated with Talmy ( 1983 the figure object (i .e., the located object) and with the referenceobject or ground . For example , the figure tends to be smaller and more movable than the ground ' ): (from Talmy 1983 object. This asymmetry can be seenin the following sentences . ( 1) a. The bike is near the house b. me houseis near the bike. In English, the figure occurs first , and the ground is specified by the object of the as the figure preposition. When a large unmovable entity such as a house is expressed and This same between is odd. the sentence , figure ground semantically asymmetry objects occurs in ASL , except that the syntactic order of the figure and ground is reversedcompared to English, as shown in (2a) and (2b) (the subscripts indicate locations in space , the classifier in the first phrase is held in ) . In these examples line indicated the extended ) during the articulation of the second phrase space( by . In this with one hand ) way, the classifier handshape representing the ( produced figure can be located with respectto the classifierhandshaperepresentingthe ground ' object, as illustrated in figure 5.5 (the signer s left hand shows the classifier form for

178

Karen Emmorey

HOUSE ; her right hand showsthe classifierfonn for BIKE). The final classifier is the same for either(2a ) or (2b)- what differsis phrasalorder. configuration -CLASSI FIERa (2) a. HOUSEOBJECT BIKE VEHICLE-CLAS SI FIE Rnear a b. ?BIKE VEHICLE CLAS SI FIE Ra -CLASSI FIERneara HOUSEOBJECT to describe a seriesof fifty-six pictures , I askedeight native signers6 Recently between two objects (e.g., a dogundera chair, a car behind depicting simplerelations a tree almostinvariablyexpressed thegroundfirst, and thenlocatedthe ). The signers with to the . This figure respect ground object ordering may be an effect of the -spatialmodality of sign language visual . For example , to presenta scene visually , the ground tends to be producedfirst, and then the figure is through drawing locatedwithin that ground . Thus , whendrawinga pictureof a cup on a table , one would draw the table first and then the rather than draw the generally cup; cup in midair and then draw the table beneathit.7 More cross linguistic work will help -spatial modality conditionsall signedlanguages detenninewhetherthe visual to the groundand thenthefigurein locativeconstructions . preferto initially express like English ) also arguesthat prepositions (for languages Talmy ( 1983 ) ascribe to and . He evidence that all particular geometries figure ground objects presents ' characterize the figures geometrymuch more simply than the ground . languages Thefigureis oftenconceived of asa simplepoint, whereas thegroundobjectcanhave morecomplexgeometric . For example that the English , Talmy argues specifications across between and all out different , , along , prepositions among pick ground geo . metries At first glance in ASL. For , it appearsthat there is no such asymmetry in (2a construction , theclassifier example ) for theground(thehouse ) doesnot appear to be more geo metric to specifications ally complexthan the figure(the bike) with respect for shape(indicatedby classifierhand or for . The ) shape spatialgeometry locativeexpression in (2a that differentially ) doesnot appearto havea linguisticelement encodes in thewaythat prepositions do in spoken figureandgroundgeometries . Nonetheless that fact that signers conceive , the grammarof ASL reflects languages of thefigureasa point with respect to a morecomplex . in As shown ground (3a ) and 3b and in illustrated 5.6 of the can be to reduced a point, ( ) figure , expression figure but expression of the groundcannot : -CLASSI FIERa (3) a. HOUSEOBJECT BIKE POINTnear a b. ?HOUSEPOINTa BIKE VEHICLE-CLASSI FIERneara

The Confluenceof Spaceand Languagein Signed Languages

179

Final classifier consh"uction for (3a) .

Final classifier construction for (3b ) .

5.6 Figure -ground complexityappears to hold even about figure Thus Talmy' s generalization . relations itself to encode that canusespatialgeometry for languages spatial of Reference 5.1.5 Frames ASL can express , relative , or absoluteframe of spatialrelationsusingan intrinsic discussion of the linguisticand 4 this volume for reference see Levinson , , chapter , ( 8 a relative frame of reference frames , ). Within spatialpropertiesof thesereference . In who is signing of the person from the perspective scenes aremostoftendescribed . For this case , the origin of the coordinatesystemis the viewpoint of the signer All in 5.7. the shown were asked to describe ASL figure , eight picture signers example their one with the banana on their left but oneindicatedthat the bowl wason right ( without usingsigningspacein a toposignerprovideda descriptionof the scene ON SI -DE instead ). To indicatethat the graphicway, producingthe neutralphrase form for bowl on the left the classifier bananawason their right, signers produced was simultaneously form for banana a classifier and then side of signingspace , articulatedon the rig~t. 's from the addressee viewpoint9turn out to be more likely in the Descriptions ' is still than in the left-right dimension front-backdimension (the signers perspective in shown . In the the most likely for both dimensions figure 5.8, ) describing picture five of eight signers preferredtheir own viewpointand producedthe classifierfor for bowl articulatedaway from the chest banananear the chestwith the classifier

180

- ---~ ~~ ~ : : :A :~ = ~~ -=~
5.7 Figure
Illustration of one of the pictures that signerswere asked to describe .

Karen Emmorey

----

a. Signer 's viewpoint (5/ 8 signers) . Figure 5.8

b . Addressee 's viewpoint (3 / 8 signers ) .

The Confluenceof Spaceand Languagein Signed Languages

ISI

behind the classifierfor banana, as shown in figure 5.8a. This spatial configuration of classifiersignsmaps directly onto the view presentedin figure 5.8 (rememberthat you as the reader are facing both the signer and the picture) . In contrast, three signers 's describedthe picture from the addressee viewpoint , producing the classifierfor bowl near the chest and the classifier for banana in line with the bowl but further out in , as shown in figure 5.8b. This configuration would be the spatial arrangement signing space seenby an addressee standing opposite the signer (as you the reader are these ) . There were no overt linguistic cuesthat indicated figures doing when viewing which point of view the signer was adopting. However, signerswere very consistent in what point of view they adopted. For example , when the signerswere shown the reverseof figure 5.8, in which the banana is behind the bowl , all signersreversedtheir descriptions according to the viewpoint they had selectedpreviously. Note that the lack of an overt marker of point of view, the potential ambiguity , and the consistency within an adopted point of view also occur in English and other spoken languages ). (seeLevelt 1984 Bananas and bowls do not have intrinsic front / back features , and thus signers could not use an intrinsic frame of referenceto describethese pictures. In contrast, cars do have these intrinsic properties, and the classifier form for vehicles encodes : the front of the car is representedroughly by the tips of the index intrinsic features . Figures 5.9 and 5.10 illustrate ASL constructions and middle fingers, which are extended using the vehicle classifier, along with the corresponding pictures of a car in their different locations with respectto a tree. Again the majority of signersexpressed own view of the picture. In figures 5.9 and 5.10, the pictured female signer adopts her own perspective(describing the picture as she seesit ) , while the male signer adopts 's the addressee viewpoint . As noted above, lexical signs identifying the referents of the classifier signs are given first. Also as noted, the ground object (the tree) is first and generally held in spacewhile the lexical sign for car is articulated expressed and the vehicle classifier is placed with respectto the classifier for tree. The illustrations in figures 5.9 and 5.10 representthe final classifier construction in the description . As you can see , signersorient the vehicle classifier to indicate the direction the car is facing. Note that the orientation of the car is consistentwith the point of view lo adopted- the vehicle classifier is always oriented toward the tree. The majority of signers described figure 5.9 by placing the vehicle classifier to their left in signing . Only one signer placed the car on his right and the tree on his left. Again all space signerswere very consistentin which point of view they adopted, although one signer 's switchedfrom her own viewpoint in describing figure 5.9 to the addressee viewpoint for figure 5.10. There were no switches in viewpoint within either the left -right or front -back dimension. Signers were also consistent within the intrinsic frame of

) .

signers 7 / 2

viewpoint s Addressee . b .

) rs

signers ) in (

viewpoint Addresseels

182

's a. Signer

'

5.9 aDd 5.10 Fiaares

t ~ ~ ~ ~ ,

Karen Emmorey

TheConfluence in Signed of Space andLanguage Languages

183

reference , almost always changing the orientation of the vehicle classifier appropriately 11 (e.g., toward the left /right or away from /facing the signer). One question of interest is whether signerscan escape the relative point of view that is imposed " automatically " by the fact that signers(and addressees ) view their own articulators in spaceand thesearticulators expresslocative relations using this space . The answer appears to be that a relative framework is not necessarilyentailed in locative expressionsin ASL . That is, the expressionsshown in figure 5.9a and 5.9b could be interpreted as the rough equivalent of " the tree is in front of the car" 's without referenceto the signer' s (or addressee ) viewpoint . The car could actually be in any left-right or front -back relation with respectto the signer- what is critical to the intrinsic expressionis that the vehicleclassifieris oriented toward (facing) the tree. Thus the intrinsic frame of referenceis not dependentupon the relative frame; in ASL these two frames of referencecan be expressedsimultaneously. That is, linguistic expression within an intrinsic frame occurs via the intrinsic properties of certain classifierforms, and a relative frame can be imposed simultaneouslyon signing space if a viewpoint is adopted by the signer. Figures 5.9 and 5.10illustrate such simultaneous expression of reference frames. The linguistic and nonlinguistic factors that influence choice of viewpoint within a relative referenceframe have not been determined , although it is likely that severaldifferent linguistic and nonlinguistic factors are involved. And just as in English ( Levelt 1982a , 1984 ), frame of referenceambiguities can abound in ASL ; further researchwill detennine how addressee and signer are established altered and discourse . , , viewpoints disambiguatedduring Preliminary " " evidence suggeststhat , like English speakers(Schober 1993 ), solo ASL signers (such as those in this study) are less explicit about spatial perspectivethan signers with conversation partners. Finally , ASL signerscan usean absolute referenceframe by referring to the cardinal points east, west, north , and south. The signs for thesedirections are articulated as follows: WEST: W handshape , palm in , hand moves toward left12; EAST: E , palm out , hand movestoward right ; NORTH : N handshape , hand moves handshape SOUTH : S hand moves down. , up; handshape N handshape :~ E handshape : ~ S handshape : f' ) W handshape : SlY ( Thesesignsare articulated in this manner, regardlessof where the person is standing, that is, regardlessof true west or north . This situation contrasts sharply with how speakersgesture in cultures which employ absolute systems of reference such as

184

Karen Emmorey

certain Aboriginal cultures in Australia (see Levinson 1992b and chapter 4, this volume) . In thesecultures, directional gesturesare articulated toward cardinal points and vary dependingupon where the speakeris oriented. Although the direction of the citation forms of ASL cardinal signs is fixed, the movement of thesesigns can be changed to label directions within a " map" created in signing space . For example , the following directions were elicited from two signers describing the layout of a town shown on a map (from Taylor and Tversky 1992 ): STRAIGHT right hand traces a path outward from the signer " You drive " straight eastward. (5) UNDERSTAND MOUNTAIN R-D (4) YOU DRIVE EAST " e" handshapetracesthe samepath, palm to left

PATH NORTH " n" hand right hand shapetraces traces path samepath, palm in toward left , near signer " Understand that Mountain Road " goesnorth in this direction.

The signer who uttered (5) then shifted the map, such that north was centered outward from the signer, and the sign NORTH13 then traced a path similar to the one in (4), that is, centered and outward from the signer. It appears that ASL direction signs are either fixed with respectto the body in their citation form or they are usedrelative to the spacemapped out in front of the signer. As in English, it is the direction words themselvesthat pick out an absolute framework within which the discoursemust be interpreted. 5.1.6 Narrative Perspective In a narrative, a spatial frame of referencecan be associatedwith a particular character ; and Tversky, (seediscussionsof viewpoint in Franklin , Tversky, and Coon 1992 , this volume) . The frame of referenceis relative, and the origin of the chapter 12 coordinate system is the viewpoint of that character in the story . The linguistic mechanismsused to expresspoint of view in signed languagesappear to be more . Both signersand speakersuse linguistic devicesto explicit than in spoken languages indicate whether utterancesshould be understood as expressingthe point of view of " " the signer /speakeror of another person. Within narrative, point of view can mean either a visual perspectiveor the nonspatial perspectiveof a character, namely, that character' s thoughts, words, or feelings . Spoken languages have several different

The Confluence of Spaceand Languagein Signed Languages

185

devicesfor expressingeither type of perspective : pronominal deixis (e.g., use of J vs. demonstratives , there you), (here ), syntactic structure (active vs. passive ), and literary " " as well, styles(e.g., free indirect discourse ) . Signedlanguagesusethesemechanisms but in addition , point of view (in either sense ) can be marked overtly (and often " " continuously) by a referential shift. Referential shift is expressed by a slight shift in and or in head , body position / changes eye gaze position, or facial expression (for -Pedersen1993 discussionsof this complex phenomenon , see Loew 1983 ; Engberg ; Padden 1986 ; Lillo Martin 1995 ; Poulin and Miller 1995 ). The following is an exampleof a referential shift that would require overt marking of a spatial viewpoint . Supposea signer were telling a story in which a boy and a girl were facing each other, and to the left of the boy was a tall tree. If the signer wanted to indicate that the boy looked up at the tree, he or shecould signal a referential shift , indicating that the following sentences ) should be understood from the perspective of the boy. To do this, the signer would produce the sign LOOK -AT upward and to the left. If the signerthen wanted to shift to the perspectiveof the girl , he or shewould produce the sign LOOK -AT and direct it upward and to the right . Signers often ' ' expressnot only a character s attitudinal perspective , but also that character s spatial viewpoint through signsmarked for location and/ or deixis. Slobin and Hoiting ( 1994 , ' directional deixis p . 14 ) have noted that ~ , in that plays a key role in signedlanguages a path verb moves not only with respectto source and goal, but also with respectto sender and receiver , as well as with respect to points that may be established in to indicate the locations and viewpoints of protagonists set up in the signing space ." That spoken languagesexpressdeixis and path through separateelements discourse (either through two verbs or through a satellite expressionand a verb) reflects , they . That is, spoken languagemust , an inherent limitation of spoken languages suggest linearize deictic and path information , rather than expressthis information simultaneously . Deixis is easily expressedin signed , as is easily done in signed languages languagesbecausewords are articulated in the space surrounding the signer, such that " toward " and " away from " can be encoded simply by the direction of motion with respectto the signer or a referential locus in space . I would further hypothesize that this simultaneous expression of deictic and other locative information within the verbs of signed languagesmay lead to habitual expressionof spatial viewpoint within discourse . In sum, signedlanguages usespacein severaldifferent linguistic domains, including contrast co reference , , and locatives. The visual-gestural modality of phonological to influence the nature of grammatical encoding by comsigned languagesappears es (see pelling signed languages to prefer nonconcatenative morphological process also Emmorey 1995 ; Supalla 1991 ; Gee and Goodhart 1988 ) . Signed languagesoffer important insight into how different frames of referenceare specifiedlinguistically . A

186

Karen Emmorey

unique aspectof the visual-gegturalmodality may be that intrinsic and relative reference frames can be simultaneously adopted. In addition , shifts in referenceare often accompanied by shifts in visual perspectivethat must be overtly marked on deictic and locative verbs. Although spoken languagesalso have mechanisms to express deictic and locative relations, what is unique about signed languagesis that such . relations are directly encodedin space
5.2 Some Ramifications of the Direct Representation of Space

In the studies reported below, I explore some possible ramifications of the spatial encoding of locative and spatial contrasts for producing spatial descriptions and , I investigate ( I ) how ASL signersuse spaceto solving spatial problems. Specifically and directions commands , (2) to what extent signers use lexicalized expressspatial locatives in spatial directions, (3) whether the use of sign language provides an advantage for certain spatial tasks, and (4) how differences in linguistic encoding betweenEnglish and ASL affect the nature of spatial commands and directions. 5.2.1 Solving Spatial Puzzleswith SpatializedLanguage To investigatethesequestions , ten hearing English speakersand ten deaf ASL native a task in which they had to solve three spatial puzzlesby were compared using signers where to place blocks of different colors, shapes an , and instructing experimenter14 fit all blocks must . To solve the see 5.11 onto a sizes ) problem, figure puzzle grid ( within the puzzle outline. The data from English speakerswere collected by Mark St. . John ( 1992 ), and a similar but not identical protocol was used with ASL signers

[ ?

[ ? 1 2 3 4

. L >

ABCDEFGH

Figure5.11 : Subjects describe how to placeblockson a puzzle grid. Solvinga spatialpuzzle

in Signed of Space and Language TheConfluence Languages

187

English speakerswere instructed to si~ on their hands and were not pennitted to . Of course , ASL signers could use their hands, point to the puzzle or to the pieces but they were also not permit ted to point to the piecesor puzzle. For both signers and speakers , the subject and experimenter sat side by side, such that each had the samevisual perspectiveon the puzzle board. - encoded in either To explore how speakers and signers use spatial language spaceor sound- we examined different types of English and ASL instructions. We hypothesizedthat ASL signersmay be able to usesigning spaceas a rough Cartesian coordinate system , and therefore would rely less on the coordinates labeled on the ' puzzle board. This prediction was confirmed: 67% of the English speakers commands referred to the puzzle grid, whereasonly 28% of the commandsgiven by ASL signersreferred to the puzzle coordinates. This differencein grid referencewas statistically reliable (F ( I , 18 ) = 9.65; p < .01) . The following are sample commands containing references to the puzzle grid given by English speakers : (6) Take the blue L pieceand put it on HI H2 G2. (7) Place the red block in 3G H 2G. (8) Green pieceon EI , E2, D2 , C2, and D3. Instead of referring to grid coordinates, ASL signers used spacein various ways to indicate the positions on the puzzle board- for example , by tracing a distinctive part of the board in spaceor by holding the nondominant hand in space , representinga often an . ) part of the puzzle board ( edge We also compared how signers and speakersidentified the puzzle pieces to be ) . There were no significant differences placed for a given command (seefigure 5.12a in how either ASL or English was used to label a particular block. We had hypothesized that signers might make more referencesto shape becauseshape is often encoded in classifier handshapes(seediscussionabove). However, the numerical difference seenin figure 5.12awas not statistically significant. Languagedid not appear to influence how subjectslabeled the puzzle pieceswithin this task. There were significant differences , however, in the types of commands used by ASL signers and English speakers(see figure 5.l2b ) . Puzzle commands could be : ( I ) commands referring to a position on exhaustively divided into three categories the puzzle board, (2) commands expressinga relation between two pieces , and (3) . These categories were able to account for all of the orientation of a single piece the commands given by the twenty subjects . The only difference was that in ASL , two command types could be expressedsimultaneously. For example , signerscould simultaneously describe the orientation of a piece (through the orientation of a ' classifier handshape ) and that pieces relation to another block through two -handed

188
80 60

Karen

Emmorey

JO

8. Type of puzzle piece identification

b. Type of command reference

5.12 Figure
classifier constructions (see figure 5.15, as well as the constructions illustrated in figures 5.5, 5.9, and 5.10 ). English speakersproduced significantly more commands referring to a position on the puzzle board compared to ASL signers (F ( I , 18 ) = 4.47; p < .05) . English ' reliance on commands speakers involving coordinate specifications (see examples 6- 8) appearsto account for this differencein command type. It is interesting to note that even when ASL signers referred to grid coordinates, they often specified these coordinates within a vertical spatial plane, signing the letter coordinates moving crosswiseand the number coordinates moving downward. Thus the true horizontal " " plane of the board laying on the tabletop was reoriented into a vertical plane within signing space , as if the puzzle board were set upright . The linguistic and constraints on using a vertical versushorizontal plane to representspatial pragmatic are to be determined , but clearly useof a vertical plane doesnot necessar layouts yet ily indicate a true vertical relation betweenobjects. Subjectsdid not differ significantly in the percentageof commandsthat referred to the relation of one piece to another. Examples of English relation commands are given in (9)- ( II ): (9) Put the other blue L next to the green one. . ( 10 ) Put it to the left of the green piece ( II ) Switch the red and the blue blocks.

speakers

Orientation

English

Relation

on

board

Position

puzzle

60

10

50

20

40

30

. 'a C . E E 0 (,) '0 'E . e l

signers

Deaf . 70 Other 8peakers 81gners P08lt1on -

D88f Engl18h 8On Cortin . ~ P8 Shape Color 0 10 40 20 30 50

The Confluence of Spaceand Languagein Signed Languages

189

ASL signersalso produced thesetypes of commands , but generally space , rather than the relation . For betweenpieces , conveyed , the nondominant prepositional phrases example hand can representone block , and the dominant hand either points to a spatial locus to the left or right (somewhat like the construction illustrated in figure 5.6a) or the dominant hand representsanother block and is positioned with respect to the nondominant hand (seefigure 5.15 ). Finally , ASL signers produced significantly more commands that referred to the orientation of a puzzle piece (F ( I , 18 ) = 5.24; p < .05) . Examples from English of commands referring to orientation are given in ( 12 )- ( 14) : . ( 12 ) Turn the red one counterclockwise . ( 13 ) Rotate it 90 degrees ( 14 ) Flip it back the other way. For English speakers , a change in orientation was often inferred from where the had to fit on the board, given other non-orientation -specific commands . In piece contrast, ASL signers often overtly specified orientation . For example , figure 5.13 illustrates an ASL command that indicates a change in orientation by tracing a block ' s ultimate orientation in signing space (the vertical plane was often used to trace shapeand orientation ) . Figure 5.14 illustrates a command in which orientation change is specified by a change in the orientation of the classifier handshapeitself. Figure 5.15 illustrates the simultaneous production of a command indicating the

[pictured ]

:G 5.13 GREEN CL Figure

CL:G-orientation

. Orient the . green block in this wayo See green block in figure 5.11 ; note signe ~s perspective . Figure 5.13

190

Karen Emmorey

[pictured ] Figure5.14 BLUE L CL:L-orientation -Move the blue L so it is orientedwith the .long end outward

[pictured] 5. 15 RED L CL:B Figure CL:L -orientation - Move the red L so it is oriented len .Qthwiseat the top of another block [the green block ] . Figures5.14 and 5.15

. Signersalso used orientation of an L-shapedpiece and its relation to another piece the sign ROTA TE quite often and indicated the direction of rotation by movement of the wrist (clockwise vs. counterclockwise ). ASL also has a set of lexicalized locative signs that are used much lessfrequently than classifier constructions in spatial descriptions. The lexicalized locatives that were produced by signers in this study included IN , ON , AGAINST , NEAR , and BETWEEN . Only about 20% of ASL commandsinvolved lexical locatives, and these were almost always produced in conjunction with commands involving classifier constructions. The grammatical structure of theseforms is not well understood- are -Kegi I985) ?- and their ) or verbs (seeShepard they adpositions (seeMcIntire 1980

The Confluence of Spaceand Language in Signed Languages

Figure5.16 . Illustrationby Frank Allen Paulin Newell( 1983 ASL lexicalized locativesigns ).

IN

semantics has not beenwell studiedeither (seeMcIntire 1980for somediscussion of IN , UNDER, and OUT ) . The linguistic data from our study provided some of IN and ON (these signsare shownin figure interesting insightinto the semantics 5.16 ). in and on interchange usedthe prepositions ably to specifygrid Englishspeakers " " " " in H2 H2 commands 6 and7 for G2 or on G2 see coordinates , , ( sample example above usedthe lexicallocativeON in this context , but neverIN : ). ASL signers ( 15 ) PUT RED LON G2 H2 1213 15 ) PUT BLUE ] ( 16 [CL:G- shape tracedin shape verticalplane ( 17 ) . PUT RED L IN G2 H2 ON 3E 4F 3F 3G

The useof the prepositionin for describing grid positionson the puzzleboard falls " ' s 1986 " " under Herskovitz , namely , the reference ( ) category spatialentity in area " areasarisingfrom a dividing surface (p. 153 ). This objectmust be one of several structuredoesnot appearto be availablefor the ASL sign IN . particularsemantic -like as container did use IN when of the puzzlecould be construed aspects Signers " ' " , signers (falling under Herskovitzs spatial entity in a container ). For example woulddirectpieces to beplacedIN CORNER , two linesmeetto form ; 16in this case Herskovitz1986 a typeof container(see , 149 ). IN wasalsousedwhena block (most " " often the smallblue square ) wasplacedin a hole createdby other blockson the board or whena part of a block wasinsertedinto the part of the puzzlegrid that stuckout (seefigure 5.11 , the reference ) . In both cases objectforms a type of container . The useof the ASL lexicallocativeIN into which a block could be placed to be more restrictedthan Englishin, applyingonly whenthereis a clear appears . containment relation

192

Karen Emmorey

One might conjecture that the iconicity of the sign IN rendersits semanticstransparent - one hand representsa container, and the other locates an object within it . However, iconicity can be misleading. For example , the iconic properties of ON lead one to that its use might expect depends upon a support relation , with the nondominant hand representingthe support object. The data from our experiment, however, are not compatible with this hypothesis . ASL signersusedON when placing one block next to and contacting another block (e.g., the red piece ON the green in ): figure 5.11 ON GREEN ( ] 8) RED MOVE [CL :G - Lorientation] new orientation traced in horizontal plane " Move the red one so that it is oriented " lengthwisenext to the green. ( ] 9) RED [CL :G - shape ] THAT -ONE ROTATE [CL :L - orientation] clockwise [CL :B- reference shapetraced obj.] in upper to lower L classifier(right hand) is horizontal left oriented and positioned with respectto B classifier plane (left hand) as in figure 5.] 5 ON GREEN " Rotate that red L shaped block clockwise so that it is oriented lengthwise at the top of the green." English speakers never produced commands relating one block to another using " only the preposition on. Given the nature of the puzzle, subjectsnever said put the " red block on the green one. The support requirementsdescribedby Herskovitz for on in English do not appear to apply to the lexical locative glossedas ON in ASL . This difference in semantic structure highlights the difficulties of transcribing one -Kegl ] 985) . English of another (seealso discussionin Shepard languageusing glosses on is not equivalent in semanticsor syntax to ASL ON (seeBowerman, chapter ] 0, this volume, for further discussionof languagevariation and topological concepts ). Finally , the ability to linguistically represent objects and their orientations in spacedid not provide signerswith an advantageon this complex spatial task. Signers and speakersdid not differ in the number of moves required to solve the puzzlesnor in the number of commands within a move. In addition , ASL signers and English speakersdid not differ significantly in the time they took to solve the puzzles , and both groups appeared to use similar strategiesin solving the puzzle. For example , subjectstended to place the most constraining piece first (the green block shown in figure 5.] I ) . In summary, English speakersand ASL signersdiffered in the nature of the spatial commands that they used for positioning objects. Signers used both vertical and

of Space TheConfluence

Languagein Signed Languages

193

. Changesin horizontal planes of spaceitself as a rough Cartesian coordinate system in the were orientation position of spatial changes directly through expressed object . In contrast in orientation and and , English classifiers signing space by tracing shape speakerswere lesslikely to overtly expresschangesin orientation and relied heavily on direct referenceto labels for coordinate positions. The heart of this different useof spatial languageappearsto lie in the properties of the aural vocal and visual manual , in ASL , the hands can directly expressorientation linguistic modalities. For example by their own orientation in space such direct representation within the linguistic . Finally , ASL and English differ in the signal is not available to English speakers semanticsthey assign to lexicalized locatives for the topological concepts in and on, and the semantic structure of the ASL locatives cannot be extracted from the iconic properties of the forms. In the following study, we further explore the effect modality . may exert on the nature of spatial languagefor both spoken and signedlanguage 5.2.2 Room Description Study Eight ASL signersand eight English speakerswere asked to describe the layout of " " objects in a room to another person ( the manipulator ) who had to place the objects 17 (pieces of furniture ) in a dollhouse. In order to elicit very specific instructions " and to eliminate (or vastly reduce , and interruptions , the , feedback ) interchanges " describer (the person giving the instructions) could not see the manipulator , but the manipulator could seethe describer through a one-way mirror (seefigure 5.17 ).

Figure5.17 -up for room descriptions . set Experimental

. . . , . J . , . . ~ A . ( ~ ~ Describer , , I ! f ; I . I way . I . . mirror . one Manipulator ~ ft ~ 8 Q ~ n g

194
100

Karen Emmorey
0 . 4

~
60 40 20 Haphazard Normal

8. Doll house room description .

b. Accuracy of manipulators .

5.18 Figure
The manipulator could not ask questions but could request that the describer pause or produce a summary. Subjectsdescribed six rooms with canonical placementsof furniture (" normal rooms" ) and six rooms in which the furniture had been strewn about haphazardly without regard to function (" haphazard rooms" ) . The linguistic data and analysis arising from this study are discussed elsewhere (Emmorey, Clothier , and McCullough ) . However, certain results emerged from the study that illuminate some ramifications of the direct representation of space for signed . languages Signerswere significantly faster than speakersin describing the rooms (F ( I , 14 )= 5.00; p < .05; seefigure 5.18a . Mean time for ASL was 2 min ) description ,4 signers ' sec an of 2 min ; English speakers required , 48 sec to describe the same average ' rooms. In one way, the speedof the signers descriptions is quite striking because , on , ASL signs take twice as long as English words to articulate (Klima and average ; Emmorey and Corina 1990 Bellugi 1979 ) . However, as we have seenthus far in our discussion of spatial language in ASL , there are several modality -specific factors that would lead to efficient spatial descriptions and lessenthe need for discourse linearization ( Levelt 1982a . For example , b), at least to some degree , the two hands can represent two objects simultaneously through classifier handshapes , and the orientation of the hands can also simultaneously representthe objects' orientation . The position of the hands in spacerepresentsthe position of the objects with respect to each other. The simultaneousexpressionof two objects, their position, and their

type

Room

Room type

Haphazard

arms

80

Speakers

Signers English . ~

Deaf

c E .. E . c 0 " 'S .o c C ) 'I a C I ~


5 0 5 0 5 0 . . . . . _ 3 1 3 2 2 1

TheConfluence of Space andLanguage in Signed Languages

195

orientation standsin contrast to the linear strings of prepositions and adjunct phrases that must be combined to expressthe sameinformation in English. -accuracy trade-off. The difference in description time was not due to a speed Signers and speakersproduced equally accurate descriptions, as measured by the percent of furniture placed correctly by the manipulators in each group (seefigure 5.18b ) . There was no significant differencein percent correct, regardlessof whether a lenient scoring measurewas used(object misplacedby more than 3 cm or misoriented ; representedby height of the bars in figure 5.18b by 45 degrees ) or a strict scoring measurewas used(object misplacedby I cm or misoriented by 15 degrees ; shown by the line in each bar in figure 5.18b . ) To summarize , this second study suggeststhat the spatialization of American Sign Languageallows for relatively rapid and efficient expressionof spatial relations and locations. In the previous study, we saw that ASL signersand English speakers focused on different aspectsof objects within a spatial arrangement , as reflected by instructions for the of blocks within a coordinate differing placement plane. These differences arise, at least in part, from the spatial medium of signed languages , . compared to the auditory transmission of spoken languages

S.3 Interplaybetween andSpatialCognition Spatialized Language


We now turn to the relation between general nonlinguistic spatial cognition and processinga visual-spatial linguistic signal. Does knowing a signed language have ? In a recent investigation, Emmorey, any impact on nonlinguistic spatial processing and 1993 examined the relation betweenprocessingASL and the Kosslyn, ) Bellugi ( useof visual mental imagery. Specifically , we examinedthe ability of deaf and hearing , to generate mental images subjects to mentally rotate images , and to maintain in this last skill will not be discussed . We here images memory ( ) hypothesized that theseimagery abilities are integral to the production and comprehensionof ASL and that their constant use may lead to an enhancementof imagery skills within a nonlinguistic domain. In order to distinguish the effectsof using ASL from the effectsof deaf from birth , we also tested a group of hearing subjectswho were born to being deaf parents. These subjectslearned ASL as their first languageand have continued to useASL in their daily lives. If thesehearing native signershave visual-spatial skills similar to those found for deaf signers , this would suggestthat differencesin spatial . On the other hand, if these cognition arise from the useof a visual-spatial language have visual skills similar to those found in signers , this would spatial hearing subjects that differences in suggest spatial cognition may be due to auditory deprivation from birth .

196

Karen Emmore~

We hypothesized that mental rotation may playa crucial role in sign language processingbecauseof the changesin spatial perspectivethat can occur during referential shifts in narrative (seeabove) and the shifts in visual perspectivethat occur . As discussedearlier, during sign comprehension the between signer and addressee must mentally reversethe spatial arrays created often i.e. the addressee ) perceiver ( , for , a spatial locus established on the right of example by the signer such that , the person signing (and thus on the left of the addressee ) is understood as on the ). Because right in the scenebeing describedby the signer (seefigures 5.9a and 5.10a 's ' scenes are most often describedfrom the signer s perspectiveand not the addressee , this transformation processmay occur frequently. The problem is not unlike that facing understandersof spoken languageswho have to keep in mind the directions " " left " and " . The crucial difference for ASL is that right with regard to the speaker thesedirections are encodedspatially by the signer. The spatial loci usedby the signer to depict a scene(e.g., describing the position of objects and people ) must therefore observes be understood as the reverseof what the addressee during discourse actually (assuming a face to face interaction) . Furthermore, in order to understand and would must perceivethe reverseof what they themselves processsign, the addressee of with this have . aspect learning great difficulty produce Anecdotally, hearing subjects ' s articulations into the reversal that transform a do not ASL ; they signer easily . Given theselinguistic processingrequirements must be usedto produce the signs , we at than would be better that mentally rotating hearing subjects signers hypothesized , we used imaged objects and making mirror imagejudgments. To test this hypothesis in which a task similar to the one devised by Shepard and Metzler ( 1971 ) subjects . Subjects were shown two forms createdby juxtaposing cubesto form angular shapes were asked to decide whether the two shapes were the same or mirror images , . see 5.19 of orientation ) ( figure regardless Our results support the hypothesis that use of ASL can enhancemental rotation skills (seethe top illustration in figure 5.19 ); both deaf and hearing signershad faster of rotation . Note that the slopes reaction times compared to nonsignersat all degrees for the angle of rotation did not differ betweensigning and nonsigning groups, and this indicates that signersdo not actually rotate images faster than nonsigning subjects . Emmorey Kosslyn, and Bellugi ( 1993 ) originally suggestedthat ASL signers , particularly because they were faster even may be faster in detecting mirror reversals when no rotation was required (i.e., at zero degrees ) . However, recent researchby 18 es may be involved when Ilan and Miller ( 1994 ) indicates that different process within a mental rotation experiment made at zero are mirror samejudgments degrees , compared to when mental rotation is not required on any of the trials. In addition , preliminary results from Emmorey and Bettger indicate that when native ASL signersand hearing nonsignersare asked to make mirror -samejudgments in a

18A81

UO

-0r

The Confluence of Spaceand Languagein Signed Languages

. i e i 8 . ! ~ mm c c I I i . c 0 I I ~ ' i 6 0 ~ . m I I ~ I 0 '. ~ m ! ! ii ~ ~ o < t & a . . s ~ i ~ I r fD c 6c ~ ~ t t


6J

198

Karen Emmorey

comparison task that does not involve mental rotation , these groups do not differ in accuracy or reaction time . The faster response times exhibited by signers on the mental rotation task may reflect faster times to initiate mental rotation or faster times to generate a mental image ( as suggested by the next experiment ) . Finally , the finding that hearing native signers performed like deaf signers indicates that enhancement on this mental rotation task is not a consequence of auditory deprivation . Rather , it appears to be due to experience with a visual language whose production and interpretation may involve mental rotation ( see also Talbot and Haude 1993) . Another visual imagery skill we investigated was the ability to generate mental images, that is , the ability to create an image ( i .e., a short - term visual memory representation ) on the basis of information stored in long - term memory ( see Kosslyn et al . 1985) . In ASL , image generation may be an important process underlying aspects of referential shift . Liddell ( 1990) argues that under referential shift , signers may imagine referents as physically present , and these visualized referents are relevant to the expression of verb agreement morphology . Liddell gives the following

example involving the verb ASK which is lexically specified to be directed at chin height (seefigure 5.20) : To directthe verbASK towardan imagined referent mustconceive of the location , the signer ' s head . For example andaddressee wereto imagine that of theimaginary referent , if thesigner Wilt Chamberlain wasstanding beside themreadyto givethemadviceon playingbasketball , ' s head thesignASK wouldbedirected upwardtowardtheimaged heightof Wilt Chamberlain 's chin (figure (figure[5.20a ]). It would be incorrectto signthe verbat the heightof the signer is present . Naturally workswhena referent , if the [5.20b ]). This is exactlythe way agreement ., the heightand directionof the as layingdown on a chair, etc referentis imagined , standing the locationof body partsof verbreflects this. Since the signermustconceptualize agreement

a. addressee- ASK - imagined tall referent

b. * addressee- ASK - imagined tall referent

Figure5.20 verbsand referents imagined as present. Illustration fromLiddell Agreement ( 1990 ).

The Confluenceof Spaceand Languagein Signed Languages

199

the referent to be present in whichan invisiblebody is present . The , thereis a sense imagined sucha body in order to properlydirect agreement verbs . (Liddell signermustconceptualize 1990 , 184 ) If deaf subjectsare in fact generatingvisual imagesprior to or during sign production , then the speed of forming these images would be important , and we might . The imagegeneration expectsignersto developenhancedabilities to generateimages task we used is illustrated at the bottom of figure 5.19. Subjects first memorized uppercaseblock letters and then were shown a seriesof grids (or setsof brackets ) that contained an X mark. A lowercaseletter precededeachgrid , and subjectswere asked to decide as quickly as possible whether the corresponding uppercaseblock letter would cover the X if it were in the grid . The crucial aspectof the experiment was that the probe mark appearedin the grid only 500 ms after the lowercasecue letter was . This was not enough time for the subjectsto complete forming the letter presented thus image; responsetimes reflect in part the time to generatethe image. Kosslyn and colleagueshave used this task to show that visual mental images are constructed ; Roth and Kosslyn 1988 serially from parts (e.g., Kosslyn et ale 1988 ). Subjectstend to generate letter images segment by segment in the same order that the letter is drawn. Therefore, when the probe X is covered by a segmentthat is generatedearly (e.g., on the first stroke of the letter F ), subjectshave faster reaction times, compared to when the probe is located under a late-imaged segment . Crucially, this difference in responsetime basedon probe location is not found when image generation is not involved, that is, when both the probe X and letter (shaded gray) are physically . present Our results indicated that both deaf and hearing signersformed imagesof complex letters significantly faster than nonsigners(seefigure 5.19 that ) . This finding suggests with ASL can affect the to visual . Results experience ability mentally generate images from a perceptual baselinetask indicated that this enhancementwas due to adifference in image generation ability , rather than to differencesin scanning or inspection - signersand nonsignersdid not differ in their ability to evaluate probe marks when the shapewas physically present . The signing and nonsigning subjectswere equally accurate which that , suggests although signers create complex images faster than .. Furthermore, deaf and hearing , both groups generateequally good images nonsigners subjects appeared to image letters in the same way: both groups of subjects required more time and made more errors for probes located on late-imaged segments , and theseeffectswere of comparable magnitude in the two groups. This result indicatesthat neither group of subjectsgeneratedimagesof lettersas completewholes , and both groups imaged segmentsin the sameorder. Again, the finding that hearing that their enhancedimagegeneration signersperformed similarly to deaf signerssuggests is due to with ASL rather than to auditory deprivation . , ability experience

200

Karen Emmorey

esa relation betweenvisual-spatial imagery within linguistic This researchestablish and nonlinguistic domains. Image generation and mental rotation appear to be es that must obviously be deeply embeddedin using ASL , and theseare not process involved in both visual imagery and ASL perception. Note that these experiments have focused on ASL processing ; whether there is a more direct relation in sign between linguistic representations(e.g., conceptual structure, see Jacken language . doff , chapter I , this volume) and spatial representationsis a topic for future research

andSpoken for Signed 5.4 NeuralCorrelates Languages


Finally , sign languageexhibits properties for which each of the cerebral hemispheres of hearing people showsdifferent predominant functioning . In general , the left hemisphere has beenshown to subservelinguistic functions, whereasthe right hemisphere es linguistic functions is dominant for visual-spatial functions. Given that ASL express brain contrasts what is the , organization for sign by manipulating spatial ? Is sign languagecontrol led by the right hemispherealong with many other language visual-spatial functions or does the left hemispheresubservesign languageas it does of the ? Or is sign languagerepresentedequally in both hemispheres spoken language that the Edward Klima have shown and brain? Howard Poizner, Ursula Bellugi, brain honors the distinction betweenlanguageand nonlanguagevisual spatial functions ; Bellugi, Poizner, and Klima 1989 ) . Despite (Poizner, Klima , and Bellugi 1987 the visual-spatial modality of signedlanguages , linguistic processingoccurs primalily within the left hemisphereof deaf signers , whereasthe right hemisphereis specialized . Poizner, Bellugi, and for nonlinguistic visual spatial processing in these signers Klima have shown that damage to the left hemisphereof the brain leads to sign , adult aphasiassimilar to classicaphasiasobservedin speakingpatients. For example " " charac with left hemispheredamage may produce agrammatic signing, signers terized by a lack of morphological and syntactic markings and often accompaniedby halting , effortful signing. An agrammatic signer will produce single-sign utterances that lack the grammatically required inflectional movements and use of space (see discussion above) . In contrast, right -hemispheredamage produces impairments of . When many visual-spatial abilities, but does not produce sign language aphasias given testsof sign languagecomprehensionand production (e.g., from the Salk Sign ), signers with right -hemisphere Aphasia Exam; Poizner, Klima , and Bellugi 1987 damage perform normally , but these same signers show marked impairment on , when given a set of nonlinguistic tests of visual-spatial functions. For example them to match a model (the W AIS blocks test), colored blocks and asked to assemble have signers great difficulty and are unable to capture the right hemispheredamaged

in Signed of Space and Language TheConfluence Languages

201

overall configuration of the block design. Similar impairments on this task are found . with hearing, speaking subjectswith right -hemispheredamage 1987 also that some Poizner, Klima , and Bellugi ( ) reported signing patients with right hemispheredamageshow a selectiveimpairment in their ability to use spaceto expressspatial relations in ASL , for example when describing the layout of furniture in their room or apartment. Their descriptions are not ungrammatical, but they are incorrect when compared to the actual layout of objects. One hypothesis for this dysfunction following right -hemispheredamageis that , unlike spoken language , ASL requires that the cognitive representationof spatial relations be recoveredfrom and instantiated within a spatialized linguistic encoding (i.e., cognitive spatial relations , not to sound) . Evidencesupporting this hypothesiscomesfrom a map to space bilingual hearing patient with right -hemisphere damage studied by David Corina and colleagues(Corina et al. 1990 ; Emmorey, ; Emmorey, Corina , and Bellugi 1995 . from this case that there be more Hickok , and Corina 1993 The data may ) suggest information encoded within a involvement when right hemisphere processingspatial . linguistic description for signedcompared to spoken languages The caseinvolves female patientD .N ., 19a young hearing signer (age 39), bilingual in ASL and English, who was exposed to ASL early in childhood. She underwent surgical evacuation of a right parietal-occipital hematoma and an arteriovenous malformation . Examination of a magnetic resonanceimaging (MRI ) scan done six months after the surgery revealeda predominantly mesial superior occipital-parietal lesion. The superior parietal lobule was involved, while the inferior parietal lobule was spared , although someof the deepwhite matter coming from this structure may also be involved. The comparison test betweenEnglish and ASL spatial commands (seebelow and figure 5.21) was conducted by Corina approximately one year after DiNis surgeryD .N . was not aphasic for either English or ASL . Her performance on the Salk , and she showed no linguistic deficits Sign Diagnostic Aphasia Exam was excellent for English. Nevertheless , sheexhibited a striking dissociation betweenher ability to comprehendand produce spatial descriptionsin English compared to ASL . Although her English description had no evident spatial distortions , she was impaired in her ability to describe the spatial layout of her room using ASL . Her ASL description showeda marked disorganization of the elementsin the room. Her attempts to place one set of objects in relation to others were particularly impaired, and sheincorrectly specifiedthe orientation and location of items of furniture (seealso Emmorey, Corina, and Bellugi 1995 ). Corina ( 1989 ) developeda specificset of tasks to investigateD .Nis comprehension of locative relations in English and ASL . One of thesetasks required DiN . to set up

202
: instruction ASL

Karen Emmorey

Figure5.21 's differential ASL in comprehending English versus Illustrationof a RHO patient performance . shown are not PENCIL and ) (the lexicalsignsPAPER spatialcommands real objects in accordance with spatial descriptions given in either English or in " " ASL . An exampleof a simple English instruction would be The pen is on the paper. The English and ASL instructions along with DiNis responsesare illustrated in figure 5.21. DiN . correctly interprets the English command, but fails with the ASL instructions. This particular example was elicited through informal testing by Corina in which the same instructions were given in both English and ASL . DiN . was later given 36 different spatial commands ( 18 in English and 18 in ASL ) which involved from two to four objects (e.g., cup, pen, book ) . The instructions were . When matched for number of spatial relations that were encodedin each language D .N . was given instructions in English to locate objects with respectto one another, she performed relatively well- 83% correct. Her score was worse than her normal -matchedbilingual control ( 100 % correct), but better than other right -hemisphere age damaged subjects who were given the English test (69% correct) . However, when presentedwith similar information in ASL in which spatial relations are presented topo graphically in sign spaceD .N . made many more spatial errors, scoring only 39% correct. This result is particularly striking , given the iconicity of the ASL descriptions (seefigure 5.21) .

CL

instruction

paper

on

ASL

PENCIL

to

response

CL

incorrect

PAPER

DiNis

: paper the response instruction on

is

instruction lish correct En pencil ~ to DiNis he English ~

The Confluence of Spaceand Languagein Signed Languages

203

We hypothesize that the dissociation betweenD .Nis comprehension of English of the highly specificspatial realization of and ASL spatial commands arisesbecause ASL classifier constructions. That is, spatial relations must be recovered from a visual-spatial signal in which much more information is encoded about the relative position and orientation of objects, compared to English. Furthermore, the requirement of reading off spatial relations directly from the orientation and position of es classifier signs in spacemay make additional demandson spatial cognitive process . Nis comprehensionimpairment is not linguistic per within the right hemisphereD se , but stemsfrom the fact that linguistic information about spatial relations must be recoveredfrom a representationthat itself is spatialized; DiN . doesnot have difficulty understanding ASL spatial contrasts that do not encodeinformation about location or orientation . Thus the caseof DiN . also bears on our earlier discussionconcerning . referential versustopographic functions of spacein ASL . DiN . exhibits a dissociation -level between the use of signing space as a linguistic device for marking sentence referential distinctions and the useof signing spaceas a topographic mapping device (seeEmmorey et al. 1995for a complete discussionof this dissociation and for additional -processingexperimentswith normal ASL signers evidencefrom language ). In conclusion, signed languagesoffer a unique window into the relation between . All current evidence indicates that signed languagesare constrained language and space . Thus far , there is no by the same principles that shape spoken languages evidence that signed languagesgrammaticize different aspectsof the spatial world ). What is different and unusual compared to spoken languages(see Supalla 1982 about signed languagesis their visual spatial form - the fact that spaceand movement can be used to linguistically representspaceand movement in the world . This chapter has explored the ramifications of this spatialized encoding for the nature of , for spatial cognition in general , and for linguistic structure, for languageprocessing . Future researchmight include investigations of the neural substrateof sign language the following : ( I ) the semanticand grammatical structure of locative constructions in different sign languages(how do sign languagesvary in the way they utilize physical ?); (2) when and how signing spaceto representtopological and other spatial concepts the children acquire locative vocabulary (what is developmental relation between spatial cognition and sign languageacquisition? SeeMandler , chapter 9, this volume, and Bowerman, chapter 10, this volume, for discussion of spatial cognition and spoken language acquisition); (3) spatial attention in sign language perception and nonlinguistic visual-spatial perception (do signers show differencesin spatial attention ?); (4) how signersbuild that could be attributed to experiencewith sign language spatial mental models (doessigning spaceoperate like a diagram? SeeJohnson- Laird , chapter II , this volume); and ( 5) the neural substrateand psychological mechanisms

204

Karen Emmorey

that underlie the mapping betweena linguistic signal (both signed and spoken ) and . Theseare only someof the areasin which the study an amodal spatial representation of sign languagecould enhanceour understanding of the relation betweenlanguage and space .
Acknowledgments

This work wassupported , ROI DC by National Institutesof Health grantsROI DC 00201 and HD 13249 . I thank David Corina Hickok and Ed Klima for many 00146 R37 , , Greg , here . Merrill Garrett and Mary Peterson about the issues presented insightful discussions comments on an earlierdraft of this chapter . I alsothank BonitaEwanand providedvaluable consultants and who werethe sign language SteveMcCullough , who weremy primary language . Mark Williamshelped in this chapter . models for the figures create of many the figures who participatedin Finally, I am particularlygratefulto the GallaudetUniversitystudents these studies .
Notes

. The glossrepresents I . Wordsin capital lettersrepresent for ASL signs the Englishglosses . A subscripted of the unmarked root form of a sign word followinga , unmodulated meaning in form associated with a that the signis madewith someregularchange signglossindicates , and thus indicatesgrammatical morphologyin ASL (e.g., systematic changein meaning connected areusedwhenmorethan oneEnglish GI VEbabltu by hyphens ..)' Multiword glosses are usedto indicate word is requiredto translatea singlesign(e.g., LOOK-AT) . Subscripts verbsare markedwith a subscript to indicatethe , pronouns , and agreeing spatialloci; nouns CL, loci at which they are signed(e.g. INDEX . , BIT~ ). Classifierformsare abbreviated in italics(CL:Gof theclassifier anda description of themeaning followedby thehandshape . the gloss of how a classifier ). Descriptions signis articulated may be givenunderneath shape areprovidedin quotes . Englishtranslations in the lexiconfor location(see 2. Some pronouns maynot be specified signssuchaspersonal Lillo-Martin and Klima 1990 ; Liddell 1994 ). verbsare indicating 3. OthertenDSthat havebeenusedfor these ) and inflecting (Liddell 1995 1988 (padden ). ' s movement 4. Whethersubjectis associated with the beginning or end of the verb depends " " 1988 of verb(cf. backwards verbs , Padden ; Brentari1988 ). upontheclass that a sentence is mark (1) indicates 5. Followingtraditionallinguistictypography , a question is unacceptable . that the sentence considered ; a star(* ) indicates marginal weredeafindividuals who wereexposed to ASL from birth. 6. In this study , nativesigners to me by Dan globin, who has made similar 7. The exampleof drawing was suggested about scene (Slobin and arguments settingand the effectof modality on signedlanguages ). Hoiting 1994

The Confluenceof Spaceand Languagein Signed Languages

205

" in a 8. Sign linguists often use " frame of reference , referring to anaphoric nonspatial sense referencein a discourse(seeespeciallyEngberg Pedersen1993 ). 9. The addressee is assumedto be facing the signer. Signersdescribedthesepictures to a video camera rather than to an actual addressee . In understanding this discussionof point of view in ASL , it might be useful for you the reader to imagine that you and the signer viewed the display from the same vantage point , and now the signer is facing you (the addressee ) to describeit . 10. It should be noted that occasionally a signer may ignore the orientation features of the vehicle classifier , say, pointing the vehicle classifier toward the tree classifier , when in actual fact the car is facing away from the tree. This may occur when it is difficult to produce the correct orientation , say, pointing the vehicleclassifierto the right with the right hand, palm out (try it ) . II . There were only six examples(out of thirty -five) in which a signer ignored the orientation of the car becauseit was awkward to articulate. Also , signersdid not always alternate which hand produced the classifier for TREE , as might be implied by figures 5.9 and 5.10. 12. Except for the sign LEfT , WEST is perhaps the only sign that is specified as moving toward the signer' s left rather than toward the " nondominant side." For both left- and , the sign WEST moves toward the left , and the sign EAST moves toward the right -handers ' right . The direction of movement is fixed with respectto the signer s left and right , unlike other , right - and left -handers would articulate the signs illustrated in figure 5.1, signs. For example which also move acrossthe body, with opposite directions of motion (left to right vs. right to left , respectively ) . However, there is somechangein articulation for left -handers , perhaps due to phonological constraints. For EAST and WEST, the orientation of the palm is reversed : outward for WEST and inward for EAST. This changein palm orientation also occurs when a right -handed signer articulates EAST or WEST with the left hand (switches in hand dominance are phonologically and discoursegoverned ). 13. When the signs NORTH and SOUTH are used to label paths within a spatial map, they often retain someof their upward and downward movement. 14. This study was conducted in collaboration with Shannon Casey ; the experimenter was either a native speakerof English (for the English subjects ) or a deaf ASL signer (for the deaf subjects ). IS. This is not an orientation command but a shapedescription, namely, a classifierconstruction in which the shapeof the blue puzzle piece is traced in the vertical plane (seefigure 5.13 for an example ). 16. CORNER is a frozen classifier construction produced with nominal movement (Supalla and Newport 1978 ) . The sign can be articulated at various positions in spaceto indicate where the comer is located (e.g., top left or bottom right) . 17. This study was conducted with Marci Clothier and StephenMcCullough . 18. I thank Mary Petersonfor bringing this work to my attention. 19. Poizner and Kegl ( 1992 ) also discussthis patient, but usethe pseudonyminitials A .S.

206 Refereaces

Karen Emmorey

Battison, R. ( 1978 . Silver Spring, MD : Linstok ) . Lexica/ borrowing in American Sign Language Press .

- 388 . Neurosciences , 10 , 380 Brentari verbsin ASL: Agreement . In Papers re-opened , D. ( 1988 ). Backwards from the - 27. Chicago Parasession on Agreement in Grammatical : Chicago , vol. 24 , no. 2, 16 Theory . LinguisticSociety Brown in Tzeltal . Working paperno. 6, CognitiveAn, P. ( 1991 ). Spatialconceptualization . , Nijmegen thropologyResearch Group, Max PlanckInstitutefor Psycholinguistics Corina relations testbatteryfor ASL. Unpublished , D. ( 1989 , Salk ). Topographic manuscript Institutefor BiologicalStudies , La Jolla, CA. ' -Batch Corina , D., Bellugi , U., Kritchevsky , M., O Grady , L., andNonnan , F. ( 1990 ). Spatial in relations signed versus : Cluesto right parietalfunctions . Paperpresented spoken language at the Academy of Aphasia . , Baltimore Corina in signlanguage structure . , D., and Sandier , W. ( 1993 ). On the natureof phonological - 207 . , 1O , 165 Phonology Coulter and , G. R., andAnderson , S. R. ( 1993 ). Introductionto G. R. Coulter(Ed.) Phonetics : Currentissues in ASLphonology . SanDiego . Press , CA: Academic phonology in signlanguage : Effects of phonetic , K., andCorina , D. ( 1990 ). Lexicalrecognition Emmorey - 1252 structure andmorphology . Perceptual . andMotor Skills, 7J, 1227 of topographic and , K., Corina , D., and Bellugi , U. ( 1995 Emrnorey ). Differentialprocessing referentialfunctionsof space . In K. Emrnorey and J. Reilly (Eds , , ), Languagegestureand . , 43- 62. Hillsdale , NJ: Erlbaum space , K., Hickok, G., and Corina , D. ( 1993 Emmorey ). Dissociationbetween topographicand in ASL. Paperpresented at the Academyof AphasiaMeeting , syntacticfunctionsof space Tucson . , AZ , October -spatial , K ., Kosslyn , S. M., and Bellugi , U. ( 1993 Emmorey ). Visual imageryand visual : Enhanced abilities in deaf and ASL . 46 , language imagery hearing signersCognition , 139 181 . -Pedersen in Danish : Thesemantics andmorphosyntax , E. ( 1993 ). Space Engberg SignLanguage in a visual . InternationalStudies on SignLanguage Research and of theuseof space language Communication of the Deaf . Hamburg : Signum . , vol. 19 Franklin, N., Tversky , B., and Coon , V. ( 1992 ). Switchingpoints of view in spatialmental - 518 models . MemoryandCognition . , 20(5), 507 Gee andthehumanbiological , J., andGoodhart , W. ( 1988 ). American SignLanguage capacity for language . In M. Strong(Ed.), Language anddeafness , 49- 74 , New York: Cambridge learning . UniversityPress

BeUugi, U ., Poizner, H ., and Klima , ES . ( 1989 ). Language , modality, and the brain. Trendsin

The Confluenceof Spaceand Languagein Signed Languages

207

Herskovits : An interdisciplinary andspatialcognition , A. ( 1986 ). Language studyof theprepositions in English . Cambridge : Cambridge . UniversityPress : Mental rotation and choice nan, A. B., and Miller, J. ( 1994 ). A violation of pure insertion reaction time. Journalof Experimental : Human andPerformance . 20(3), Psychology Perception - 536 520 . on ASL verbagreement . In K. Emmorey Janis andJ. , W. ( 1995 ). A cross linguisticperspective - 224 .), Language . Hillsdale . , gesture , andspace , 195 , NJ: Erlbaum Reilly(Eds Klima, E. S., and Bellugi . Cambridge , U. ( 1979 , MA: HarvardUniversity ). Thesigns of language Press . in , K . R., andWallach , S. M., Brunn , J. L., Cave , R. W. ( 1985 ). Individualdifferences Kosslyn - 243 mentalimagery . Cognition . , 18 , 195 ability: A computational analysis esin image , S., Cave , K., ProvostD ., and Von Gierke , S. ( 1988 Kosslyn ). Sequential process . 20 319 343 . , , generationCognitive Psychology " " " " Landau and spatial , B., and Jackendoff , R. ( 1993 ). What and where in spatiallanguage . Behavioral andBrainSciences . , 16 , 217 238 cognition Levelt in the useof spatialdirectionterms . In R. Jarvella andW. , W. ( 1982a ). Cognitive styles - 268 .), Speech Klein (Eds . NewYork: Wiley. , place , andaction , 251 Levelt in describing . In S. Peters and E. saarinen , W. ( 1982b ). Linearization spatialnetworks - 220 .), Process es . Dordrecht : Reidel . , beliefs , andquestions , 199 (Eds Levelt . In A. J. vanDoom, W. limitationson talkingaboutspace , W. ( 1984 ). Some perceptual - 358 A. van de Grind, and J. J. Koenderink(Eds . Utrecht .), Limits in perception : VNU , 323 Science Press . -part tenninology Levinson : Tzeltalbody , S. ( 1992a , shape , and linguisticdescription ). Vision and object descriptions . Working paperno. 12 , CognitiveAnthropologyResearch Group, Max PlanckInstitutefor Psycholinguistics . , Nijmegen Levinson andcognition : Thecognitive of spatialdescription , S. ( 1992b ). Language consequences in GuuguYimithirr. Working paperno. 13 , CognitiveAnthropologyResearch Group, Max PlanckInstitutefor Psycholinguistics . , Nijmegen Liddell, S. ( 1990 : Reexamining ofa locus thestructure of space in ASL. In C. ). Four functions - 198 Lucas(Ed.), Signlanguage research : Theoretical issues . Washington , 176 , DC: Gallaudet Press . College Liddell andlinguisticissues in spatialmapping : Comparing and , S. ( 1993 ). Conceptual spoken . Paper Workshop at thePhonology andMorphologyof SignLanguage signed languages presented . , August , Amsterdam . In I. Ahlgren .), Liddell, S. ( 1994 , B. Bergman , and M. Brennan ). Tokensandsurrogates (Eds onsignlanguage . Durham structure , UK : ISLA. Perspectives Liddell, S. ( 1995 : Grammatical in ASL. In K. , surrogate , andtokenspace ). Real consequences - 42. Hillsdale andJ. Reilly (Eds .), Language . , gesture , andspace , 19 , NJ: Erlbaum Emmorey

208
Lillo - MartinD . ( 1991 ) . Universal : Kluwer ) . The ( Eds Klima grammar . of view , gesture ) . Pointing in American and American sign language

Karen Emmorey
: Setting the nullargumentparameters

. Dordrecht Lillo - MartinD . ( 1995 J . Reilly . , and

point

predicate , andspace out

Emmoreyand Lillo - MartinD

. ) , Language , E . ( 1990 P . Siple of

Sign - 170 . Hillsdale , 155 : ASL in sign

Language , NJ

In

K . .

: Erlbaum in

differences issues

pronouns language

syntactic , vol . I ,

. In S . D . Fischer and theory 191 - 210 . Chicago : University Loew , R . ( 1983 . PhiD McIntire , Los Meier Newell of the . diss ) . Roles . , University ) . Locatives and

( Eds

. ) , Theoretical Press .

research

Chicago in

reference of Minnesota in

American .

Sign

Language

: A

developmental

perspective

, M . ( 1980 Angeles .

American

Sign

Language

. PhiD

. diss

. , University

of

California

, R . ( 1991 , W . ( Ed Deaf .

) . Language . ) ( 1983

acquisition sign

by

deaf

children . Silver

. American

Scientist , MD

- 70 . , 79 , 60 Association

) . Basic

communication

Spring

: National

Newport Siobin Hillsdale Padden Fourth MD

, E . , and ( Ed .) , The

Meier Cross .

, R . ( 1985 linguistic

) . The study of

acquisition language

of

American . Vol

Sign . I ,

Language The data ,

. In 881

D . I . - 938 .

acquisition

, NJ

: Erlbaum

, C . ( 1986 National

) . Verbs Symposium

and on of

role Sign

shifting Language .

in

ASL

. In

C . Padden and Teaching

( Eds

Research

. ) , Proceedings - 57 . Silver , 44

of Spring

the ,

: National

Association ) . / nteraction

the

Deaf

Padden

, C . ( 1988 in Linguistics .

of

morphology York

and : Garland

syntax . 1983

in

ASL

. Garland . diss

Outstanding of

Dissertations California ,

, ser . 4 . New

PhiD

. , University

San

Diego

Padden Lucas University Poimer from Poimer Cambridge Poulin Language Hillsdale Roth

, C . ( 1990 ( Ed . ) , Sign Press , H . , and American H ., .

) . The language

relation research

between

space

and

grammar

in

ASL

verb

morphology , DC

. In : Gallaudet

C .

: Theoretical

issues

- 132 . , 118 Washington

Kegl Sign

, J . ( 1992 Language

) . Neural . Aphasiology Bellugi ,

basis

of

, 6 ( 3 ) , 219 U . ( 1987 ).

language - 256 What

and . the

motor

behavior

: Perspectives

Klima

E . S . , and Press .

hands

reveal

about

the

brain

, MA , C . , and . In

: MIT Miller K .

, C . ( 1994 and

) . On J .

narrative Reilly ( Eds

discourse .) , Language

and

point , gesture

of ,

view and

in space

Quebec ,

Emmorey . , S . M , 20 , 344 ) . Phonological . Dordrecht ) . Spatial

Sign 117 - 132 .

, NJ , J . , and

: Erlbaum

Kosslyn

Cognitive Sandier Sign Schober

Psychology , W . ( 1989 Language , M . ( 1993

. ( 1988 ) . Construction - 361 .

of

the

third

dimension

in

mental

imagery

representation : Foris .

of

the

sign

: Linearity

and

nonlinearity

in

American

perspective

taking

in

conversation

. Cognition

, 47 , 1 24 .

The Conftuenceof Spaceand Languagein Signed Languages

209

-dimensional . Science , objects , J. ( 1971 , R., and Metzler ). Mental rotation of three Shepard 703 . 171 701 , -Kegi, J. ( 1985 word formation , ). Locative relationsin AmericanSign Language Shepard . of Institute . Massachusetts . PhiD. diss and discourse , Technology , syntax : in spoken and signed to movement Slobin languages , D., and Hoiting, N. ( 1994 ). Reference AnnualMeetingof the Berkeley . Proceedings of the Nineteenth Typological considerations . . Berkeley , CA: Berkeley Society , 1- 19 Linguistics Society Linguistic in the serviceof a task. In Proceedings St. John of the , M. F. ( 1992 ). Learninglanguage . Erlbaum Hillsdale NJ: . Science the AnnualConference Fourteenth , Society Cognitive of development in.signed : The coded 1991 . S. language modalityquestion , ( English ) Manually Supalla research in signlanguage .), Theoretical issues . In P. SipleandS. D. Fischer , vol. 2, (Eds - 109 Press . : Universityof Chicago . Chicago 85 of verbsof motionandlocationin American andacquisition Sign , T. ( 1982 ). Structure Supalla . of California . . Ph.D. diss , SanDiego , University Language in a chair?Thederivationof nounsand , E. ( 1978 ). How manyseats , T., and Newport Supalla . In P. Siple(Ed.), Understanding verbsin AmericanSign Language throughsign language . Press . NewYork: Academic research , 91- 132 language skill and Talbot, K . F., and Haude sign language , R. H. ( 1993 ). The relationshipbetween and . Perceptual -dimensional objects spatial visualizationability: Mental rotation of three . 77 1387 1391 3 Motor Skills , ( ), .), Spatial . In H. Pick and L. Acredolo(Eds structures L. 1983 , ( space ). How language Talmy Press . . NewYork: Plenum : Theory orientation , andapplication , research in - Ostyn . In B. Rudzka to cognition (Ed.), Topics , L. ( 1988 ). The relationof grammar Talmy - 207 . : Benjamins . Amsterdam , 165 cognitive linguistics , B. ( 1992 ). Spatialmentalmodelsderivedfrom surveyand route Taylor, H., and Tversky - 292 . and . Journalof Memory Language , 31, 261 descriptions . Boston : Little, dimensions andapplied : Linguistic Wilbur, R. ( 1987 SignLanguage ). American . Brown andJ. frames . In K . Emmorey discourse in comparative Winston , E. ( 1995 ). Spatialmapping . . Hinsdale .), Language , NJ: Erlbaum , andspace , gesture , 87- 114 Reilly (Eds

Chapter 6 Fictive Motion in Language and " Ception "

Leonard Talmy

6.1

Introduction

This chapter proposesa unified account of the extensivecognitive representation of - especially forms of motion - both as they are expressed nonveridical phenomena linguistically and as they are perceivedvisually. Thus, to give an immediate senseof the matter, the framework posited here will cover linguistic instances that depict motion with no physical occurrence : Thisfence goes , for example from the plateau to the valley; The cliff wall faces toward/away from the island; I looked out past the ,. The vacuumcleaneris downaroundbehindthe clotheshamper ; and Thescenery steeple rushedpast us as we drovealong. In a similar way, our framework will also cover visual instances in which one : the perceived" apparent , for example perceivesmotion with no physical occurrence " motion in successive flashesalong a row of lightbulbs, as on a marquee ; the perceived " induced motion " of a rod when only a surrounding frame is moved; the es like indentation perception of a curved line as a straight line that has undergoneprocess and protrusion ; the possibleperception of an obliquely oriented rectangle(e.g., a picture frame) as having been tilted from a vertical-horizontal orientation ; and the " " of a vertical stroke possible perception of a plus figure as involving the sequence followed by a horizontal stroke. 6.1.1 OveraUFramework Our unified account of the cognitive representationof nonveridical phenomena , just " " exemplified, is a particular manifestation of the overlapping systems model of cognitive organization . This model seespartial similarities and differences across distinct cognitive systemsin the way they structure perceptual, conceptual, or other . We will mainly consider similarities between two such cognitive representations : languageand visual perception. cognitive systems

212

Leonard Talmy

The particular manifestation of overlap we address involves a major cognitive , this pattern : a discrepancy within the cognition of a single individual . Specifically discrepancy is between two different cognitive representationsof the same entity, as being more veridical than the other. where one of the representationsis assessed We presumethat the two representationsare the products of two different cognitive itself is produced by a third cognitive , and that the veridicality assessment subsystems . subsystemwhosegeneral function it is to generatesuch assessments In the notion of discrepancy we intend here, the two cognitive representations consist of different contents that could not both concordantly hold for their represented object at the sametime- that is, they would be inconsistent or contradictory , as judged by the individual ' s cognitive systemsfor general knowledge or reasoning . On the other hand, the individual need not have any active experienceof conflict or clash betweenthe two maintained representations , but might rather experiencethem as alternative perspectives . Further , in saying that the two discrepant representations differ in their assessed degreeof veridicality , we usethe lesscommon term veridicalrather than, say, a term like true- to signal that the ascription is an assessment , with no appeal to some notion of absolute or external produced by a cognitive system . reality Of the two discrepant representationsof the sameobject, we will characterizethe to be more veridical as " factive" and the representation assessed representation assessed to be less veridical as " fictive." Adapted from its use in linguistics, the term of greater veridicality, factive is hereagain intended to indicate a cognitive assessment but not to suggest(as perhaps the word factual would ) that a representation is in somesense objectively real. And the termfictive has beenadopted for its referenceto the imaginal capacity of cognition, not to suggest(as perhaps the word fictitious would ) that a representationis somehowobjectively unreal. As a whole, this cognitive pattern of veridically unequal discrepant representationsof the sameobject will here be called the pattern of " general fictivity ." In the general fictivity pattern, the two discrepant representations frequently- disagreewith respectto somesingle dimension, representing though not exclusively opposite poles of the dimension. Several different dimensions of this sort can be . One example of such a dimension is state of occurrence observed . Here, factive of someentity in the more veridical representation (the presence presence ) is coupled with fictive absence(the absenceof that entity from the lessveridical representation ) or vice versa. Another example of a dimension is state of change . Here, the more veridical representation of an object could include factive stasis , while the less - or vice versa. One form of this last veridical representationincludes fictive change -time is the more specific dimension when applied to a physical complex in space dimension state of motion. Here, the more veridical representation could include

" and" Ception Motionin Language Fictive

213

stationariness , while the less veridical representation has motion - or vice versa. Thus, frequently in conjunction with their factive opposites , we can expect to find cases of fictive presence , fictive stationari, fictive change , fictive stasis , fictive absence ness , and fictive motion . In fact, to a large extent, general fictivity can accommodate " " any fictive X. Of thesetypes, the present chapter focuseson fictive motion , usually in combination . It will be seenthat such fictive motion occurs preponderantly with factive stationariness more than doesfictive stationarinesscoupled with factive motion . As will be discussed , this fact reflectsa cognitive bias toward dynamism. The general fictivity pattern can be found in a perhaps parallel fashion in both , the pattern is extensively exhibited in the case language and vision. In language where one of the discrepant representationsis the belief held by the speakeror hearer about the real nature of the referent of a sentence , and the other representationis the . Here the literal literal referenceof the linguistic forms that make up the sentence as less veridical than the representation based on belief. representation is assessed the literal representation is fictive, while the representation based on Accordingly , belief is factive. Given our focus on the pattern in which fictive motion is coupled , we here mainly treat the linguistic pattern in generally with factive stationariness which the literal meaning of a sentenceascribes motion to a referent one would otherwise believeto be stationary. In vision, one main form of the generalfictivity pattern is the casewhere one of the discrepant representationsis the concrete or fully palpable percept an individual has of a scene on viewing it , and the other is a particular , less palpable percept the individual has of the same sceneconcurrently. Here the less palpable percept is assessed . Parallel to the linguistic case as the less veridical of the two representations , and the visual more to the be the term factive may , representation palpable applied " " . We will say that an individual sees termfict ;ve to the lesspalpable representation " " the factive representation , but only senses the fictive representation(when it occurs later) . Here, too , we focus on at a particular lower level of palpability , to be discussed visual less where the fictive motion , representation is of motion , while the palpable . fully palpable representationis generally of stationariness To accommodate this account of visual representations that differ with respect to their palpability , we posit the presencein cognition of a gradient parameter of palpability . Moreover, one may identify a number of additional cognitive parameters " that largely tend to correlate with the palpability parameter. All of these palpabilityrelatedparamet " are characterizedbelow in section 6.9.1. Further these , parameters than that domain a to extend larger cognitive continuously through appear of in the combination that fact covers alone one with associated , perception generally and of domains with what is usually associateddifferentially perception separate

214

Leonard Talmy

conception. Accordingly , to accommodatethe full range of each such parameter , we advancethe idea of a single continuous cognitive domain, which we call " ception." In the presentchapter we largely restrict our study of general fictivity in language to the casewhere both of the two discrepant representationsare of a physical complex -time. In this way, there is generally the potential for any linguistic in space -domain example to have an analogue in a visual format . Accordingly, in a cross of this sort we could to , correspondence expect find two component parallels. One parallel would hold between the two factive representations ; the other between the two fictive representations . In particular , one parallel would hold betweenthe linguistic believedto be veridical and the concrete representationof a sentence , fully palpable of the appearance corresponding visual display. The other parallel would then hold between the less veridical literal referenceof the sentenceand a less palpable associatedimage perceivedon viewing the display. If we view this correspondence starting from the languageend, a linguistic example -time of general fictivity whose representationspertain to physical entities in space in can, effect , be mappedonto a visual exampleof generalfictivity . In sucha mapping, the linguistic referential difference betweencredenceand literality is then translated in the visual domain into a difference in palpability . Experimental methods are needed to determine whether the parallel between the two fictive representations holds. In fact, one aim for the presentchapter is to serveas a guide and as a call for such experimental research . The restriction of the present study to the representation of physical forms in -time excludestreatment of nonspatial metaphor. For example space , a metaphor like Her mood wentfrom good to bad would be excluded ; although its source domain is -time, its target domain is the nonphysical one of mood states motion in space . However , as discussedlater, linguistic metaphor as a whole fits as a category within the framework of generalfictivity . General fictivity can serveas the superordinateframework because , among other reasons , its conceptsand terms can apply as readily to visual representationsas to linguistic ones , whereasmetaphor theory is cast in concepts and terms more suitable for languagealone. Using the perspectiveand methods of cognitive linguistics, the present study of fictive motion is basedin language , but extendsout from there to considerationsof visual perception.

6.1.2 FictiveMotion in Language Fictive motion in language es a numberof relativelydistinct categories encompass
(first set forth in Talmy 1990 ) . These categories include emanation, pattern paths, frame-relative motion , advent paths (including site manifestation and site arrival ), accesspaths, and coverage paths. This last category, perhaps the type of fictive

" " Fictive Motion in Languageand Ception

215

motion most familiar in the previous linguistic literature , was called " virtual motion " " " " " in Talmy ( 1983 ), abstract motion in Langacker ), extension in Jackendoff ( 1983 " " ( 1987 ), and subjective motion in Matsumoto. Our current tenD coverage paths is used as part of the more comprehensivetaxonomy of fictive motion presentedhere. . Illustrating coveragepaths can serveas an orientation to fictive motion in general Modesto This category is most often demonstratedby fonDSlike This road goes from to Fresnoor The cord runsfrom the TV to the wall. But a purer demonstration of this type of fictive motion would exclude referenceto an entity that supports the actual motion of other objects (as a road guides vehicles ) or that itself may be associated with a history of actual motion (like a TV cord) . The " mountain range" example in ( I ) avoids this problem. ( 1) a. That mountain range lies betweenCanada and Mexico. b. That mountain range goesfrom Canada to Mexico. c. That mountain range goesfrom Mexico to Canada. Here ( 1a) directly express es the more veridical static spatial relationships in a stative fonD of expression without , evoking fictive motion . But ( 1b) and ( lc ) representthe static linear entity, the mountain range, in a way that evokesa sense or aconceptualization of something in motion respectively , from north to south and from south to north . These latter two sentences manifest the general fictivity pattern. They each involve two discrepant representationsof the same object, the mountain range. Of - that is, the one that is assessed thesetwo representations , the fictive representation and experienced as lessveridical consistsof the literal referenceof the words, which , the one directly depict the mountain range as moving. The factive representation assessed and experiencedas more veridical, consistsof our belief that the mountain range is stationary. This factive representation is the only representation present in sentence ( la ), which accordingly does not manifest the generalfictivity pattern . Most observerscan agree that languagessystematically and extensively refer to stationary circumstanceswith fonDS and constructions whose basic referenceis to motion . We can tenD this constructionalfictive motion. Speakersexhibit differences , however, over the degreeto which such expressions evoke an actual senseor concep tualization of motion - what can be tenDed experienced fictive motion. Thus, for the same instance of constructional fictive motion , some speakerswill report a strong semantic evocation of motion , while other speakerswill report that there is none at all. What does appear common, though, is that every speakerexperiences a senseof motion for somefictive motion constructions. Where an experienceof motion does occur, there appears an additional range of differences in what is conceptualized as moving . This conceptualization can vary

216

Leonard Talmy

acrossindividuals and types of fictive motion ; eventhe sameindividual may deal with . Included in the the sameexample of fictive motion differently on different occasions manifested be by the named conceptualizationsof this range, the fictive motion may , by the mountain range in ( I ); by some unnamed object that entity, for example moves with respect to the named entity, for example , a car or hiker relative to the mountain range; in the mental imagery of the speakeror hearer, by the imagistic or conceptual equivalent of their focus of attention moving relative to the named entity ; of motion moving relative to the named entity by some abstracted conceptual essence ; or by a senseof abstract directednesssuggestingmotion relative to the named entity . The strength and character of experiencedfictive motion , as well as its clarity and homogeneity , are a phenomenologicalconcomitant of the presentstudy that will needmore investigation. The severaldistinct categoriesof fictive motion indicated above differ from each . Each category of fictive other with respect to a certain set of conceptual features motion exhibits a different combination of values for these features , of which the main ones are shown in (2) . (2) Principalfeatures distinguishingcategoriesof fictive motion in language I . Factive motion of someelementsneednot / must be presentfor the fictive effect; 2. The fictively moving entity is itself factive/ fictive; - and, if observer -based -based -neutral/ observer 3. The fictive effect is observer , scans the observer is factive/ fictive and moves ; / 4. What is conceivedas fictively moving is an entity/ the observation of an entity . Out of the range of fictive motion categories , this chapter selectsfor closestexamination . which of emanation the category , appearsto have been largely unrecognized in The other indicated categories of fictive motion will be more briefly discussed section 6.8.1 6.1.3 Propertiesoftbe EmanationType as a Whole Amid the range of fictive motion categories , emanationis basically the fictive motion . In most subtypes of something intangible emerging from a source , the intangible on somedistal terminates and its emanation continues by impinging path along entity exhibited by 2 that are of fictive features of the values . The ( ) general object particular the in . the emanation category are listed , intangible entity is what (3) Specifically moves fictively and is itself fictive, and its fictive motion does not depend on any factive motion by some tangible entity nor on any localized observer.

" Fictive Motion in Languageand " Ception

217

(3) Thefeature values for emanationpaths in language I . Factive motion of someelementsneednot be present for the fictive effect; 2 . The fictively moving entity is itself fictive; -neutral; 3. The fictive effect is observer 4. What is conceivedas fictively moving is an entity . The category of emanation comprises a number of relatively distinct types. We presentfour of theseemanation typesin sections6.2- 6.5: orientation paths, radiation paths, shadow paths, and sensory paths. The illustrations throughout will be from English only in the present version of this chapter, but examples from other languages can be readily cited. The demonstrations of at least constructional fictive motion will rely on linguistic forms with basically real-motion referentssuch as verbs like throw and prepositions like into and toward. In the exposition, wherever some form of linguistic conceptualization is posited, we will raise the possibility of a corresponding perceptual configuration . Then, in section 6.7, we will specifically suggest . perceptual analoguesto the emanation types that have beendiscussed
6.2 Orientation Paths

The first type of emanation we consider is that of orientation paths. The linguistic conceptualization- and possibly a corresponding visual perception- of an orientation path is of a continuous linear intangible entity emerging from the front of some object and moving steadily away from it . This entity may be conceivedor perceived as a moving intangible line or shaft- the only characterization used below. Alternatively , though, the entity might be conceivedor perceivedas some intangible abstraction moving along a stationary line or shaft- itself equally intangible- that is already in place and joined at one end to the front of the object. In addition to fictive motion along the axis of such a line, in somecasesthe line can also be conceptualized or perceivedas moving laterally . In this characterization, the " front " of an object is itself a linguistic conceptualiza tion or perceptual ascription based on either a particular kind of asymmetry in the ' ' object s physical configuration; or on the object s motion along a path, where the 2 leading side would generally constitute the front . In the main casesrelevant here, " such a front can be either a planar or " face -type front , consisting of an approximately planar surface on a volumetric object, or a point -type front , consisting of an endpoint of a linearly shapedobject. Presentednext are five subtypes of orientation paths that variously differ with respectto severalfactors, including whether the front is a face-type or a point -type, and whether the fictive motion of the intangible line is axial or lateral. First , though,

218

Leonard Talmy

we note the occurrenceof constructions that are sensitiveto the fictive presence of an line with the front of an before we to its fictive intangible aligned object, proceed motion . Consider the sentences in (4) : (4) a. Shecrossedin front of me/ the TV . b. Shecrossed?behind/ * besideme/ the TV . The sentences here show that the verb cross can felicitously be used when walking transverselyin front of an object with a front , but only poorly when walking behind, and not at all when walking to one side.3 This usagepattern seems to suggestthere is linear to walk across in front of an something present directly object, but not elsewhere with respectto that object. We would argue that what is thus being crossedis the posited intangible line conceivedto emergefrom the front of an object, that will next be seento exhibit fictive motion in a further set of construction types. 6.2.1 ProspectPaths The first type of orientation path that we exarninecan be termed a prospectpath. The orientation that an object with a face-type front has relative to its surroundings can - in terms of fictive rnotion. be conceptualizedlinguistically - and perhapsperceived " " " " With its front face, the object has a particular prospect," " exposure , or vista relative to sorneother object in the surroundings. This prospect is characterizedas if sorneintangible line or shaft ernergesfrorn the front and rnovescontinuously away frorn the rnain object relative to the other object. The linguistic constructions, in effect, treat this line as Figure rnoving relative to the other object as Ground or Reference ' , 1983 Object (in Talrny s [ 1987b ] terms) along a path indicated by directional . In English, suchconstructionsgenerallyemploy verbslike/ aceor look out. adpositions -type front . The cliff ' s In the exarnplein (5), the vertical side of a cliff acts as its face prospect upon its surroundings is characterizedin terms of a fictive course of rnotion ernergingfrorn its face and rnoving along the path specifiedby the preposition relative to a valley as ReferenceObject. Again , this exarnple rnanifests the general fictivity of its words depicts a fictive, lessveridical representationin pattern. The literal sense which sornething rnovesfrorn the cliff wall along a path that is oriented with respect to the valley. But this representation is discrepant with the factive, rnore veridical representation consisting of our belief that all the referent entities in the sceneare static and involve no rnotion. (5) The cliff wall facestoward /away frorn / into /past the valley. 6.2.2 Alignment Paths The alignment path type of orientation involves a stationary straight linear object with a point -type front . The orientation of such a linear object is here conceptualized

Fictive Motion in Languageand " Ception"

219

- in terms of something intangible moving linguistically - and perhaps perceived along the axis of the object, emerging from its front end, and continuing straight , along a prepositionally determined path relative to somedistal object. As it happens the English constructions that evoke this arrangement are not free to representjust where the linear object is aligned with any orientation , but are limited to the two cases the distal object- the front being the end either closer to or further from the distal in (6) illustrate this type.4 object, the sentences (6) The snake is lying toward /away from the light . Here the snake is the linear object with its head as the point -type front , and the light is the distal object. Of note, this construction combines a verb of stationariness , lie, with a path preposition, toward or awayfrom , that coercesthe verb' s semanticproperties . A sentence with lie alone would permit an interpretation of the snakeas coiled and, say, pointing only its head at or away from a light . But in the normal understanding of (6), the snakesbodyforms an approximately straight line that is aligned with the light . That is, the addition of a path preposition in this construction has the effect of forcing a fictive alignment path interpretation that requires a straight-line ' contouring of the snake s body. The hypothesis that fictive orientation paths emerge ' from an object s front and move away from the object correctly accountsfor the fact that the sentence with " toward " refers to the head end of the snake as the end closer to the light , while the sentencewith " away from " indicates that the head end is the further end. 6.2.3 DemormtrativePaths The demonstrativetype of orientation path also involves a linear object with a point . But here the fictively moving line type front from which an intangible line emerges ' s attention functions to direct or guide someone along its path . The particular orientation of the linear object can either be an independent factor that simply occasions ' s attention or can be an instance of directing someone , intentionally set to servethe ' purpose of attentional guidance. This function of directing a person s attention can be the intended end result of a situation. Or it can be a precursor event that is instantiated or followed by another event, such as the person' s directing his or her , or moving bodily along the fictive path . gaze Thus, in the examplesin (7), a linear object with a front end, such as an arrow or an extendedindex finger, seems to emit an intangible line from its front end. This line ' s attention movesin the direction of the object' s orientation so as to direct someone , or motion the the . , physical gaze along path specifiedby preposition (7) a. lIThe arrow on the signpost pointed toward /away from / into /past the town. bIpointed / directed him toward/past/away from the lobby .

220

Leonard Talm }'

6.2.4 Targeting Paths In a targeting path, an Agent intentionally sets the orientation of a front -bearing object so that the fictive line that is conceptualizedor perceivedas emergingfrom this ' front follows a desired path relative to the object s surroundings. This fictive motion esa path along which the Agent further intends that a particular subsequent establish motion will travel. This subsequentmotion either is real or is itself fictive. Although of intentions and actions, with comparatively complex, something like this sequence ~ single or double fictive path, seemsto underlie our conceptsof aiming, sighting, or in (8) in this regard. targeting. Consider the sentences ) into /past/away from the living room. (8) I pointed/ aimed (my gun/ camera Here the caseof a bullet shot from the aimed gun exemplifiesreal motion following the preset fictive path . In contrast, the camera provides an instanceof fictive motion following the fictive path, with a so-conceivedphotographic probe emergingfrom the ' s front . camera One might ask why the camera example is included here under the targeting type " of orientation path, rather than below under sensory paths along with " looking . The reason is that the act of looking is normally treated differently in English from " " the act of photographic shooting. We normally do not speak of " aiming or pointing " our , and we do not conceive of the act of looking as involving first the gaze establishmentof a targeting path and then a viewing along that path . 6.2.5 Line of Sight Line of sight is a concept that underlies a number of linguistic patterns, and perhaps also a component of perceptual structure. It is an intangible line emerging from the visual apparatus canonically located on the front of an animate or mechanicalentity . The presentdiscussiondealsonly with lateral motion of the line of sight, that is, with shifts in its orientation . Axial fictive motion along the line of sight will be treated in section6.5 on sensorypaths. Additional evidencefor treating the shifting line of sight as an orientation path is that the sentences exhibiting this phenomenoncan use not verbs like look but also nonsensoryverbs like turn~ just sensory In the examplesin (9), the object with the vision-equippedfront - whether my head with its eyesor the camera with its lens- swivels , thus causing the lateral motion of the line of sight that emergesfrom that front . The path preposition specifiesthe particular path that the line of sight follows. Consider how fictive motion is at work in the caseof a sentencelike I slowly turned / looked toward the door. A path preposition ' like toward normally refers to a Figure object s executinga path in the direction of the ReferenceObject, where the distance between the two objects progressively decreases . But what within the situation depicted by the example sentencecould be

" Fictive Motion in Languageand " Ception

221

? The only object that is physically moving is my turning exhibiting thesecharacteristics head, yet that object stays in the samelocation relative to the door , not moving closer to it . Apparently what the preposition toward in this sentencerefers to is the . As I turn my head in the motion of the line of sight that emergesfrom my eyes this line of sight does indeed direction , appropriate clockwise or counterclockwise follow a path in the direction of the door and shorten its distance from it . (9) I slowly turned/ looked- III slowly turned my cameratoward the door ./ around the room./away from the window.1 from the painting, past the pillar , to the tapestry. We can note that English allows each linguistic form in a successionof path indications to specify a different type of fictive motion . Thus, in ( 10 ), the first pathof a line of sight, of the specifying form , the satellite down, indicates a lateral motion , the likely interpretation is that type discussedin this section. Under its specification " " I am horizontal of is line ( looking straight ahead ), and then sight initially my swivelsdownward so as to align with the axis of a well. The secondspatial form , the preposition into, indicates that once my line of sight is oriented at a downward angle, then the fictive motion of my vision proceedsaway from me axially along the line of sight, thus entering the well. ( 10 ) I quickly looked down into the well. 6.3 Radiation Paths The second type of emanation we consider is that of radiation paths. The linguistic conceptualization of a radiation path is of radiation emanating continuously from an energy source and moving steadily away from it . This radiation can additionally be understood to comprise a linear shaft and to subsequently impinge on a second object. This additional particularization is the only type treated here. In this type, then, the radiating event can be characterizedas involving three entities: the radiator , the radiation itself, and the irradiated object. And this radiating event then involves es: the (generation and) emanation of radiation from the radiator , the three process motion of the radiation along a path, and the impingement of the radiation upon the irradiated object. A radiation path differs from an orientation path in that the latter consistsof the motion of a wholly imperceptible line. In a radiation path, though, one can often indeed detect the presenceof the radiation - for example , in the case of - and, hence detect . What one cannot see the one can radiation , directly , light light . this radiation motion of is what remains imperceptible any The sentencesin ( 11 ) reflect the preceding characterization of radiation for the particular caseof light in the way they are linguistically constructed. This linguistic

222

Leonard Talmy

construction mainly involves the choices of subject, of path-specifying preposition, and of prepositional object. In both sentences , then, the generalunderstanding is that the visible light is a radiation ; that the sun is the source of the light (perhaps its generator, but at least its locus of origination ); that the light emanatesfrom the sun and moves steadily as a beam along a straight path through space ; and that the light movesinto the cave or impinges on its back wall to illuminate that spot. / onto the back wall of the cave. ( II ) a. The sun is shining into the cave b. The light is shining (from the sun) into the cave / onto the back wall of the cave. Now , as compelling as this characterization of light radiation may be felt to be, it is, in the end, purely a conceptualization. Although physicists may tell us that photons in fact move from the sun to the irradiated object, we certainly cannot actually . Therefore, any correspondencebetweenthe scientific charseeany such occurrence acterization and the conceptualization of the phenomenonmust be merely coincidental . In other words, the so-conceivedmotion of radiation from the radiator to the irradiated must be fictive motion . Becausedirect sight does not bring a report of ' light s motion , it must be other factors that lead to a conceptualization in terms of motion away from the sun, and we will speculateon those factors in section 6.6. At this point , however, the task is to suggesta number of viable alternatives to the normal conceptualization. Thesealternativesshow that the unique appearanceof this conceptualization cannot be explained by virtue of its being the only conceptualiza . tion possible One alternative conceptualization is that there is a radiation path, but that it moves in the reversedirection from that in the prevailing conceptualization. Imagine the following state of affairs. All matter contains or generatesenergy. The sun (or a comparable entity) attracts this energy. The sun draws this energy toward itself when there is a straight clear path between itself and the matter. Matter glows when its energy leavesit . The sun glows when energyarrives at it . An account of this sort is in principle as viable as the usual account. In fact, it is necessarilyso, becauseany phenomenon that could be explained in terms of imperceptible motion from A to B must also be amenableto an explanation in terms of a complementary imperceptible motion from B to A . However, for all its equality of applicability , the fact is that this -direction scenario is absent from - even resistedby- our normal conceptual reverse apparatus. And it is certainly absent from extant linguistic constructions. Thus ), and we suspect that any counterpart English lacks any sentencelike that in ( 12 formulation is universally absent from the languagesof the world . ) * The light is shining from my hand onto the sun. ( 12

" Fictive Motion in Languageand " Ception

223

The conceptualization that an object like the sun, a fire, or a flashlight produces light that radiates from it to another object is so intuitively compelling that it can be -direction conceptualization in of value to demonstrate the viability of the reverse . Consider, for example, a vertical pole and its shadow on the different circumstances . The sun as Source conceptualization here has the pole as blocking the light ground that would otherwise proceedfrom the sun onto the ground directly behind the pole. -direction conceptualization works here as well. The sun attracts But the reverse energy from the side of the pole facing it , but cannot do so from the portion of the ground directly behind the pole becausethere is no straight clear path betweenthat portion of the ground and the sun- the pole blocks the transit of energy in the no energyis drawn out of the portion of the ground behind reversedirection. Because the pole, it fails to glow, whereasthe potions of ground adjacent to it , from which energy is being directly drawn, do glow . Or consider a fire. Here one can seethat the surfacesof oneself facing the fire are brighter than the other surfacesand, in addition , one can feel that they are warmer as well. Further , this effect is stronger the closer one is to the fire. Once again, the fireas-Sourceof both light and heat is not the only possibleconceptualization. The same -direction conceptualization used for the sun holds as well for the fire. The reverse ' additions in this exampleare that when the fire attracts energyfrom the parts of one s body facing it , the departure of that energy causesnot only a glow but also the sensationof warmth. (Such warmth is of course also the casefor the sun, but more ) . And the one saliently associatedwith fire, hence saved for the present example ' further factor here is that the attraction that the fire exerts on an object such as one s body is stronger the closer it is. -direction conceptualization is not the only feasible alternative to the The reverse prevailing conceptualization of a radiation path, itself a constellation of factors, any -direction alternative attempted to invert . The reverse one of which can be challenged the directionality of the fictive motion in the prevailing conceptualization. But we can also test out the factor which holds that a radiation path originates at one of the salient physical objects and terminates at the other. Thus we can check the viability of a conceptualization in which light originates at a point between the two salient objects and fictively moves out in opposite directions to impinge on each of those two objects. ( 13 ) tries to capture this conceptualization. However, this sentence es seemswholly does not work linguistically and the conceptualization it express . counterintuitive * ( 13 ) The light shone out onto the sun and my hand from a point betweenus. Another assumptionin the normal conceptualization we can try to challengeis that the radiation movesat all. Perhapsthe radiation does not exhibit fictive motion at all

224

Leonard Talmy

but rather rests in space as a stationary beam . But sentences like ( 14) show that this

conceptualization, too , has neither linguistic nor intuitive viability . ) * The light hung betweenthe sun and my hand. ( 14
6.4 Shadow Paths

. The linguistic The third type of emanationwe consideris that of shadow paths is that the of a shadow also a and path perception perhaps conceptualization hasfictivelymovedfrom that object shadow of someobjectvisibleon somesurface . Sentences like thosein ( 15 to that surface aconceptual ) showthat Englishsuggests . Thusthese sentences setup the izationof this sort throughits linguisticconstruction it is asthe shadow asthe Figure nominalthat refersto the shadow ; the objectwhose is locatedasthe Groundobject on which the shadow Source , herefunctioning ; the surface cast or asa motionverblike throw asGoal; thepredicate ; anda path , , project . suchasinto, onto , across , or against preposition the valley . downinto/ across ( 15 ) a. The treethrewits shadow the wall. a shadow onto b. The pillar cast /against /projected could conceivably be made We can note that with radiationpaths , the argument from the sun to my hand that the directionof the fictive motion proceeds , because a weakargument . But however tenable that is the directionthat photonsactuallytravel in the case of shadow not be used could like this may be, eventhis argument of " sha . For thereis no theory of particle physicsthat positsthe existence paths " . of its shadow dowons that movefrom an objectto the silhouette
6.5 SelB) ry Paths

One category of emanation paths well representedin language is that of sensory paths, including visualpaths. This type of fictive motion involves the conceptualiza and of the the and tion of two entities, , somethingintangible Experienced Experiencer moving in a straight path betweenthe two entities in one direction or the other. By one branch of this conceptualization, the Experienceremits a Probe that movesfrom the Experiencerto the Experiencedand detectsit upon encounter with it . This is the -as-Source type of sensorypath . By the other branch of the conceptualExperiencer ization , the experiencedemits a Stimulus that moves from the Experienced to the Experiencer and sensorily stimulates that entity on encounter with it . This is the -as-Sourcetype of sensorypath . Sight, in particular , is thus treated either Experienced as a probing system that emanatesfrom or is projected forth by a viewer so as to

" Fictive Motion in Languageand " Ception

225

detect some object at a distance , or elseas a visual quality that emanatesfrom some . distal object and arrives at an individual , thereby stimulating a visual experience so as lexicalized verb a We can first illustrate this phenomenon using nonagentive . In ( 16 to take the Experiencer as subject, namely, see ) the two oppositely directed : two different are fictive motion of by path phrases represented paths ' ( 16 ) a. The enemycan seeus from where they re positioned. ' b. ' rrhe enemycan seeus from where we re standing. -as-source sentence like ( 16b Somespeakershave difficulty with with an experiencer ), as shown but this difficulty generally disappearsfor the counterpart passivesentence , in ( 17b ). ' ( 17 ) a. We can be seenby the enemy from where they re positioned. ' b. We can be seenby the enemy from where we re standing. And generally no problem arisesat all for nonvisual sensorypaths, such as those for audition or olfaction shown in ( 18 ). ' ( 18 ) a. I can hear/ smell him all the way from where I m standing. ' b. I can hear/ smell him all the way from where he s standing. The bidirectional conceptualizability of sensorypaths can also be seenin alternatives of lexicalization. Thus, among the nonagentive vision verbs in English, see is lexicalized to take the Experiencer as subject and the Experiencedas direct object, . But show is thereby promoting the interpretation of the Experiencer as Source lexicalized to take the Experienced as subject and can take the Experiencer as the object of the preposition to, thereby promoting the interpretation of the Experienced . We illustrate in ( 19 as Source ). -by can seethe old wallpaper through the paint . ) a. Even a casual passer ( 19 -by. b. The old wallpaper shows through the paint even to a casual passer Despite theseforms of alternative directionality , fictive visual paths may generally . This is the casefor English, where some forms with favor the Experienceras Source the Experiencedas Sourceoffer difficulty to somespeakers , and the useof a verb like . Further , agentiveverbs of vision in showis minimal relative to that of a verb like see English are exclusively lexicalized for the Experiencer as subject and can take directional . As shown in (20a), this is the case phrasesonly with the Experienceras Source with the verb look, which takes the Experiencer as subject and allows a range of directional prepositions. Here the conceptualization appears to be that the Agent subject volitionally projects his line of sight as a Probe from himself as Sourcealong the path specifiedby the preposition relative to a ReferenceObject (the Experienced

226

Leonard Talmy

is not named in this type of construction) . However, there is no (20b)-type construction with look in which the visual path can be represented as if moving to the Experienceras goal. (20) a. ' looked into / toward/past/away from the valley. b. * ' looked out of the valley (into my eyes ). ' where am located outside the < valley>

6.6 A UnifyingPrincipleandan Explanatory Factorfor Emanation Types


So far , this chapter has laid out the first -level linguistic phenomena that manifest different types of fictive emanation. It is now time to consider the principles that . govern and the context that generalizesthesephenomena In the preceding part of the chapter, the conceptualizations associatedwith the different types of emanation were treated as distinct. But underlying such diversity, one may discern commonalities that unite the various types and may posit still deeper . We presenthere a unifying principle phenomenathat can account for their existence and an explanatory factor. 6.6.1 The Principle that Detenninesthe Sourceof Emuadon For the emanation types in which a fictive path extendsbetweentwo objects, we can seekto ascertaina cognitive principle that determineswhich of the two objectswill be conceptualizedas the source of the emanation, while the other object is understood as the goal. On examination, the following cognitive principle appearsto be the main one in operation: the object taken to be the more active or determinative of the two is conceptualized as the source of the emanation. This will be called the " activedeterminative principle ." We can proceed through severalrealizations of this principle that have functioned in the earlier examples . Thus, as betweenthe sun and my hand, or the sun and the cave wall , the sun is perceivedas the brighter of the two objects. This greater brightness seemsto lead to the interpretation that the sun is the more active object, in particular , more energeticor powerful . By the operation of the active-determinative , and perhaps perceived , as the sourceof the principle , the sun will be conceptualized radiation moving through spaceto impinge with the other object, rather than any of the alternative feasibleconceptualizationspresentedearlier. Another application of the active-determinative principle can be seenin shadow , say, a pole and the shadow of the pole, the pole is the more paths. As between determinative entity , while the shadow is the more contingent or dependent entity . This is understood from such evidence as that in total darkness or in fully diffuse

" " Fictive Motion in Languageand Ception

227

. Further , one can move the pole light , the pole is still there but no shadow is present and the shadow will move along with it , whereasthere is no comparable operation performable on the shadow. By the operation of the active determinative principle , the shadow-bearing object is thus conceptualized as generating the shadow, which then moves fictively from that object to an indicated surface. That is, it is by the operation of the principle that this interpretation of the direction of the fictive motion prevails, rather than any alternative interpretation such as that the shadow moves from the indicated surface to the physical object. A further realization of the active-determinative principle can be seenin the caseof agentive sensorypaths, that is, ones with an Experiencer that acts as an intentional Agent as well as with an Experienced entity . Here it seemsthe very property of exercisedagency leads to the interpretation that the Agent is more active than the Experiencedentity , which is either inanimate or currently no~ manifesting relevant . By the operation of the active-determinative principle , then, the agentive agency Experienceris conceptualizedas the Sourceof the sensorypath, whosefictive motion . In the visual example presented thus proceedsfrom the Experiencer to the Experienced the referent of " I " is understood as an earlier, I looked into the valley, because " " , while the referent of valley is understood as a nonagentive agentive Experiencer Experiencedentity, the active-determinative principle requires that the Experiencer be conceptualizedas the Source of the fictive sensorymotion , and this, in fact, is the . only available interpretation for the sentence The active-determinative principle also holds for those types of orientation paths that are agentive , targeting paths and agentive demonstrative paths, , for example where the active and determinative entity in the situation is the agent who fixes the ' orientation of the front -bearing object, such as a cameraor the Agent s own arm with extended index finger. With our principle applying correctly again, it will be this object, positioned at the active-determinative locus, that will be conceptualizedas the sourceof the fictive emanation. The fact that nonagentivesensorypaths can be conceptualizedas moving in either of two opposite directions might at first seemto challengethe principle that the more active or determinative entity is treated as the source of fictive emanation. But this . It may be that either object can, by different criteria , each be need not be the case that is more active than the other. For example the one as , by one set of interpreted criteria , a nonagentivelyacting Experiencer , from whom a detectional probe is taken to emanate , is interpreted as more active than the entity probed. But under an alternative set of criteria , the Experiencedentity taken to emit a stimulus is interpreted as being more active than the entity stimulated by it . Thus the active-determinative . The task remaining, though, is to ascertainthe additional cognitive principle is saved criteria that ascribe greater activity to one set of phenomenaor to a competing set,

228

Leonard Talmy

and that are in effect in the absenceof the principle ' s already known criteria (e.g., greater agencyor energeticness ). Finally , there is a remainder of emanation types to which the active-determinative principle does not obviously apply in any direct way, namely, the nonagentiveorientation path types: prospect paths, alignment paths, and nonagentive demonstrative paths. Here the fictive motion emanatesfrom only one of the two relevant entities, but this entity is not apparently the more active or determinative of the two. In these cases , however, the directionality of the fictive motion may be set indirectly by the conceptual mapping of principle-determined cases onto the configuration, as described in the next section. 6.6.2 Poaible Basisof Fictive Emanation and its Types If we have correctly ascertainedthat the more active or determinative entity is conceptualized as the Source of fictive emanation, the next question to ask is why this should be the case . We speculate here that the active-determinative principle is a consequenceof a foundational cognitive system every sentient individual has and ' . Specifically , that of agency , the individual s exerciseof agencyfunctions experiences as the model for the Source of emanation. We remain agnostic on whether the connection is learned or innate. If it is learned in the course of development , then each individual ' s experienceof agency leads by steps to the conceptualization of fictive emanation. If it is innate, then somethinglike the samestepsmay have beentraversed by genetically determined neural configurations as theseevolved. Either way, we can suggestsomething of the stepsand their consequentinterrelationships. The exerciseof agencycan be understood to have two components , the generation of an intention and the realization of that intention (cf. Talmy 1976 , forthcoming) . An intention can be understood as one' s desirefor the existenceof somenew state of affairs where one has the capability to act in a way that will bring about that state of affairs. The realization component, then, is one' s carrying out of the actions that bring about the new state of affairs. Such exerciseof agency is experiencedas both active and determinative. It is active becauseit involves the generation of intentions and of actions, and it is determinative becauseit remodelsconditions to accord with one' s desires . In this way, the characteristicsof agencymay provide the model for the active-determinative principle . The particular form of agency that can best serve as such a model is that of an ' " Agent s affecting a distal physical object- what can be called the agent-distal object patterns In this pattern an Agent, say, intending to affect the distal object must either move to it with her whole body, reach to it with a body part , or cause(as by throwing) someintermediary object to move to it . The model-relevant characteristics

" " Fictive Motion in Languageand Ception

229

of this fonn of agencyare that the detennining event, the act of intention , takes place at the initial locus of the Agent, and the ensuingactivity that finally affects the distal es through spacefrom that initial locus to the object. But these are object progress also the characteristicsof the active-detenninative principle : namely, the more active or detenninative entity is the Source from which fictive motion emanatesthrough spaceuntil reaching the less active or detenninative entity, the distal object. Hence one can posit that the pattern of agencyaffecting a distal object is the model on which . the active-detenninative principle is based In particular , we can see how the agent-distal object pattern can serve as the model for the two main agentive fonns of emanation, namely, agentive demonstrative paths and agentive sensorypaths. To consider the fonner casefirst , the specific agent-distal object pattern of extending the ann to reach for someobject may directly act as the model for agentive demonstrative paths, such as an Agent extending his ann and pointing with his finger. In both cases , the extending ann typically exhibits actual motion away from the body along a line that connectswith the target object, ' where, when fully extended , the ann s linear axis coincides with its path of motion . Possibly some role is played by the fact that the more acute tapered end of the ann , the fingers, leads during the extension and is furthest along the line to the object . Such an agentive demonstrative path might then when the ann is fully extended for the in turn serveas the model , one associatedwith nonagentive type, for example a figure like an arrow , whose linear axis also coincides with the line between the arrow and the distal object, and whose tapered end is the end closest to the distal object and the end conceptualizedas the Source from which the demonstrative line . emanates Similarly, we can see parallels between the agent-distal object pattern, in which an Agent executesfactive motion toward distal object, and agentive visual sensory paths, in which an Experiencerprojects a fictive line of sight from himself to the distal , like the Agent, the Experiencer is active and detenninative; like object. Specifically ' the Agent, the Experiencerhas a front ; like the Agent s moving along a straight line betweenhis front and the distal object, the intangible line of sight movesin a straight ' line betweenthe front of the Experiencerand the distal object; like this line s moving away from the initial locus of the Agent, the visual sensorypath movesaway from the ' ; like the Agent s motion continuing along this line until it Experiencer as Source es until it encounters the distal reachesthe object, the visual sensory path progress ' s motion in the the of . Thus the physical world appears to Agent object perception be mapped onto the conceptualization of an intangible entity moving along a line. ' Again, such a mapping might either be the result of learning during an individual s , or might have beenevolutionarily incorporated into the perceptual and development

230

Leonard Talmy

' conceptual apparatus of the brain. Either way, an organism s production of factive motion can becomethe basis for the conceptualization of fictive motion . In turn , this agentive visual type of fictive emanation may serve as the model for severalnonagentive emanation types. In particular , this modeling may occur by the conceptual mapping or superimposition of a schematizedimage- that of an ' Experiencers front emitting a line of sight that proceedsforward into contact with a distal object- onto situations amenableto a division into comparably related components . Thus, in the prospect type of orientation path, the Experiencercomponent may be superimposedonto , say, a cliff , with her face corresponding to the cliff wall , with her visual path mapped onto the conceptualized schematic component of a prospect line moving away from the wall , and with the distal object mapped onto the vista toward which the prospect line progress .6 es In a similar way, the schemafor the agentivevisual path may get mapped onto the radiation situation , where the Experiencer , as the active determinative Agent, is associated - the brightest component with the most energeticcomponent of the radiation scene in the caseof light , say, the sun. The visual path is mapped onto the radiation itself, for example , onto light visible in the air (especially , say, a light beam, as through an aperture in a wall ), and the distal object is mapped onto the less bright . The direction of motion conceptualizedfor the visual path is also object in the scene onto the radiation , which is thus conceptualizedas moving from the brighter mapped to the duller object object. An association of this sort can explain why much folk iconography depicts the sun or moon as having a face that looks outward. As for shadow paths, the model may be the situation in which the agentive Experiencer herself stands and views her own shadow from where she is located. Once again, the visual path moving from this Experiencerto the ground location of the shadow is mapped onto the conceptualization of the fictive path that the shadow itself traversesfrom the solid body onto the ground. A reinforcement for this mapping is that the Experiencer is determinative as the Agent and the solid object is determinative over the shadow dependenton it . The only emanation types not yet discussedin terms of mapping are the nonagentive sensory paths that can proceed in either direction. The direction from Experiencerto Experiencedis clear becausethat is the sameas for agentive viewing. - where the Experiencedemits a Stimulus- on We may account for the reversecase the grounds that it , too , can serveas a receptiveframe onto which to superimposethe model of an Agent emitting a visual path . What is required is simply the conclusion that the conceptualization of an object emitting a Stimulus can be taken as active enough to be treated as a kind of modest agencyin its own right , and henceto justify this conceptual superimposition of an Agent onto it .

" " Fictive Motion in Languageand Ception

231

in OtherCognitive to Counterparts in Language of Emanation 6.7 Relation Systems


In this section we present a number of apparent similarities in structure or content between the emanation category of fictive motion in language and counterparts of . We mainly consider emanation in cognitive systemsother than that of language similarities that language has to perception and to cultural conceptual structure, as well as to folk iconography, which may be regardedas a concretesymbolic expression of perceptual structure. A brief description of our model of cognitive organization, referred to in the introduction , will first provide the context for this comparison. " Model of 6.7.1 " OverlappingSystems Cognitive Organization ' ' Converging lines of evidencein the author s and others researchpoint to the following picture of human cognitive organization. Human cognition comprehends a certain number of relatively distinguishable cognitive systems of fairly extensive . This research has considered similarities and dissimilarities of structure compass - in particular , of conceptual structure- between language and each of these : visual perception, kinesthetic perception, reasoning other cognitive systems , attention , memory, planning, and cultural structure. The general finding is that each cognitive system has some structural properties that may be uniquely its own; some further structural properties that it shareswith only one or ~ few other cognitive ; and some fundamental structural properties that it has in common with all systems . We assumethat each such cognitive systemis more integrated the cognitive systems and interpenetrated with connectionsfrom other cognitive systemsthan is envisaged " ) . We call this view the overlapping by the strict modularity notion (cf. Fodor 1983 " model of 7 cognitive organization. systems 6.7.2 Fictive Emanationand Perception The visual arrays that might yield perceptualparallels to the emanation type of fictive motion have been relatively less investigated by psychological methods than in the caseof other categoriesof fictive motion (seebelow) . One perceptual phenomenon related to orientation paths has beendemonstratedby Palmer ( 1980 ) and Palmer and co oriented of in certain who found that Bucher ( 1981 equilateral arrays consisting ), triangles, subjectsperceiveall the triangles at once pointing by turns in the direction . Moving the array in the direction of one of one or another of their common vertices of the common verticesblasesthe perception of the pointing to be in the direction of that vertex, although theseexperimentsdid not test for the perception of an intangible " " line emerging from the vertex currently experiencedas the pointing front of each triangle or of the array of triangles. One might needexperiments , for example ,

232

Leonard Talmy

' that test for any difference in a subject s perception of a further figure depending on whether or not a fictive line was perceivedto emergefrom the array of triangles and pass through that figure. But confirmation of a perceptual analogue to emanation . paths must await such research " ' " We can also note that Freyd s work on representationalmomentum (e.g., Freyd 1987 ) does not demonstrate perception of orientation paths. This work involved the more forward locations. The subjects sequential presentation of a figure in successively the last did exhibit a bias toward perceiving presentedfigure further ahead than its actual location. But this effect is presumably due to the factively forward progression of the figure. To check for the perceptual counterpart of linguistic orientation paths, experimentsof this type would need to test subjectson the presentation of a single picture containing a forward -facing figure with an intrinsic front . The robust and extensiverepresentationof fictive emanation in languagecalls for psychologicalresearchto test for parallels to this category of fictive motion in perception . That is, the question remains whether the appropriate experimental arrangements will show particular perceptionsfor this category that accord with the general fictivity pattern, hencewith the concurrent perception of two discrepant representations , , one of them more palpable and veridical than the other. Consider, for example visual arrays that include various front -bearing objects, designed to test the perception of fictive orientation paths in their several distinct types- prospect paths, alignment paths, demonstrative paths, and targeting paths. One would need to determine whether subjects , on viewing these arrays, see the factive stationariness of the depicted objects at the fully palpable level of perception, but concurrently ' sensethe fictive motion of something intangible emanating from the objects fronts at a faintly palpable level of perception. Similarly , to probe for visual counterparts of linguistic radiation paths, research will need to test for anything like a fictive and less palpable perception of motion along a light beam, in a direction away from the brighter object, that is concurrent with , perhapssuperimposedon, the factive and more palpable perception of the beam as static. Similarly , to test for a visual parallel to linguistic shadow paths, experimental , on viewing a scenethat contains procedureswill need to probe whether subjects an object and its shadow, have some fictive, less palpable senseof the shadow as , concurrently with having moved from that object to the surfaceon which it appears a factive and palpable perception of everything within the scene as stationary. , one Finally , to check for a perceptual analogue of visual sensorypaths in language ' can use either a scenethat depicts someonelooking or a subject s own processof looking at entities to ascertain whether subjects simply perceive a static array of entities or superimposeon that array a lesspalpable perception of motion along the probing line of sight.

Fictive Motion in Languageand " Ception"

233

6.7.3 Fictive Emanationand Folk Iconography Fictive representationsthat are normally only sensedat a lower level of palpability . An example to be cited can sometimesbe modeled by fully palpable representations below is the use of stick figure drawings or of pipe,cleaner sculptures to explicitly ' . In the sameway, image objects schematicstructure, which is normally only sensed various other aspectsof fictive emanation normally only sensedhave been made explicit in the concrete depictions of folk iconography. For example , fictive sensory paths of the agentive visual type are linguistically conceptualized as intangible lines that Agents project forward from their eyes through space into contact with distal objects. But this is exactly the character of ' " " Supermans Xray vision as depicted in comic books. Supermansendsforth from his eyesa beam of Xrays that penetratesopaque materials to make contact with an 's . Note that Superman otherwise obscured object and permits it to be seen Xray vision is not depicted as stimuli that emanate from the obscured object and proceed 's . Such toward and into Superman eyeswhere they might be perceptually registered an Experiencedto Experiencer path direction might have been expected from our understanding of Xray equipment, where the radiation moves from the equipment . This plate might have onto a photographic plate on which the image is registered 's in which the Agent but the model been analogized to Superman , eyes conceptual emits a sensoryProbe appearsto hold sway in the cartoon imagery. Comparable examplesbasedon the linguistic conceptualization of an Agent emitting a visual Probe are representednot only by grammatical constructions and other -classforms, but also by metaphoric expressions . Thus the expression" to look closed ' " daggersat , as in Jane lookeddaggersat John, representsthe notion that Jane s mien, reflecting a current feeling of hate for John, can be elaborated as the projection of weapons from her eyes to John; indeed, cartoon depictions actually show a line of ' . daggersgoing from the experiencers eyesto the body of the experienced The linguistic conceptualization of fictive demonstrative paths emerging from the point -type front of a linear object, as from a pointing finger, seemsalso to parallel a type of iconographic depiction. This is the depiction of magical power beamsthat an , movies and comic Agent can project forth from his extendedfingertips. For example books often have two battling sorcerersraise their extendedhands and direct destructive beamsat each other. ' Finally , it is the author s observation- though a careful study would be neededto confirm this- that in the processof drawing the sun, schematically , after completing and adults for the of the sun both children a circle , represent its radiation body with lines drawn radially outward from the circle, not inward toward it . If so, this iconographic procedure reflects the linguistic conceptualization of fictive radiation paths as emanating and moving off from the brightest object. Further , iconographic

234

Leonard Talmy

representationsof the sun and moon often depict a face on the object, as if to represent the object as containing or comprising an Agent that is emitting the radiation of light . As noted in section 6.6.2, a representationof this sort can be attributed to the mapping of the schemaof an agentive visual sensorypath onto the radiation situation , much as it may be mapped onto other fictive motion types. 6.7.4 Relation of Fictive Emanationto Ghost Physicsand Other Anthropological Phenomena We can discern a striking similarity between fictive motion - in particular, orientation paths- and the properties exhibited by ghosts or spirits in the belief systemsof many traditional cultures. The anthropologist Pascal Boyer ( 1994 ) seesthese properties as a culturally pervasiveand coherent conceptual system which he calls " ghost , " . Boyer holds that ghost and spirit phenomena obey all the usual causal physics expectations for physical or social entities, with only a few exceptions that function as " attention attractors." Certain of these exceptions have widespread occurrence acrossmany cultures, such as invisibility or the ability to passthrough walls or other solid objects, but other kinds of potential exceptions , which on other grounds might have seemed just as suited for conceptualization as specialproperties, instead appear never to occur. An exampleof this is temporally backward causality; that is, cultural belief systemsseemuniversally to lack a concept that a ghost can at one point in time bring about somestate of affairs at a prior point of time. Boyer has no explanation for the selection of particular exceptions that occur in ghost physics and may even find them arbitrary . However, we can suggestthat the pattern of standard and exceptional properties is structured and cognitively principled . In fact, the findings reported in this chapter may supply the missing account. The exceptional phenomena found to occur in ghost physics may be the same as certain cognitive phenomenathat already exist in other cognitive systems , and then are tapped for service in cultural spirit ascriptions. The linguistic expression of fictive demonstrative paths and its gestural counterpart may well afford the relevant properties. To consider gesturefirst , if I , for example , am inside a windowlessbuilding and am asked to point toward the next town , I will not , through gesticulations , indicate a at leads the out the exit of the that path begins my finger, through open doorway, building, turns around and then movesin the direction of the town. On the contrary, I will simply extend my arm with pointed finger in the direction of the town , regardless of the structure around me. That is to say, the demonstrative path, effectively conceptualizedas an intangible line emergingfrom the finger, itself has the following crucial properties: ( I ) it is invisible, and (2) it passesthrough walls- the very same properties ascribedto spirits and ghosts.

Fictive Motion in Languageand " Ception"

235

Thesesameproperties hold for the conceptualization that accompaniesthe linguistic this , in the set of sentences expressionof fictive demonstrative paths. For example arrow points to/ toward/past/away from the town, the use of any of the directional prepositions suggeststhe conceptualization of an intangible line emerging from the front end of the arrow , following a straight coursecoaxial with the arrow ' s shaft, and moving along the path representedby the preposition. Once again, this imaginal line is invisible and would be understood to passthrough any material objects presenton its path . In addition to such demonstrative paths, we can observefurther relations between cultural conceptualizations and another type of fictive emanation, that of agentive " visual paths. Consider the notion of the " evil eye , found in the conceptual systems of many cultures. In a frequent conception of the evil eye, an agent who bearsmalevolent feelings toward another person is able to transmit the harmful properties of thesefeelingsalong the line of his gazeat the other person. This is the sameschema as for a fictive visual path : the Agent as Sourceprojecting forth something intangible along his line of sight to encounter with a distal object. Relations between fictive motion and cultural conceptualizations extend still further . One may look to such broadly encounteredcultural conceptsas those of mana, power, fields of life force, or magical influence emanating from entities; these forms of imagined energy- just like the fictive emanations of linguistic construals- are ?) as being invisible and intangible, as being (generated conceptualized(and perceived and) emitted by someentity, as propagating in one or more directions away from that entity, and in some forms as then contacting a second distal entity that they may affect. The structural parallel between such anthropological concepts of emanation and the emanation type of fictive motion we have here described for language is evident and speaksto a deepercognitive connection. It thus seemsthat the general fictivity pattern generatesthe imaginal schemasof fictive motion in the cognitive systemsnot only of languageand of visual perception, but also of cultural cognition, specifically in its conceptualizations of spirit and , the structure of such conceptions as power. That is, in the cognitive culture system harmful influence and , , ghost phenomena magical energyappearsnot to be arbitrary . Nor does it exhibit its own mode of construal or constitute its own domain of conceptual constructs of the sort posited, for example , by Keil ( 1989 ) and Carey ( 1985 ) for other categoriesof cognitive phenomena . Rather, it is probably the same as or a parallel instance of conceptual organization already extant in other cognitive " framework outlined above . In terms of the " overlapping systems , general systems fictivity of this sort is thus one area of overlap across at least the three cognitive , visual perception, and cultural cognition . systemsof language

236 6.8 Further Categories of Fictive Motion

Leonard Talmy

As indicated earlier, languageexhibits a number of categoriesof fictive motion beyond the emanation type treated thus far. We here briefly sketch five further categories ; for each, we suggestsomeparallels in visual perception that have already been or might be examined.8 The purpose of this section is to enlarge both the linguistic -perception parallelism. In the illustrations scopeand the scopeof potential language are provided, as a foil for comparison, with that follow , the fictive motion sentences factive motion counterpart sentences , shown within brackets. 6.8.1 Pattern Paths The pattern paths category of fictive motion in languageinvolves the fictive conceptu. In this type, the literal alization of some configuration as moving through space senseof a sentencedepicts the motion of some arrangement of physical substance along a particular path, while we factively believethat this substanceis either stationary or movesin someway other than along the depicted path . For the fictive effect to occur, the physical entities must factively exhibit some form of motion , qualitative , but thesein themselvesdo not constitute the , or appearance /disappearance change fictive motion . Rather, it is the pattern in which the physical entities are arranged that exhibits the fictive motion . Consider the example in (21) . (21) Pattern paths As I painted the ceiling, (a line of ) paint spots slowly progressedacrossthe floor . [cf. As I painted the ceiling, (a line of ) ants slowly progressedacrossthe floor .] Here eachdrop of paint does factively move, but that motion is vertically downward in falling to the floor . The fictive motion , rather, is horizontally along the floor and involves the linear pattern of paint spots already located on the floor at any given time. For this fictive effect, one must in effect conceptualize an envelope located around the set of paint spots or a line located through them. The spots thus enclosed within the envelopeor positioned along the line can then be cognizedas constituting a unitary Gestalt linear pattern. The appearanceof a new paint spot on the floor in front of one end of the linear pattern can then be conceptualizedas if that end of the envelope or line extended forward so as now to include the new spot. Such is the forward fictive motion of the configuration . By contrast, if the sentencewere to be were to be treated interpreted literally - that is, if the literal referenceof the sentence as factive one would have to believethat the spots of paint physically slid forward along the floor .

Fictive Motion in Languageand " Ception"

237

In one respect , the pattern paths type of fictive motion is quite similar to the emanation type. In both these categories of fictive motion , an entity that is itself fictive- an imaginal construct- moves fictively through space . One difference , though, is that the emanation type does not involve the factive motion of any elements within the referent scene . Accordingly, it must depend on a principle - the active-determinative principle- to fix the source and direction of the fictive motion . But the pattern paths type does require the factive motion or the change of some components of the referent situation for the fictive effect to occur; indeed, this determines the direction of the fictive motion , so that no additional principle need come into play . The perceptual phenomenagenerally termed apparentmotion in psychology would seemto include the visual counterpart of the pattern paths type of fictive motion in . But to establish the parallel correctly, one may needto subdivide apparent language motion into different types. Such types are perhaps largely basedon the speedof the . process viewed and, one may speculate , involve different perceptual mechanisms Most researchon apparent motion has employed a format like that of dots in two locations appearing and disappearing in quick alternation. Here, within certain , subjects perceive a single dot moving back and forth between the two parameters locations. In this fast form of apparent motion , the perceptual representation most palpable to subjectsis in fact that of motion , and thus would not correspond to the . linguistic case On the other hand, there may exist a slower type of apparent motion that can be . One example might consist of a perceivedand that would parallel the linguistic case a row of bulbs in which one after another bulb is briefly turned subject viewing light on at consciously perceivable intervals. Here, it may be sUr Dlised , a subject would have an experiencethat fits the generalfictivity pattern . The subject will perceiveat a higher level of palpability , that is, as factive, the stationary state of the bulbs, as well as the periodic flashing of a bulb at different locations. But the subject would concurrently it as being at a lower level perceiveat a lower level of palpability - and assess of veridicality - the fictive motion of a seeminglysingle light progressing along the row of bulbs. 6.8.2 Frame -Relative Motion With respect to a global frame of reference , a language can factively refer to an 's observeras moving relative to the observer stationary surroundings. This condition is illustrated for English in (20a) and is diagrammedin figure 6.la . But a languagecan alternatively refer to this situation by adopting a local frame around the observer as center. Within this frame, the observer can be represented as stationary and her surroundings as moving relative to her from her perspective . This condition is

238

[!] 0
Figure 6.1 Frame-relative motion : global and local.

Leonard Talmy

illustrated in (20b) and diagrammed in figure 6.1b. This condition is thus a form of fictive motion , one in which the factively stationary surroundings are fictively depicted as moving . In a complementary fashion, this condition also contains a form of fictive stationariness , for the factively moving observeris now fictively depicted as stationary. Stressingthe depiction of motion , we term the fictive effect here observer based frame relative motion. Further , a languagecan permit shifts between a global and a local framing of a . For instance situation within a single sentence , (22C) shifts from the global frame to the local frame and, accordingly, shifts from a directly factive representation of the . But one condition no language seems spatial conditions to a fictive representation able to representis the adoption of a conceptualization that is part global and part local, and accordingly, part factive and part fictive. Thus English is constrained against sentenceslike (220 ), which suggeststhe adoption of a perspective point midway betweenthe observerand her surroundings. (22) Frame-relative motion : With factively moving observer A . Globalframe : Fictive motion absent I rode along in the car and looked at the scenerywe were passingthrough . B. Local frame : Fictive motion present I sat in the car and watched the sceneryrush past me. [cf. I sat in the movie set car and watched the backdrop sceneryrush past me.] C. Shift in midreference from global to localframe , andfrom factive to fictive motion I was walking through the woods and this branch that was sticking out hit me. [cf. I was walking through the woods and this falling pineconehit me.]

" " Fictive Motion in I .ADg \ lage and Ception

239

D . Lacking: Part -global, part -localframe withpart -factive, part-fictive motion * We and the sceneryrushed past each other. cf. We and the [ logging truck rushed past each other.] In the precedingexamples , the observerwas factively in motion while the observed (e.g., the scenery ) was factively stationary- properties expressedexplicitly in the global framing . In a complementary fashion, a sentencecan also expressa global . This framing in which, factively, the observeris stationary while the observedmoves situation is illustrated in (23 Aa , Ab ) . However, this complementary situation differs from the earlier situation in that it cannot undergo a local reframing around the , one could find stationary observer as center. If such a local frame were possible sentences that the observer as and the observedas acceptable fictively depict moving . But sentences stationary , (23 Ba) with a attempting this depiction for example uniform local framing and (23 Bb) with a shift from global to local framing - are . The unacceptable fictive local framing that they attempt is diagrammed unacceptable in figure 6.lc . (23) Frame-relative motion : With factively stationary observer A . Globalframe : Fictive motion absent a. The stream flows past my house . b. As I sat in the stream, its water rushed past me. B. Local frame : Blocked attempt at fictive motion a. * My house advancesalongsidethe stream. b. * As I sat in the stream, I rushed through its water. We can suggest an account for the difference between moving and stationary observers in their acceptanceof fictive local framing . The main idea is that stationarinessis basic for an observer . Accordingly, if an observeris factively moving, a sentenceis free to representthe situation as such, but a sentencemay also " ratchet down" its representationof the situation to the basic condition in which the observer is stationary. However, if the observer is already stationary, that is, already in the basic state, then a sentencemay only representthe situation as such, and is not free to " ratchet up" its representationof the situation into a nonbasic state. If this explanation holds, the next question is why it should be that stationariness is basic for an observer. We can suggesta developmentalaccount. An infant experiences optic flow from forward motion while being held by a parent long before the at stage which it locomotes, a stageat which it will agentively bring about optic flow itself. That is, before the infant has had a chanceto integrate its experienceof moving into its perception of optic flow , it has months of experienceof optic flow without an experienceof motion . This earlier experiencemay be processedin terms of the

240

Leonard Talmy

surrounding world as moving relative to the self fixed at center. This experiencemay be the more foundational one and persist to show up in subtle effects of linguistic . representationslike thosejust seen One possible corroboration of this account can be cited. Infants at the outset do have one fonn of agentive control over their position relative to their surroundings, namely, turning the eyesor head through an arc. Rather than the forward type of , this action brings about a transverse type, although not optic flow just discussed extended rotation . Becausethe infant can thus integrate the experienceof motor control in with experienceof transverseoptic flow at a foundational level, we should not expectto find a linguistic effect that treats observerstationarinessas basic relative ' s arc-sized to an observer , typically turning motion . Indeed, English, for one language pennits only factive representationsof such turning by an observer , for example ' . It does not , I looked over all the room s decorations , As I quickly turned my head . ratchet down to a fictive state for the observer as in As I quickly , typically stationary 's decorations turned my head the room in of the latter , spedby front of me. A sentence sort would be used only for special effect, not in the everyday colloquial way the forward motion case is treated. On the other hand, as still further corroboration , becauseextended spinning is not part of the infant ' s early experience , it should behave like forward translational motion and pennit a linguistic refraIning. Indeed, this is readily found , as in English sentenceslike As our space shuttle turned, we watchedthe heavens spin around us, or I rode on the carouseland watchedthe world go round. Psychological experiments have afforded severalprobable perceptual parallels to frame-relative motion in language . One parallel is the " induced motion " of the " rod " and frame genreof experiments . Here, prototypically , while a rectangular shapethat surrounds a linear shapeis factively moved, somesubjectsfictively perceivethis frame as stationary while the rod moves in a complementary manner. However, this genre -basedin our sensebecausethe observer is not one of of experimentsis not observer the objects potentially involved in motion . Closer to our linguistic caseis the " motion aftereffect," present where a subject has been spun around and then stopped. The a perception subject factively knows that he is stationary, but concurrently experiences - assessed as lessveridical, hencefictive- of the surroundings as turning about him in the complementary direction. Perhaps the experimental situation closest to our linguistic type would in fact be a subject' s moving forward through surroundings, much as when riding in a car. The question is whether such a subject will concurrently perceivea factive representation of herself as moving through stationary surroundings , and a fictive representation of herself as stationary with the surroundings as moving toward and past her.

Fictive Motion in Languageand " Ception 6.8.3 . Advent Pat I L

"

241

' An adventpath is a depiction of a stationary object s location in terms of its arrival . The stationary state of the object is factive, or manifestation at the site it occupies whereasits depicted motion or materialization is fictive and, in fact, often wholly implausible. The two main subtypes of advent paths are site arrival , involving the fictive motion of the object to its site, and site manifestation , which is not fictive the object at its site. fictive manifestation of the motion but fictive change , namely This category is illustrated in (22) . (24) Advent paths A . Site arrival I . With active verbform . a. The palm treesclusteredtogether around the oasis the ice cream clustered around The children cf: together [ quickly truck .] b. The beam leans / tilts away from the wall. / tilted away from the wall.] [cf: The loose beam gradually leaned 2. With passiveverbform c. Termite mounds are scattered / distributed all over the / strewn/spread plain . / distributed all over / strewn/spread [cf. Gopher traps were scattered the plain by a trapper.] B. Site manifestation d. This rock formation occurs/ recurs /reappears / shows up near /appears volcanoes. / shows up near /appears /reappears [cf. Ball lightning occurs/ recurs volcanoes.] For a closer look at one site arrival example , (24a) uses the basically motion specifying verb to cluster for a literal but fictive representation of the palm trees as having moved from some more dispersedlocations to their extant neighboring locations . But the concurrent factive representation of this scene is around the oasis contained in our belief that the treeshave always beenstationary- located in the sites they occupy. Comparably, the site manifestation examplein (24d) literally represents the location of the rock formation at the sites it occupiesas the result of an event of materialization or manifestation. This fictive representation is concurrent with our believed factive representation of the rock formation as having stably occupied its sitesfor a very long time. We can cite two psychologistswho have made separateproposals for an analysis of visual forms that parallels the linguistic site arrival type of fictive motion . Pentland

242

Leonard Talmy

( 1986 ) describesthe perception of an articulated object in terms of a processin which a basicportion of the object, for example , has the remaining portions , its central mass moved into attachment with it . An example is the perception of a clay human figure as a torso to which the limbs and head have beenaffixed. Comparably, Leyton ( 1992 ) describesour perception of an arbitrary curved surface as a deformed version of a , a smooth closed surface is describedas the deformation ; for example simple surface of a sphere , andresis , one that has undergone protrusion , indentation , squashing es correspondsto the psychologically salient tance. He shows that this set of process causal descriptions that people give of shapes , say, of a bent pipe or a dented door. In a similar way, as describedin the tradition of Gestalt psychology, certain forms are regularly perceivednot as original patterns in their own right , but rather as the result of some processof deformation applied to an unseenbasic form . An example is the -shapedpieceremoved perception of aPac-Man -shapedfigure as a circle with a wedge from it . To consider this last example in terms of our general fictivity pattern, a subject looking at such aPac -Man shape may concurrently experiencetwo discrepant perceptual . The factive representation , held to be the more veridical and representations PacMan configuration per se. of the static will be that more as palpable, perceived and less veridical The fictive representation , felt as being perceivedas lesspalpable, that starts with a circle, proceedsto the demarcation will consist of an imagined sequence of a wedge shape within the circle, and ends with that wedge exiting or being removed from the circle. 6.8.4 AccessPadis ' An access path is a depiction of a stationary object s location in tenns of a path that some other entity might follow to the point of encounter with the object. What is factive here is the representation of the object as stationary, without any entity traversing the depicted path; what is fictive is the representation of some entity traversing the depicted path, whether this is plausible or implausible. Though it is not specified , the fictively moving entity can often be imagined as being a person, ' some body part of a person, or the focus of a person s attention , depending on the , as can be seenin the examplesof (25) . particular sentence (25) Accesspaths a. The bakery is acrossthe street from the bank. [cf. The ball rolled acrossthe street from the bank.] b. The vacuum cleaner is down around behind the clothes hamper. [cf. I extendedmy ann down around behind the clothes hamper.] c. The cloud is 1,000 feet up from the ground . [cf. The balloon rose 1,000 feet up from the ground.]

" Fictive Motion in Languageand Ception

"

243

In greater detail, (25a) characterizesthe location of the bakery in terms of a fictive path that beginsat the bank, proceedsacrossthe street, and terminatesat the bakery. This path could be followed physically by a person walking, or perceptually by someone , or solely conceptually by someone shifting her shifting the focus of his gaze attention over her mental map of the vicinity . The depicted path can be reasonable for physical execution, as when I use (25a) to direct you to the bakery when we are inside the bank. But the samedepicted path may also be an improbable one, as when I use (25a) to direct you to the bakery when we are on its side of the street- it is unlikely that you will first cross the street, advanceto the bank, and then recrossto find the bakery. Further , a depicted access path can also be physically implausible or that in That quasar is 10 million lightlike impossible. Such is the casefor referents , an paths such as these yearspast the North Star. Apart from the useof fictive access ' s location can in a factive characterized also be representation directly generally object , as in The bakery and the bank are oppositeeachother on the street. Does the fictivity pattern involving access paths occur perceptually? We can suggest . Subjectscan a kind of experimental design that might test for the phenomenon be shown a pattern containing some point to be focusedon, where the whole can be as involving paths perceived factively as a static geometric Gestalt and/ or fictively " " be a would an plus figure with the letter leading to the focal point . Perhaps example A at the top point and, at the left hand point , a B to be focusedon. A subject might factively and at a high level of palpability perceive a static representation of this , with the B simply located on the left. But concurrently figure much as just described , the subject might fictively and at a lower level of palpability perceivethe B as located at the endpoint of a path that starts at the A and, say, either slants directly toward the B, or moves first down and then left along the lines making up the " " plus. 6.8.5 CoveragePaths A coveragepath is a depiction of the form , orientation , or location of a spatially ' extendedobject in terms of a path over the object s extent. What is factive here is the of any entity traversing the representationof the object as stationary and the absence some of entity moving along or depicted path . What is fictive is the representation , the fictively moving over the configuration of the object. Though it is not specified of attention the focus an observer as be , or the object often can , imagined being entity in the be seen as can itself, depending on the particular sentence examplesof (26) . , it is in 26b is linear Note that in (26a) the fictive path , radially outward over a ( ) two dimensional plane, and in (26c) it is the lateral motion of a line (a north -south line advancing eastward ), that is further correlated with a second fictive change . redness ) (increasing

244

Leonard Talmy

(26) Coveragepaths from the plateau to the valley. a. The fencegoes /zigzags / descends from the plateau to the valley. / descended [cf. I went/zigzagged b. The field spreadsout in all directions from the granary. [cf. The oil spreadout in all directions from where it spilled.] c. The soil reddenstoward the east. [cf. ( I ) The soil gradually reddenedat this spot due to oxidation. (2) The weather front advancedtoward the east.] Consider the fictivity pattern for (26a) . On the one hand, we have a factive representation of the fence as a stationary object with linear extent and with a particular contour , orientation , and location in geographic space . Concurrently, though, we have the fictive representationevoked by the literal sense of the sentence , in which an observer or our focus of attention or some of the fenceitself advancing , , perhaps image along its own axis, moves from one end of the fence atop the plateau, along its length, to the other end of the fencein the valley. We can ask as before whether the generalfictivity pattern involving coveragepaths hasa perceptualanalogue . The phenomenonmight be found in a visual configuration perceived factively at a higher level of palpability as a static geometric form and, concurrently, perceivedfictively at a lower level of palpability in terms of pathways " " , perhapsa subject viewing a plus configuration along its delineations. For example " " will seeit explicitly as just such a plus shape , while implicitly sensingsomething intangible sweepingfirst downward along the vertical bar of the plus and then rightward along the horizontal bar (cf. Babcock and Freyd 1988 ). 6.9 " Ception" : Generalizingover Perceptionand Conception In this section, we suggesta general framework that can accommodate the visual representationsinvolved in general fictivity , together with representationsthat appear in language . Much psychological discussion has implicitly or explicitly treated what it has termedperceptionas a singlecategory of cognitive phenomena . If further distinctions have been adduced , they have been the separatedesignation of part of perception as sensation , or the contrasting of the whole category of perception with that of /cognition. One motivation for challenging the traditional categorization is conception that psychologistsdo not agree on where to draw a boundary through observable psychological phenomenasuch that the phenomenaon one side of the boundary will be considered" perceptual," while those on the other side will be excluded from that , as I view a particular figure before me, is my identification designation. For example

" Fictive Motionin Language and" Ception

245

of it as a knife to be understood as part of my perceptual processingof the visual ? And if stimuli , or instead part of some other, perhaps later, cognitive processing such identification is consideredpart of perception, what about my thought of potential danger that occurs on viewing the object? Moreover, psychologists not only disagreeon where to locate a distinctional boundary, but also on whether there is a principled basison which one can even adduce such a boundary. advisable to establish a theoretical framework that does not Accordingly, it seems a cognitive discrete imply categoriesand clearly located boundaries, and that recognizes domain encompassingtraditional notions of both perception and conception. Such a framework would then further allow for the positing of certain cognitive parametersthat extend continuously through the larger domain (as describedbelow) . To this end, we here adopt the notion of " ception" to cover all the cognitive phenomena , consciousand unconscious , understood by the conjunction of perception and , conception. While perhaps best limited to the phenomena of current processing ception would include the processingof sensory stimulation , mental imagery, and ongoingly experiencedthought and affect. An individual currently manifesting such " " 9 processingwith respectto someentity could be said to ceive that entity . The main advantageof the ception framework in conjoining the domains of perception and conception is not that it eliminates the difficulty of categorizing certain . Though helpful, that characteristic, taken by itself problematic cognitive phenomena could also be seen as , throwing the baby out with the bathwater, in that it by fiat discards a potentially useful distinction simply becauseit is troublesome. The strength of the ception framework , rather, is precisely that it allows for the positing or recognition of distinctional parametersthat extend through the whole of the new domain, parameters whose unity might not be readily spotted across agerryman dered category boundary . Further , such parametersare largely gradient in character and so can reintroduce the basis of the discrete perception- conception distinction in a graduated form . We here propose thirteen parameters of cognitive functioning that appear to extend through the whole domain of ception and to pertain to general fictivity . Most of these parameters seem to have an at least approximately gradient character- with their highest perhapsranging from a fully smooth to a merely rough gradience value at the most clearly perceptual end of the ception domain and with their lowest value at the most clearly conceptual end of the domain. It seemsthat theseparameters tend to covary or correlate with each other from their high to their low ends, that is, any particular cognitive representationwill tend to merit placementat a comparable . Someof the parameters distancealong the gradients of the respectiveparameters seemmore to have discrete regions or categorial distinctions along their lengths than to involve continuous gradience , but these , too , seemamenableto alignment with the

246

Leonard Talmy

. One of the thirteen parameters other parameters , the one that we term palpability , -related general fictivity . Given vision involved with be the most to centrally appears that the other twelve parameterslargely correlate with this one, we term the whole set . that of the palpability -relatedparameters This entire proposal of palpability related parameters is heuristic and programmatic . It will require adjustmentsand experimental confirmation with regard to several issues . One issue is whether the set of proposed parameters is exhaustive with to , whether respect palpability and generalfictivity (presumably not ), and, conversely . the proposed parameters are all wholly appropriate to those phenomena Another issueis the partitioning of general visual fictivity that results in the particular cognitive parametersnamed. Thus perhapssomeof the parameterspresentedbelow should be merged or split . More generally, we would first need to show that our proposed parameters are in synchrony- aligned from high end to low end- sufficiently to . Conversely justify their being classedtogether as components of a common phenomenon , though, we would need to show that the listed parameters are sufficiently , instead of independent from each other to justify their being identified separately treated as aspectsof a single complex parameter. 6.9.1 Palpability and RelatedParameten The parameter of palpability is a gradient parameter that pertains to the degreeof , from the fully palpability with which some entity is experiencedin consciousness concrete to the fully abstract. To serveas referencepoints, four levels can be designated , and , the semiabstract , the semiconcrete along this gradient: the (fully ) concrete the (fully ) abstract. These levels of palpability are discussedthe next four sections and illustrated with examplesthat cluster near them. In this section, we present the . As they are discussedhere, these thirteen proposed palpability -related parameters thirteen parametersare treated strictly with respectto their phenomenologicalcharacteristics . There is no assumption that levels along theseparameterscorrespond to . other cognitive phenomenasuch as earlier or later stagesof processing 1. The parameter of palpability is a gradient at the high end of which an entity is , manifest, explicit , tangible, and palpable. At the low experiencedas being concrete end, an entity is experiencedas being abstract, unmanifest, implicit , intangible, and impalpable. 2. The parameter of clarity is a gradient at the high end of which an entity is experienced as being clear, distinct , and definite. At the low end, an entity is experiencedas being vague, indistinct , indefinite, or murky . 3. The parameter of strength is a gradient in the upper region of which an entity is 1O experiencedas being intense or vivid . At the low end, an entity is experiencedas being faint or dull .

" " Fictive Motion in Languageand Ception

247

4. The ostensionof an entity is our tenD for the overt substantive attributes that an entity has relative to any particular sensory modality . In the visual modality , the " " , ostensionof an entity includesits appearance and motion - thus, more specifically . In the movements of and coloration its fonD auditory , , pattern , texturing including ' modality , ostension amounts to an entity s overt sound qualities, and in the taste modality , its flavors. As a gradient, the parameter of ostensioncomprisesthe degree to which an entity is experiencedas having such overt substantiveattributes. 5. The parameter of objectivity is a gradient at the high end of which an entity is , and as having its experiencedas being real, as having autonomous physical existence " further is an . Such own intrinsic characteristics experiencedas being out entity ' ' - specifically there," that is, as external to oneself , to one s mind , if not also one s ,a body . At the low end of the gradient, the entity is experiencedas being subjective ' s own mental !! . of one a construct activity , product cognitive an entity as 6. The parameter of /oca/izabi/ity is the degreeto which one experiences having a specific location relative to oneself and to comparable surrounding entities ' within somespatial referenceframe. At the high end of the gradient, one s experience is that the entity doeshave a location , and that this location occupiesonly a delimited portion of the whole spatial field, can be detennined, and is in fact known. At midrange levels of the gradient, one may experiencethe entity as having a location but as being unable to detennine it . At the low end of the gradient, one can have the experience that the concept of location does not even apply to the ceived entity . 7. The parameter of identifiability is the degreeto which one has the experienceof recognizing the categorial or individual identity of an entity . At the high end of the ' gradient, one s experienceis that one recognizesthe ceivedentity , that one can assign it to a familiar category or equateit with a familiar unique individual , and that it thus has a known identity . Progressingdown the gradient, the componentsof this experience diminish until they are all absent at the low end. for its 8. The content /structure parameter pertains to whether an entity is assessed content as against its structure. At the content end of this gradient- which correlates - the assessments with the high end of other parameters pertain to the substantive makeup of an entity . At the structure end of the parameter which correlates with the low end of other parameters the assessments pertain to the schematic " " delineations of an entity . While the content end deals with the bulk fonD of an " " entity, the structural end reducesor boils down and regularizes this for In to its . A fonD can be a simplex entity composedof parts abstractedor idealized lineaments or a complex entity containing smaller entities. Either way, when such a fonD is considered overall in its entirety, the content end can provide the comprehensive ' summary or Gestalt of the fonn s character. On the other hand, the structure end can

248

Leonard Talmy

reveal the global framework , pattern, or network of connectionsthat binds the components of the form together and integrates them into a unity . 9. The type of geometry parameter involves the geometric characterization imputed of one' s charto an entity, together with the degreeof its precision and absoluteness acterization. At the high end of this parameter, the assessments pertain to the content of an entity and are (amenableto being) geometrically Euclidean, metrically quantitative , and so on, and absolute. At the low , preciseas to magnitude, form , movements end of the parameter, the assessments pertain to the structure of an entity, and are (limited to being) geometrically topological or topology-like , qualitative or approximative , schematic , and relational or relativistic. 10. Along the gradient parameterof accessibilityto consciousness , an entity is accessible to consciousness everywherebut at the lowest end. At the high end of the parameter or in the foreground of attention. At a , the entity is in the center of consciousness lower level, the entity is in the periphery of consciousness or in the background of or attention , but attention. Still lower, the entity is currently not in consciousness could readily become so. At the lowest end, the entity is regularly inaccessibleto . consciousness 11. The parameter of certainty is a gradient at the high end of which one has the experienceof certainty about the occurrenceand attributes of an entity . At the low end, one experiences uncertainty about the entity - or , more actively, one experiences doubt about it . 12. The parameterof actionability is a gradient at the high end of which one feelsable to direct oneself agentively with respect to an entity - for example , to inspect or manipulate the entity . At the low end, one feelscapable only of receptiveexperience of the entity . is the degreeto which a particular kind of 13. The parameter of stimulus dependence current on line sensorystimulationI in order to occur. experienceof an entity requires At the high end, stimuli must be present for the experienceto occur. In the midrange of the gradient, the experience can be evoked in conjunction with the impingement of does not . At the low end, the experience stimuli , but it can also occur in their absence . to stimulation for its occurrence relation or has no , sensory require, The terms for all the above parameters were intentionally selectedso as to be neutral to sensemodality . But the manner in which the various modalities behave - in possibly different ways- remains an issue . We with respect to the parameters briefly addressthis issuelater. But for the sake of simplicity , the first three levels of palpability presentednext are discussedonly for the visual modality . Our characterization of each level of palpability below will generally indicate its standing with . respectto each of the thirteen parameters

Fictive Motion in

: and " Ception"

249

6.9.2 ConcreteLevel of Palpability At the concrete level of palpability , an entity that one looks at is experiencedas fully manifest and palpable, as clear and vivid , with the ostensivecharacteristicsof precise form , texture, coloration , and movement, and with a precise location relative to oneselfand to its surroundings, where this precision largely involves a Euclidean-type geometry and is amenableto metric quantification . The entity is usually recognizable for its particular identity , and is regardedas an instance of substantivecontent. The - hence not as entity is experiencedas having real, physical, autonomous existence ' s own " on one of it . It is dependent cognizing accordingly experiencedas being out " ' there, that is, not as a construct in one s mind. The viewer can experiencethe entity with full consciousness and attention , has a sense of certainty about the existenceand the attributes of the entity, and feelsvolitionally able to direct his or her gazeover the entity, change position relative to it , or perhaps manipulate it to expose further attributes to inspection. Outside of abnormal psychologicalstates(such as the experiencing of vivid hallucinations), this concreteexperience of an entity requirescurrently on-line sensory stimulation - for example , in the visual case , one must be actually at the . In short one the at the , looking entity experiences entity high end of all thirteen . palpability related parameters Examplesof entities experiencedat the concretelevel of palpability include most of the manifest contents of our everydayvisual world , suchas an apple, or a street scene . With respectto general fictivity , a representationceived at the concrete level of palpability is generally experiencedas factive and veridical. It can function as the background foil against which a discrepant representationat a lower level of palpability is compared. 6.9.3 Semiconcrete Level of Palpability We can perhaps best begin this section by illustrating entities ceived at the semiconcrete level of palpability , before outlining their generalcharacteristics . A first example " at each intersection of a semiconcreteentity is the grayish region one " sees (except the one in direct focus) of a Hermann grid . This grid consistsof evenly spacedvertical and horizontal white strips against a black background and is itself seenat the fully concrete level of palpability . As one shifts one' s focus from one intersection to another, a spot appearsat the old locus and disappearsfrom the new one. Another example of a semiconcreteentity is an afterimage. For example , after staring at a colored figure, one ceivesa pale image of the figure in the complementarycolor when looking at a white field. Comparably, after a bright light has beenflashedon one spot of the retina, one ceives a medium grayish spot- an " artificial scotoma" - at the corresponding point of whatever scene one now looks at. An apparently further

250

LeonardTalmy

semiconcreteentity is the phogpheneeffect- a shifting pattern of light that spansthe visual field- which results from , say, pressureon the eyeball. In general , an entity ceived at the semiconcretelevel of palpability , by comparison with the fully concrete level, is experiencedas lesstangible and explicit , as lessclear, and as less intense or vivid . It has the quality of seemingsomewhat indefinite in its ostensivecharacteristics , perhaps hazy, translucent, or ghostlike. Although one has " the experienceof directly " seeing the entity , its lessconcrete properties may largely lead one to experiencethe entity as having no real physical existenceor , at least, to experiencedoubt about any such corporeality . Of the semiconcreteexamplescited " above, the grayish spots of the Hermann grid may be largely experiencedas out " there, though perhaps not to the fullest degree becauseof their appearanceand as one shifts one' s focus. The " out there" status is still lower or more disappearance dubious for afterimages , artificial scotomas , and phosphenesbecausethese entities . The Hermann grid spots are fully localizable move along with one' s eyemovements ceived only in with respectto the concretely ceived grid and, in fact, are themselves or relation to that grid . But an afterimage, artificial scotoma, phospheneimage ranks lower on the localizabilityparameter because , although each is fixed with respectto one' s visual field, it moves about freely relative to the concretely ceived external . The identifiability of a semiconcrete environment in pace with one' s eye movements in some , but the entity is otherwise afterimage cases entity is partially preserved largely not amenableto categorization as to identity . ' Generally, one may be fully consciousof and direct one s central attention to such entities as Hermann grid spots, afterimages semiconcrete , and phosphenes , , scotomas ' but one experiencesless than the fullest certainty about one s ception of them, and one can only exercisea still lower degreeof actionability over them, being able to ' manipulate them only by moving one s eyes about. The ception of Hermann grid spots requiresconcurrent on-line sensorystimulation in the form of viewing the grid . But , once initiated , the other cited semiconcreteentities can be ceived for a while without further stimulation , even with one' s eyesclosed. level of With respectto generalfictivity , a representationceivedat the semiconcrete and more fictive as a scene is on viewing generally experienced relatively palpability -level representationthat is usually being ceived at the lessveridical than the concrete sametime. The type of discrepancypresentbetweentwo such concurrent representations of a single scene is generally not that of fictive motion against factive stationariness, as mainly treated so far. Rather, it is one of fictive presence , as against of Hermann for that is the fictive factive absence , , , ; grid example representation as being , is assessed spots, of an afterimage, of an artificial scotoma, or of phosphenes present only in a relatively fictive manner, while the factive representation of the scenebeing viewed is taken more veridically as lacking any such entities.

Fictive Motionin Languageand " Ception"

251

6.9.4 SemiabstractLevel of Palpability An entity at the semiabstractlevel of palpability is experiencedas present inassociation with other entities that are seenat the fully concrete level, but it itself is intangible and nonmanifest, as well as vagueor indefinite and relatively faint . It has little or ' no ostension , one s experience , and with no quality of direct visibility . In viewing a scene " " " its " is that one does not see such an entity explicitly , but rather senses . In fact, we will adopt sensingas a technical term to refer to the implicit presence ception of an entity at the semiabstractlevel of palpability , while engagingin on-line " 12 viewing of something concrete. One experiencesan entity of this sort as out " there, perhaps localizable as a genuinely present characteristic of the concrete entities viewed, but not as having autonomous physical existence . Insofar as such a sensedentity is accorded an identity , it would be with respectto some approximate or vague category. A sensed or attention , seems less entity is of relatively low saliencein consciousness certain, and is difficult to act on. Often a sensedentity of the present sort is understood as a structural or relational characteristic of the concrete entities viewed. Its type of geometry is regularly topology-like and approximative. Such sensedstructures or relationshipscan often be captured for experiencingat the fully concretelevel schematic , such as line drawings or wire sculptures , but they lack by representations this degreeof explicitnessin their original condition of ception. Because the semiabstractlevel of palpability is perhaps the least familiar level, we presenta number of types and illustrations of it , characterizing the pattern of general fictivity that holds for three of thesetypes. General fictivity works in approximately the same way for all three types: object structure, referenceframes, and force dynamics . In order to characterize the general fictivity pattern for these three types " " together, we refer to them here collectively as structurality . The representation of structurality one sensesin an object or an array is generally experiencedas more fictive and lessveridical than the factive representationof the concreteentities whose structurality it is. The representation of structurality is a case of fictive presence rather than of fictive motion . This fictive presence contrasts with the factive absence . Unlike most forms of general of such structurality from the concrete representation the of concrete content and that of sensed structurality fictivity , representation that may seemso minimally discrepant with each other they are rather experienced as complementary or additive. (The type in section 6.9.4.4 involving structural .) history and future has its own fictivity pattern, which will be described separately Much of visually sensedstructure is similar to the structure represented by linguistic closed -class forms, and this parallelism will be discussed later in section 6.9.11.

252

Leonard Talmy

6.9.4.1 Sensingof Object Structure One main type of sensed entity is the structure we senseto be present in a single object or over an array of objects due to its -object case . To illustrate first for the single , when one views a arrangement in space certain kind of object such as a vase or a dumpster, one seesconcretely certain particulars of ostensionsuch as outline , delineation, color , texture, and shading. But in addition , one may sensein the object a structural pattern comprising an outer - in , an object of this sort is sensed portion and a hollow interior . More precisely terms of an idealized schematization as consisting of a plane curved in a way that definesa volume of spaceby forming a boundary around it . A structural schemaof this sort is generally sensedin the object in a form that is abstracted away from each " " of a number of other spatial factors. This envelope / interior structuring can thus be sensed equally acrossobjects that differ in magnitude, like a thimble and a volcano; of closure, like a beachball and a in shape , like a well and a trench; in completeness in and of bowl ; degree continuity /discontinuity, like a bell jar and a birdcage. punch - as appropriate to the semiabstractlevel of palpability This pattern of ception shows - that the type of geometry (parameter 9) here sensedin the structure of an object is topological or topology-like. In particular , object structure sensedas being of the -neutral, as well as being -interior type is magnitude-neutral and shape envelope neutral. and closure neutral discontinuity For a more complex example , on viewing a person, one seesat the fully concrete level of palpability that person' s outline and form , coloration and shading, textures, the delineations of the garments , and so on. However, one does not seebut rather senses the person' s bodily structure in its current configuration , for example , when in a squatting or leaning posture. A sensedstructural schemaof this sort can be made concretely visible, as when a stick figure drawing or a pipe cleanersculpture is shaped to correspond to such a posture. But one does not concretely seesuch a schemawhen . The Marrian abstractions (Marr its presence looking at the person- one only senses 1982 ) that representa human figure in terms of an arrangementof axesof elongation provide one theoretization of this sensedlevel of ception. A comparable sensingof structure can occur for an array of objects. For example , a person may ceive one object as located at a point or points of the interior spaceof another object that she sensesas having the envelope / interior structure described - what she above. The person may sensein this object complex a structural schema " " . may categorizeas the inside schema wherein the first object is inside the second -object case like a number of also exhibits this As in the single , topology object array . Thus not only can the first object and the second object themselves characteristics each vary in magnitude and shape , but in addition the first object can exhibit any orientation relative to the secondobject and can be located throughout any portion

" FictiveMotion in Language and " Ception

253

or amount of the secondobject' s interior space as manifesting , while still being sensed the " inside" schema . For a more intricate example , when one views the interior of a restaurant, one senses a hierarchically embeddedstructure in spacethat includes the schematicdelineations of the dining hall as the largest containing frame and the spatial pattern of tables and people situated within this frame. Perhapsone can seesome of the hall ' s framing delineations concretely, for example , some ceiling-wall edges ; but for the most part, the patterned arrangement in spaceseemsto be sensed . Thus, if one were to represent this sensedstructure of the scenein a schematic drawing, one might include some lines to representthe rectilinear frame of the hall , together with some. spots or circles for the tables and someshort bent lines for the people that mark their relative positions within the frame and to each other. However, though it can be so , this is an abstraction for the most part not concretely seenas such, but represented rather only sensed as present. Further casesperhaps also belong in this object structure type of sensing . Thus parts of objects not concretely seenbut known or assumedto be presentin particular locations may be sensedas present at those locations. This may apply to the part of an object being occluded by another object in front of it , or to the back or underside of an object not visible from a viewer' s current perspective . 6.9.4.2 Se18ingof Path Structure When one views an object moving with respect to other objects, one concretely sees the path it executes as having Euclidean specifics such as exact shapeand size. But in addition , one may sensean abstract structure in this path . The path itself would not be a caseof fictive motion , for the path is factive. But the path is sensedas instantiating a particular idealized path schema , and it is " this schemathat is fictive. Thus one may senseas equal instantiations of an " across ' schemaboth the path of an ant crawling from one side of one s palm to the opposite side and the path of a deer running from one side of a field to the opposite side. This " " visually sensed across schemawould then exhibit the topological property of being " schemain magnitude-neutral. Comparably, one may equally sensean " across the path of a deer running in a straight perpendicular line from one boundary of a field to the opposite boundary, and in the path ofa deer running from one side of the field " schema to the other along a zigzag slanting course . The visually sensed" across would then also exhibit the topological property of being shape -neutral. 6.9.4.3 Sensingof Reference Frames Perhapsrelated to the sensingof object/array structure is the sensing of a reference frame as present amid an array of objects. For example , in seeingthe sceneryabout oneselfat the concrete level, one can sense a grid of compass directions amid this scenery . One may even have a choice of

254

Talm Leonard }'

) . For alternative referenceframes to senseas present (as described in Talmy 1983 with a the toward at a church who is right facing looking , consider a person example based an earth scene manifest bicycle at its rear. That person can sensewithin this frame, in which the bike is west of the church. Or she can sensethe presenceof an object-basedframe, in which the bike is behind the church. Or shesensethe presence of a viewer-basedframe radiating out from herself, in which the bike is to the left of the church. Levinson ( 1996 ) have performed experiments on ) and Pederson( 1993 , with findings of strong linguistic-cultural biasing for the particular exactly this issue . frame sensedas present reference of type One may also sensethe presenceof one or another alternative referenceframe for the case of a moving object executing a path . Thus, on viewing a boat leaving an island and sailing an increasing distance from it , one can senseits path as a radius extending out from the island as center within the concentric circles of a radial reference frame. Alternatively , one can sensethe island as the origin point of a rectilinear ' reference frame and the boat s path as an abscissal line moving away from an ordinate. 6.9.4.4 Se18ingof Structural History and Future Another possible type of sensed phenomenon also pertains to the structure of an object or of an array of objects. Here, however, this structure is sensednot as statically present but rather as having shifted into its particular configuration from someother configuration . In effect, one a probable, default, or pseudohistory of activity that led to the present structure senses . A sensedhistory of this sort is the visual counterpart of the fictive site arrival paths described for language in section 6.8.3. The examples of visual counterparts already given in that section were of a figurine perceivedas a torso with head and es like limbs affixed to it ; of an irregular contour perceived as the result of process with a a circle as Man aPac and of and indentation figure perceived protuberation ; wedgeremoved. In addition to such relatively schematicentities, it can be proposed that one regularly not as static configurations certain complex forms within everydayscenes senses of deviation from someprior , result as the rather but own in their subsistent self right an on , equal-sided picture frame viewing generally more basic, state. For example that is hanging on the wall at an oblique angle, one may not ceivethe frame as a static diamond shape , but may rather senseit as a square manifesting the result of having beentilted away from a more basic vertical-horizontal orientation . Another example is the sensingof a dent in a fender not as a sui generiscurvature but as the result of a deformation. One sensesa set of clay shards not as an arrangement of separate distinctively shapedthree-dimensional objects but as the remains of a flowerpot that had beenbroken. One may even sensetoys that are lying over the floor not simply as

Fictive Motion in Languageand " C ,eption

255

comprising some specific spatial static pattern but rather as manifesting the result of having beenscatteredinto that configuration from a home location within a box. not only a history of its current configuration Viewing an entity may lead one to sense but also to sense a or , of changesaway from potential probable future succession its current configuration . Such a sensedfuture might involve the return of the entity to a basic state that it had left. For example , on viewing the previous picture frame hanging at an angle, one may senseits potential return to the true ( probably as part of imagining one' s manipulations to right it ) . In terms of generalfictivity , the sensingof an entity ' s structural history or future is a less veridical representation of fictive motion in a sensory modality . It is superimposed on the factively and veridically seenstatic representationof the entity . Thus, with respectto the picture frame example , the difference betweenthe factive and the fictive modes of ceiving the frame is the difference betweenseeinga static diamond and sensinga squarewith a past and a future. 6.9.4.5 Sel8ing of Projected Paths Another type of sensedception can be tenned projected paths. One fonn of path projection is based on motion already being exhibited by a Figure entity, for example , a thrown ball sailing in a curve through the air. A viewer observing the concretely occurrent path of the object can generallysense - but not palpably see - the path that it will subsequentlyfollow . Here we do not refer simply to unconsciouscognitive computations that , say, enable the viewer to move to the spot at which she could catch the ball ; rather, we refer to the conscious experiencea viewer often has of a compelling senseof the specific route that the . One may also project backward to sensethe path that the ball is object will traverse to have traversed before it was in view. Path projection of this sort is thus likely akin to the in the preceding wholly sensingof structural history and future discussed section. The main difference is that there the viewed entity was itself stationary, whereashere it is in motion . Accordingly , there the sensedchangesbefore and after the static configuration were largely associationsbased on one' s experienceof frequent occurrence , whereas here the sensedpath segmentsare projections mostly basedon one' s naive physicsapplied to the viewed motion . Another fonn of projected path pertains to the route that an agentive viewer will . It applies to a viewer, volitionally proceed to executethrough some region of space at one corner of a restaurant crowded with tables who wants to get to say, standing the opposite corner. Before starting out , such a viewer will often senseat the semiabstract level of palpability an approximate route curving through the midst of the tables that he could follow to reach his destination. The viewer might sense the shape of this path virtually as if it were taken by an aerial photograph . It may be that the initially projected route is inadequate to the task, and that the route-sensingprocess

256

Leonard Talmy

is regularly updated and reprojectedas the viewer movesalong his path . But throughout such a process , only the physical surroundings are seenconcretely, whereasthe . This form of projected path is akin to the fictive access is sensed paths follow to path 6.8.4. describedin section 6.9.4.6 Se18ingof Force Dynamics Also at the semiabstractlevel of palpability is the sensingof force interrelationships among otherwise concretely seenobjects. Included forces such as in such sensed force dynamicsare the interactions of opposing 's ' s intrinsic another rest or motion an object ; object opposition to tendency toward ; and the this tendency ; resistanceto such opposition ; the overcoming of resistance , or absenceof blockage. (SeeTalmy 1988bfor , disappearance , appearance presence an analysisof the semanticcomponent of languagethat pertains to force dynamics.) To illustrate , Rubin ( 1986 ) report that subjectsperceive ) and Engel and Rubin ( 1986 ) forces at the cusps when viewing a dot that moves along (in our terms, sense , then a path like a bouncing ball. When the bounce is progressively heightened . Complementarily, when the perception is that a force has been added at the cusps the ball ' s bounce is reduced , the force is perceived as being dissipated. Jepson and Richards ( 1993 ) also note that when a block is drawn with one face coplanar to and in the middle of the vertical face of a larger block , then the percept is as if the smaller " in the block is " attached or glued to the larger block , analogously to what is sensed " an of such is no But there wall. to a stuck attaching of an perception object viewing force" when the samesmall block is similarly positioned on the top face of the larger , block (i.e., when the original configuration is rotated 90 degrees ) . In this latter case an in be would as is attachment not contact viewing , expected , just , perceived only . object resting on a horizontal surface a scenein which a large concrete slab is consider For a less schematicexample , leaning at a 450 angle against the outer wall of a rickety wooden shed. A person viewing this scenewould probably not only seeat the concrete level the slab and the shedin their particular geometric relationship, but also would sensea force dynamic . This sensedforce structure might structure implicit throughout theseovert elements include a force (manifestedby the shed fully but tenuously resisting ) that is now success an unrelenting outside force impinging on it (manifestedby the slab), and that is capable of incrementally eroding and giving way at any moment. 6.9.4.7 SeI Wingof Visual Analoguesto Fictive Motion in Language Finally , the fictive motion types presentedbefore this section on ception can now be recalled for . Most of the visual patterns suggestedas their relevanceto the present discussion abstract level motion fictive of the types seemto fit at the semi linguistic counterparts in of terms . Further of palpability that is, they are sensed , general fictivity , these

" Fictive Motion in Languageand Ception

"

257

visual analogueshave involved the sensingof fictive motion; they do not involve the " " sensingof fictive presence(as was the casefor the representationsof structurality ) . As a summary, we can list here the fictive types from sections6.2 6.5 and just seen . Thus, we may senseat the semiabstract 6.8, all of which participate in this phenomenon level of palpability the fictive motion of the visual counterparts of orientation paths (including prospect paths, alignment paths, demonstrative paths, and targeting paths), radiation paths, shadow paths, sensory paths, pattern paths, frame-relative motion , advent paths, accesspaths, and coverage paths. With the addition of the casesof structural history/ future and projected paths characterized just above, this is a complete list of the fictive types proposed, in this chapter, to as fictive motion . have a visual representationsensed

6.9.5 Abstract Level of Palpability The casescited thus far for the first three levels of palpability have all dependedon , artificial concurrent on-line sensorystimulation (with the exception that afterimages scotomas ) . But we can adduce , and phosphenesrequire stimulation shortly beforehand a level still further down the palpability gradient, the (fully ) abstract level. At this level, one experiences conceptual or affective entities that do not require on line sensory stimulation for their occurrence and may have little direct relation to any such stimulation. Largely clustering near the lower endsof the remaining palpabilityrelatedparamete , and , such entities are thus largely impalpable, abstract, vague in to localization not amenable and characteristics in ostensive faint , , lacking perhaps hence as often are . , experienced subjective spaceor identification as to category They " " in oneselfrather than out there. They do seemto exhibit a range acrossthe remaining . Thus, they can range from full salienceto elusiveness palpability -related parameters or virtual inaccessibility to consciousness ; one can range from certainty to ' to from a and capacity manipulate them in one s mind to an puzzlementover them, experienceof being only a passivereceptor to them. Finally , they can exhibit either content or structure, and, insofar as they manifest a type of geometry, this, too , can exhibit a range, though perhaps tending toward the approximative and qualitative type. Such abstract entities may be ceived as components in the course of general ongoing thought and feeling. They might include not only the imagined counterparts of , the experience entities normally ceived as a result of on-line stimulation - for example -line while on sense otherwise would one the structure of in imagination only - but also phenomenathat cannot normally or viewing an object or array in space ever be directly ascribedas intrinsic attributes to entities ceivedas the result of on-line of sensorystimulation . Such phenomenamight include the following : the awareness 's the within one ; experience knowledge representation relationships among concepts

258

Leonard Talmy

of implications betweensetsof concepts ; assessments , and the formation of inferences of change occurring over the long term; experiencesof of veridicality ; assessments social influence (such as permissionsand requirements , expectationsand pressures ); " " a wide range of affective states ; and propositional attitudes (such as wish and intention ) . Many cognitive entities at the abstract level of palpability are the semantic referents of linguistic forms and thus can also be evoked in awarenessby hearing or thinking of those forms. These forms themselvesare fully concrete when heard, and of courselessconcretewhen imagined in thought , but the degreeof concreteness they do have tends to lend a measureof explicitnessto the conceptual and affectivephenomena associatedwith them. And with such greater explicitness may come greater . However, these to consciousness cognitive manipulability (actionability ) and access are phenomenathat , when experienced directly without associationwith such linguistic forms, may be at the fully abstract level of palpability . Despite such upscaling lent , it is easiestto give further examplesof ceptually abstract by linguistic representation the meanings of certain linguistic forms. Becauseopen-class phenomena by citing -classforms tend to forms tend to representmore contentful concepts , while closed , we next cite a number , more abstract- concepts representmore structural - and hence -classmeaningsso as to further convey the character of the fully abstract of closed .13 end of the palpability gradient, at least insofar as it is linguistically associated First , a schematicstructure one might otherwise senseat the semiabstractlevel of palpability through on-line sensorystimulation - as by looking at an object or scene - can also be ceived at the fully abstract, purely ideational level in the absenceof -classlinguistic form current sensory stimulation by hearing or thinking of a closed on For that refers to the sameschematicstructure. , viewing a scenein which example " " a log is straddling a road, one might sensethe presenceof a structural across " " . But one can also ceivethe same across schemaat the abstract schemain that scene level of palpability by hearing or thinking of the word across either alone or in a sentencelike The log lay acrossthe road. We can next identify a number of conceptual categories expressedby linguistic closed -class forms that are seemingly never directly produced by on-line sensory stimulation . Thus the conceptual category of tense , with such specific member concepts as past, present, and future , pertains to the time of occurrence of a referent . This category is well representedin the event relative to the presenttime of speaking languagesof the world but has seemingly scant homology in the forms of ception higher on the palpability scale that are evoked by current sensory stimulation . A secondlinguistically represented category can be termed reality status- a type largely included under the traditional linguistic term mood. For any event being referred to ,

Fictive Motion in Languageand " Ception"

259

this category would includesuchindications as that the eventis actual , conditional , or counterfactual , potential , andwouldalsoincludethesimplenegative e. . , ( g English not). Again, aspects of situations that arecurrentlyseen , heard , smelled , and soon at the concretelevel or sensed at the semiabstract level are seemingly not ceivedas havingany reality statusother than the actual . Similarly , the linguisticallyrepresented of modality category , with suchmember notionsasthose expressed by English can must and should , , or sensed , haslittle concrete . counterpart To continuethe exemplification , a further setof categories at the abstractlevelof -classformspertainto the palpabilitythat can beevokedby closed cognitivestateof some sentient entity; these , too, seem categories at the higherlevels of unrepresented 's . Thusa conceptual palpability that can be termedspeaker category status knowledge " , represented by linguisticforms called" evidentials the statusof , particularizes 's thespeaker of theeventthat sheis referringto. In a numberof knowledge languages (e.g., in Wintu, whereit is expressed inflections on theverb thiscategory by hassuch ), "" member notionsas : " known from personal as factual experience asfactual , accepted " " inferred " shared throughgenerally from , knowledge evidence accompanying , " inferredfrom " " entertained temporalregularity aspossible , because of havingbeen " , and "judged as probable reported ." Another linguisticcategoryof the cognitive 's ' s inference statetypecan be termedtheaddressee status . This is the speaker knowledge ' as to the addressee s ability to identify somereferentthe speaker is currently . Onecommonlinguisticform representing specifying this category is that of determiners - for example that mark definiteness , the Englishdefiniteand indefinitearticles theanda or an. Furthergrammatically areintentionand represented cognitivestates volition, purpose , desire . , wish , and regret For somefinal examples , a linguisticcategorythat can be termedparticularity to whether an entity in reference pertains is to beunderstood asunique( Thatbirdjust flew in), or asa particularoneout of a setof comparable entities(A birdjustjiew in), or generically as an exemplarstandingin for all comparable entities(A bird has . feathers ) But the on-line ceptionof an entity at the concrete or semiabstract level this rangeof options may not accommodate . In particular , it apparentlytendsto - for example exclude the generic case , looking at a particularbird doesnot tendto evokethe ceptionof all birds generically . Thus the ceptionof genericness in human cognitionmayoccuronly at theabstractlevelof palpability . Finally, manylinguistic -classforms specifya variety of abstract closed , suchas kinship and relationships ' . The Englishendings express possession esboth of these , asin Johns relationships 's book motherand John . Again, on-line ception , suchas viewingJohn in his house andMrs. Smithin hers , or viewingJohnin thedoorwayanda book on the table , may not directly evokethe relationalconcepts of kinship and possession the linguistic formsdo.14

260 6.10 FurtherTypesandProperties D of Ceptio

Leonard Talmy

The full structure of the entire system of ception certainly remains to becharacterized, but some brief notes here will sketch in a few lineaments of that structure. We

6.10.1

Imagistic Fonns of Ception

What can be termed imagistic forms of ception include mental imagery, whether related to vision or to other sensory modalities. Along the gradient parameter of stimulus dependence , imagistic ception seemsto fall in the midrange. That is, it can be evoked in association with an entity ceived at the concrete level during on-line stimulation by that entity . For example , on seeinga dog, one can imagine the sight and sound of it starting to bark , as well as the sight and kinesthesiaof one' s walking over and petting it . But imagistic ception can also occur without on-line stimulation , as during one' s private imaginings. It needs to be determined whether imagistic ception can also occur at the low end of the stimulus dependence parameter, that is, whether aspects of it are unrelated to sensory attributes, as in the case of many . conceptual categoriesof language 6.10.2 AssociativeFOrlDS of CeptioD What can be tenDedassociativeforms of ception pertain to ceptual phenomena evoked in associationwith an entity during on-line sensorystimulation by it , but not ascribed to that entity as intrinsic attributes of it . Such associatedphenomenacould include the following type: ( I ) mental imagery, asjust discussed ; (2) actions one might undertake in relation to the entity; (3) affective statesexperiencedwith respectto the ' with the entity; (4) particular conceptsor aspectsof one s knowledge one associates and (5) inferencesregarding the entity . entity ; mental imagery, we can here illustrate the remaining four Having already discussed of thesetypes of associativeception. As examplesof associatedaction (2), on viewing a tilted picture frame, one might experiencea motoric impulse to manipulate the frame so as to right it . Or , on viewing a bowling ball inexorably heading for the side " " gutter, one might experienceor executethe gyrations of body English as if to effect ' a correction in the ball s path . In fact, with respectto such kinesthetic effects , there be a of to what we have for may gradient palpability parallel posited ception that to motor control . from the least to the most applies Proceeding palpable, at the low ' end would be one s experienceof intending to move; in the midrange would be one' s

Fictive Motion in Languageand " Ception"

261

experienceof all -but-overt motion , including checked movement and covert body ' ' . English; and at the high end would be one s experienceof one s overt movements Associated affect (3) has such straightforward examplesas experiencingpleasure , disgust, or fear at the sight of something, e.g., of a child playing, of roadkill , or of a mugger. Associated knowledge or concepts(4) could include exampleslike thinking of danger on seeinga knife , or thinking of one' s childhood home on smelling fresh bread. And examplesof associatedinference(5) might be gathering that Mrs . Smith is John' s mother from the visual apparencyof their agesand of their resemblance , or ' inferring that a book on a table belongs to John from the surroundings and John s manner of behaving toward it . 6.10.3 Parameterof Intriaicality Associative forms of ception like thosejust adducedmay be largely judged to cluster near the semiabstractlevel of palpability . In fact, the phenomenadescribedin section " at the 6.9.4 as " sensed semiabstractlevel and the associativephenomenareported here may belong together as a single group ceived at the semiabstract level of palpability . But the sensedtype and the associativetype within this group would still differ from each other with respect to another gradient parameter, what might be termed intrinsica/ity . At the high end of this gradient, the sensedphenomenawould be experienced as intrinsic to the entity being ceivedat the concretelevel, that is, they would be ceived as actually present and perhaps inherent attributes- such as structure and patterns of force impingement- that the ceiver is " detecting" in the concretely seenentity . But at the lower end of the intrinsicality gradient, the associative phenomenapresentedhere would be experiencedas merely associatedwith the concretely ceivedentity , that is, they would be experiencedas incidental phenomenathe ceiver brings to the entity . This intrinsicalityparameter , however, is actually just the objectivity gradient (parameter 5) when applied to phenomenaconnected with an entity rather than to the entity itself. To be sure, where a particular phenomenon is placed along the in trinsicality gradient varies according to the type of phenomenon , the individual , the culture, and the occasion . For a classicalexample , if one ceivesbeauty in conjunction with seeinga particular person, one may experiencethis beauty as an intrinsic ' attribute of the person seen , much like the person s height, or , alternatively, as a personal interpretive responseby the beholder. 6.10.4 Diaociatio18 amongthe Palpability-Related Parameters While the thirteen palpability -related parameterslisted in section 6.9.1 generally tend to correlate with one another for the types of ception that had beenconsidered , some

262

Leonard Talmy

. For example dissociations can be observed , with respect to the imagistic forms of a can have visual mental fairly high degreeof ostension(parameter imagery ception, 4), for instance , having relatively definite form and movement. At the same time, level and the semiabstract however, it may rank somewherebetweenthe semiconcrete level along the palpability gradient (parameter I ) and at a comparably midrange level along the clarity gradient (parameter 2) . For another case of dissociation, already -classlinguistic forms are generally noted, the cognitive phenomenaexpressed by closed at the most abstract level of the palpability gradient (parameter 1) . But the conscious manipulability of the linguistic forms expressingtheseconceptual phenomena ranks them near the high end of the actionability gradient (parameter 12 ) . Or the low on most of states rank some affective parameters for may quite again, , intangible on the palpability gradient (parameter 1), murky on the clarity example gradient (parameter 2), and nonostensiveon the ostension gradient (parameter 4) while ranking quite high on the strength gradient (parameter 3) becausethey are experiencedas intenseand vivid . The observation of further dissociationsof this sort of the parametersadducedand ultimately justify their can argue for the independence . identification as distinct phenomena 6.10.5 ModaHty Differencesalong the Palpability Gradient In the discussion on ception, we have mostly dealt with phenomena related to the visual modality , which can exhibit all levels along the palpability gradient except perhaps the most abstract. But we can briefly note that each sensory modality may have its own pattern of manifestation along the various palpability -related ' , the kinesthetic modality , including one s sense parameters adduced. For example of one' s current body posture and movements , may by its nature seldom or never rank very high along the palpability , clarity , and ostensiongradient (parameters I , 2, and 4), perhaps hovering somewhere between the semiconcrete and the semiabstract level. The modality of smell, at least for humans, seemsto rank low with respect to the localizability gradient (parameter 6) . And the modalities of taste and smell, as engagedin the ingestion of food , may range more over the content region than over the structure region of the content/ structure gradient (parameter 8) . Comparison of the sensorymodalities with respectto ception requires much further investigation.
6.11 Content / Structure Parallelisms between Vision and Language

The analysis to this point permits the observation

of two further

between

. visionand language

" and " Ception FictiveMotion in Language

263

in Vision 6.11.1 ComplementaryFunctionsof the Content and Structure Subsystems and Language First , both cognitive systems , have a content subsystemand a , vision and language line vision for structure subsystem . Within on , , in the viewing of an object example or array of objects, the content subsystem is foremost at the concrete level of palpability , while the structure subsystemis foremost at the semiabstract level of , the referents of open-class forms largely manifest the content palpability . In language -class forms are generally limited to while the referents of closed , subsystem . The two subsystemsserve largely distinct and manifesting the structure subsystem complementary functions, as will be demonstrated next, first for vision and then for . A number of properties from both the content/ structure gradient (parameter language 8) and the type-of -geometry gradient (parameter 9) align differentially with the . Included are properties pertaining to distinctive functioning of thesetwo subsystems as bulk as against lineaments , Euclidean geometry as against topology, absoluteness and a substantive as , , , , holistically against relativity precision against approximation summary as against a unifying frameworkS We can first illustrate the properties and operations of the two subsystemsin vision. For a caseinvolving motor planning and control , as in executing a particular , the content subsystemis relevant for fine-grained local calibrations path through space while the structure , subsystemcan project an overall rough-and-ready first , a person wanting to cross the approximation . Thus, to revisit an earlier example dining area of a restaurant will likely plot an approximate, qualitative coursecurving semiabstractlevel of structure in a spatial array . through the tables, using the sensed But in the processof crossing, the person will attend to the Euclidean particulars of the tables, using the concretelevel of specificbulk content, so as not to bump into the tables' corners. If such were possible , a person operating without the overall topol ogy-like subsystemwould be reduced to inching along, using the guidelines of the precision subsystemto follow the sides of the tables and the curves of the chairs, without an overarching schematic map for guidance. On the other hand, a person lacking the precision subsystemmight set forth on an approximate journey but encounter repeated bumps and blockages for not being able to gauge accurately and negotiate the local particulars. The two subsystemsthus perform complementary functions and are both necessaryfor optimal navigation, as well as other forms of motor activity . . To do this, we can at work in language We can next illustrate the two subsystems -class observethe distinct functions servedby the open-classforms and by the closed A rustler lassoedthe steers . . Thus, consider the sentence forms in any single sentence This sentencecontains just three open-class forms, each of which specifiesa rich complex of conceptual content. These are the verb rustle, which specifiesnotions of

264

Leonard Talmy

, which specifiesa illegality , theft , property ownership, and livestock; the verb lasso that is in knotted a and swung around, cast, and particular configuration rope looped ' steer, which specifies the noun and in a certain s head an animal circled over way; for human consumption notions of a particular animal type, the institution of breeding , and castration. -class fonDS that On the other hand, the sentencecontains a number of closed . These include the function a specify relatively spare concepts serving structuring event of the current suffix ed specifying occurrencebefore the time ; the suffixs speech " " , specifying multiple instantiation , and the zero suffix (on rustler), specifying ' unitary instantiation ; the article the, specifying the speakers assumption of ready , and the article a, specifying the opposite of this; the identifiability for the addressee suffixer , specifying the performer of an action; the grammatical category of noun ) indicating ), indicating an object and that of verb (for lassoed (for rustler and steers a process ; and the grammatical relation of subject, indicating an Agent, and that of direct object, indicating a Patient. The distinct functions servedby thesetwo types of fonDScan be put into relief by , while keeping the other alternately changing one type of form in the above sentence -classforms, as in a sentencelike Will constant. Thus we can changeonly the closed the lassoersrustle a steer? Here, all the structural delineations of the depicted scene and of the speechevent have been altered, but becausethe content-specifying openclassforms . But we can now are the same , we are still in a Western cowboy landscape . Here, the change only the open class forms, as in A machinestampedthe envelopes structural relationships of the sceneand of the speechevent are the same as in the , but with the content-specifying forms altered, we are now transposed original sentence to an office building . In sum, then, in the referential and discoursecontext of a contribute the majority of the content, sentence , the open-classfonDSof the sentence -classforms determine the majority of the structure. whereasthe closed Thus, both in ceiving and motorically negotiating a visual sceneand in cognizing of content and of structure the referenceof a sentence , the two cognitive subsystems and complementary functions as they are in operation, performing equally necessary interact with each other. in Vision and in Language 6.11.2 ComparableCharacter of the Structure Subsystem The structural subsystemsin vision and in language exhibit great similarity . First , recall that section 6.9.4 on ception at the semiabstractlevel of palpability proposed that we can sensethe spatial and force-related structure of an object or an array of that any structure of this sort is sensedas objects when viewing it . It was suggested consisting of an idealized abstractedschemawith a topology-like or other qualitative , the precedingsection has shown that the type of geometry. With respectto language

" Fictive Motionin Language and" Ception

265

-classforms is dedicated to specifying the structure of the whole or systemof closed some part of a conceptual complex in reference . We can now point out that when such linguistically specified structure pertains to space or force, it , too , consists of idealized abstracted schemaswith topology -like properties. In fact, the character of the structuring yielded by visual sensingand that yielded by the linguistic closed -class system appear to be highly similar. If we can heuristically hypothesize that some particular neural systemis responsiblefor) ' rocessingschematicstructure in general , then we can supposethat both visual sensingand linguistic closed -classrepresentation are connected with , or " tap into ," that single neural system for this common characteristic of their mode of functioning . The structure subsystems of vision and languageexhibit a further parallel. Recall the observation in section 6.9.4 that the structural schemas one semiabstractlysenses to be presentin an object or array are assessed as being fictive, relative to the factive status of the way one concretely sees the object or array . Now , the structural schemas -class forms- here, specifically, those pertaining to expressedby linguistic closed , relative to the factive character of spaceand force are also fictive representations the objects and arrays that a languageuser understandsthem to pertain to. That is, - whether sensed all thesecasesof abstracted or conceptually imposed schemas visually or specified by linguistic closed -class forms can be understood as a form of - here, the fictive fictivity . They constitute not fictive motion but fictive presence presenceof structure. Accordingly, the extensivebody of linguistic work on spatial schemas(e.g., Talmy 1975 , 1983and Herskovits 1986 , 1994 , among much else ) constitutes a major contribution to fictivity theory . In particular, Herskovits has made it a cornerstone of her work to treat the spatial schemasshe describes as " virtual " which are to be distinguished structures" (previously called " geometric conceptualizations ), " " '' from the canonic representations of objects as they are." Ifwe can now extend the hypothesisof a neural systemresponsiblefor processingschematicstructure , we can add that the products of its processinghave ascribedto them the character of being fictive, relative to the products of other neural systemsfor processingthe concrete ostensionsof ceived entities. Proceedingnow to demonstrationsof similarity , we consider severalparallel vision. With respectto the structure of an array of objects, it was proposed languagecases in section 6.9.4.1 that one can visually sensethe presenceof an " inside" type of structural schema on viewing a two -object complex in which one object is sensed as located at a point or points of the interior space defined by the other object. This schemacan be topologically or qualitatively abstracted away from particulars of the objects' size , shape , state of closure, discontinuity , relative orientation , and relative location. Now , the spatial schema specified by the English preposition in -classform can thus be usedwith equal exhibits all thesesameproperties. This closed

266

Leonard Talmy

, in a well, appropriatenessto refer to someobject as located in a thimble, in a volcano in a . Further in a trench, in a beachball in in a or , apunchbowl, , it can belllar , birdcage of both be said that in abstracting or imposing their schema , the structure subsystems vision and languageproduce a fictive representation , relative to the concreta of the object array . the topology -like properties of the structure Comparably, section 6.9.4.2 addressed in the of a viewed sensed path moving object. But this type of visually sensedstructure also has linguistic closed -classparallels. Thus the English preposition across which specifiesa schemaprototypically involving motion from one parallel line to another along a perpendicular line betweenthem- exhibits the topological property of being magnitude-neutral. This is evident from the fact that it can be applied both to paths of a few centimeters , as in Theant crawledacrossmy palm, as well as to paths of thousands of miles, as in The bus drove across the country. In a related way, the ) a schemainvolving motion preposition through specifies(in one sector of its usage a But a line located within medium. , topology-like , this schema is shape along neutral; thus through can be applied equally as well to a looped path, as in I circled . And , , as to a jagged path, as in I zig-zaggedthrough the woods through the woods in the schemas thus sensed or , visually linguistically imputed to a again topological are relative to the Euclidean fictive path representations particulars seenor believed . to be present For a final case that , on viewing certain scenes , section 6.9.4.3 suggested , one may sense the presence of either a rectilinear or a radial referenceframe as the background can also be against which an object executesa path . But thesetwo alternate schemas -classforms. Thus English awayfrom indicates motion from a representedby closed -type axis within a point on an ordinate-type boundary progressingalong an abscissa rectilinear grid . But out from indicates motion from a central point along a radius within a radial grid of concentric circles. These alternative conceptual schematiza in like: The boat and out tions can be seen sentences drifted further further away/ from the isle, or Thesloth crawled 10feet away/outfrom the tree trunk along a branch. Here, both referenceframes are clearly fictive cognitive impositions upon the scene , whether this sceneis viewed visually or referred to linguistically . 6.11.3 Stnlctural Explicitnessin Vision and Language The cognitive system pertaining to vision in humans has another feature that may have a partial counterpart in language . It has a component for representing in an form the kinds of schematic structures explicit generally sensedonly implicitly at the semi abstract level of palpability . We here call this the component for " schematic ." pictorial representation

" Fictive Motionin Language and" Ception

267

In iconographic representation , a full -blown pictorial depiction manifests the content . But the structure subsystem can be made explicit through the component subsystem of schematicpictorial representation by schematicdepictions involving the use of points, lines, and planes , as in both static and filmic cartoons and caricatures , line , and the like. The very first pictorial depictions children drawings, wire sculptures " " produce- their stick figure drawings- are of this schematickind . For example ,a child might draw a human figure at an early phaseas a circle with four lines radiating from it , and later as a circle atop a vertical line from which two lines extend laterally right and left at a midpoint and two more lines slope downward from the bottom point . Thus, in depicting an object or sceneviewed, a child representsnot so much its -level characteristics as the structure that he or she can sensein it at the concrete semiabstractlevel of palpability . It must be emphasizedthat such schematizationsare not what impinges on one' s retinas. What impinges on one' s retinas are the particularities of ostension : the bulk , , textures, shadings , colorings, and so on of an entity looked at. Yet what edges ' emergesfrom the child s hand movementsare not such particulars of ostension , but rather one-dimensional lines forming a structural schematic delineation. Accordingly , much cognitive processinghas to occur between the responsesof the retinas " and these hand motions. This processingin a principled fashion reduces , or boils " down, bulk into delineations. As proposed in this chapter, such structural abstractions are in any casenecessary for the ception of visual form , both of single objects and of object arrays (cf. Marr 1982 at ); they constitute a major part of what is sensed the semiabstract level of palpability . It then appearsthat the component of the visual systeminvolved in producing external depictions taps specifically into this sameabstractional , a mechanism already in place for other functionsstructuring system where this mechanismmay be the sameas the earlier heuristically hypothesizedneural . In fact, in the developmentally earliest systemfor schematicstructure in general ' phaseof operation, a child s iconographic capacity would appear to be linked mainly to this structuring mechanism , more so than to the cognitive systemsfor concretely ceiving the full ostensionof objects. The component of languagethat may partially correspond to this representational -classsystemitself, as characterizedin the precedingsection. explicitnessis the closed The linguistic linkage of overt morphemes to the structural schemasthey represent lends some concretenessto those cognitive entities, otherwise located at the fully abstract level of palpability . Thesemorphemesconstitute tangible counterparts to the abstract forms, permit increasedactionability upon them, and perhapsafford greater consciousaccess to them. The form of such morphemes , however, does not reflect the form of the schemas in , and this way, this languagecomponent differs they represent

268

Leonard Talmy

crucially from the pictorial schematicrepresentations , which do correspond in structure to what they represent . Although this section has pointed to content-structure parallelisms betweenvision and language . It may be expectedthat the structure , it remains to chart their differences in vision and languagediffer in various respectsas to what they treat subsystems as structural , their degreeand type of geometric abstraction, the degreeand types of variation such structural featurescan exhibit acrossdifferent cultural groups, and the times and sequences in which thesestructural featuresappear in the developingchild . 6.11.4 SomeCompariso . . with Other Approach es The present analysis raises a challenge to the conclusions of Cooper and Schacter " " " " ( 1992 ) . They posit explicit and implicit forms of visual perception of objects' apparently the concepts in the literature closest to this chapter s concepts of the concreteand semiabstractlevelsof palpability . But they claim that their implicit form of perception is inaccessibleto consciousness . We would claim instead, first , that entities such as structural representations sensedat the semiabstract level of palpability at (like those treated in section 6.9.4) can in fact be experiencedin awareness least at a vague or faint degreeof clarity , rather than being wholly inaccessibleto - both largely amenable consciousness . And , second , the fact that vision and language to consciouscontrol - can generally render the structural representationsof the structure subsystemexplicit suggests that theserepresentationswere not in access ibly in the first implicit place. Separatecognitive systemsfor representingobjects and spaceshave been posited ' by Nadel and O Keefe ( 1978 ), by Ungerleider and Mishkin ( 1982 ), and by Landau " what" and the " where" and Jackendoff ( 1993 who characterized them as the ), systems . To be sure, these systemsfit well, respectively , into the content and structure " " subsystems posited in Talmy ( 1978a , 1988a ) and here. However, the where system would seemto comprise only a part of the structure subsystembecausethe former pertains to the structural representationof an extendedobject array - the field with - whereasthe latter respectto which the location of a figure object is characterized also includes the structural representationof any single object.

6.12 Relation of Metaphor to Fictivity


Metaphor theory, in particular as expoundedby Lakoffand Johnson ( 1980 ), accords readily with general fictivity . The source domain and the target domain of a metaphor . The representationof an entity within supply the two discrepant representations the target domain is understood as factive and more veridical. The representation from the sourcedomain that is mapped onto the entity in the target domain, on the other hand, is understood as fictive and lessveridical.

" Fictive Motionin Language and" Ception

269

For example often involve spaceas a sourcedomain mapped , linguistic expressions onto time as a target domain. This can be seen in sentenceslike The ordeal still " lies aheadof us, and Christmas is coming , where the static spatial relation of frontality " is " " mapped onto the temporal relation of subsequence , while the dynamic " " " ." In terms spatial relation of approach is mapped onto temporal succession of general fictivity , factive temporality is here expressedliterally in terms of fictive spatiality . One observation arising from the fictivity perspective , perhaps not noted before, is that any of the Lakoff and Johnson' s ( 1980 ) three term formulas- for example , " Love is a " " " " " journey , Argument is war, Seeingis touching - is actually a cover term for a pair of complementary formulas, one of them factive and the other fictive, as representedin (27) . (27) Fictive: X is Y Factive: X is not Y Thus, factively, love is not a journey , while in some fictive expressions , love is a . The characteristic that renders an expressionmetaphoric what metajourney very phoricity dependson is that speakersor hearershave somewherewithin their cognition a belief about the target domain contrary to their cognitive representation of what is being stated about it , and have somewherein their cognition an understanding of the discrepancybetweenthesetwo representations . One reason for choosing to adopt fictivity theory over metaphor theory as an umbrella aegis is that it is constructed to encompasscognitive systemsin general rather than just to apply to language . Consider, for example , a subject viewing a round and narrow-gapped C-like figure. In terms of general fictivity , the subject will . Concurrently likely seea C at the concrete level of palpability - its factive representation for the same figure, she will sensea complete circle at the semiabstract level of palpability - its fictive representation . She will experiencethe former representation as more veridical and the latter one as less so, and may experience a . This , then, is the way that degreeof discrepancy between the two representations the framework of general fictivity would characterize the Gestalt phenomenon of closure. As for the framework of linguistic metaphor, if its terms were to be extended to cover vision, they might characterize the perception of the C figure as involving the mapping of a sourcedomain of continuity onto a target domain of discontinuity, so that the subject experiencesa visual metaphor of continuity . An extension of this sort should indeed be assayed . But at present , both psychologistsand linguists balk at the notion of closure as a metaphor. Meanwhile, the outline of a might generalframework for addressingsuch phenomenaacrosscognitive systemsis here in place.

270 6.13 Cognitive BiastowardDynamism

Leonard Talmy

As we have noted above, phenomenaother than motion - notably , stationariness can have fictive status in both languageand vision; fictive stationarinesshas already been seen in frame-relative motion . In the examples given, when the scenery is , the observer is fictively treated as fictively treated as moving toward the observer In . addition certain formulations treat motion as if it were static. , stationary linguistic For example, instead of saying J went around the tree, which explicitly refers to my , progressiveforward motion , I can say My path wasa circle with the tree at its center which confines the fact of motion to the noun path and presentsthe remainder of the event as a static configuration . Visual counterparts of fictive stationarinesscan be found in viewing such phenomena as a waterfall or the static pattern of ripples at a particular location along a flowing stream. Here one ceivesa relatively constant configuration while all the physical material that constitutes the configuration constantly changes , that is, the physical material is factively moving, while the fictive pattern that it forms is stationary. This situation is the reverseof the pattern paths of section 6.8.1. There the physical substance was for the most part factively stationary, while the fictive pattern that it formed moved. We can now compare the relative occurrence of fictive motion and fictive stationariness in language and, perhaps also, in vision. In terms of metaphor theory, fictive motion in languagecan be interpreted as the mapping of motion as a source domain onto stationarinessas a target domain. A mapping of this sort can be seenas a form of cognitive " dynamism." Fictive stationariness : the map, then, is the reverse ping of stationarinessas a sourcedomain onto motion as a target domain. This sort of mapping, in turn , can be understood as a form of cognitive " staticism." Given this framework , it can be observed that , in language , fictive motion occurs preponderantly more than fictive stationariness . That is, linguistic expressions manifesting fictive motion far outnumber onesmanifesting fictive stationariness . In other words, linguistic exhibits a strong bias toward conceptualdynamism as against staticism. expression The cognitive bias toward dynamism in languageshowsup not only in the fact that . In stationary phenomenaare fictively representedas motion more than the reverse addition , stationary phenomenaconsideredby themselves can in somecases be represented . The factive fictively as motion even more than factively as stationariness representation of a stationary referent directly as stationary is what Talmy ( 1988a ) calls the " synoptic perspectivalmode" ; in a related way, it is what Linde and Labov " " " " ( 1975 ) call a map and what Tversky (chapter 12, this volume) calls the survey form of representation . This is illustrated in (28a) . Correspondingly, its fictive representation in terms of motion exemplifies Talmy ' s " sequential perspectival mode,"

" Fictive Motion in Languageand" Ception

271

and, comparably, what both Linde and Labov and Tversky call the " tour " form of , as illustrated in (28b) . representation (28) a. There are some housesin the valley. b. There is a houseevery now and then through the valley. While this example allows both modes of representation , other examples virtually a static preclude , permit ting only a representation in terms of fictive representation motion for colloquial usage , as seenin (29). ' (29) a. ' rrhe wells depths form a gradient that correlates with their locations on the road. b. The wells get deeperthe further down the road they are. In a similar way, factively static phenomenain cognitive systemsother than language may also be more readily cognizedin fictively dynamic terms than in static terms. For , in vision, on viewing a picture hanging on a wall at an angle, a person may example more readily ceivethe picture as a squarethat has beentilted out of true and calls for righting , whereashe may require a special effort to ceive the picture statically as a diamond. Comparably, in the cognitive systemof reasoning es , one usually progress through a proof step by step rather than seeingthe full complement of logical relationships all at once. In fact, cognitive dynamism is so much more the normal mode that the cognizing of staticism is often regardedas a specialand valued achievement . Thus an individual who suddenly ceives all the components of a conceptual domain as concurrently copresentin a static pattern of interrelationships is said to have an " aha" experience , while an individual who ceivesa successionof one consequentevent after another through time as a simultaneous static pattern of relationships is sometimesthought to have had a visionary experience . AckD Dts Owledgme I am gratefulto Lynn Cooper Palmer , AnnetteHerskovits , Kean Kaufmann , Stephen , and for muchvaluable . And my thanksto KarenEmmorey discussion for corroborative Mary Peterson data on fictive motion in AmericanSignLanguage could not be , which unfortunately included in thepresent version of this chapterfor lack of space . Notes 1. This chapteris plannedas the first installmenton a more extensive treatmentof all the fictivecategories . 2. Bucher andPalmer canprevailover ( 1985 ) haveshownthat, whenin conflict , configuration motionasa basis for ascription of " front" status . Thus , if an equilateral trianglemoves along

272

Leonard Talmy

one of its axes of symmetry, then that line is seen as defining the front -back. Whether the ' triangle s vertex leadsalong the line of motion or trails , the line is still seenas the front . Where the vertex trails , the triangle is simply seenas moving backward. 3. Note that the notion of crossing behind a front -bearing object may be partially acceptable , possibly due to a conceptualization like this: the posited intangible line, though more salient in front , actually extendsfully along the front -back axis of the object. 4. Due to the constraint noted above, this construction cannot refer to nonaligned fictive , * The snakeis lying past the light cannot refer to a snake lying straight with paths, for example its head pointing past the light . Still needing explanation, however , is why this construction " cannot also be used for aligned arrangementswith path geometriesother than " toward or " " * away from , as in Thesnakeis lying int% ut of the mouth of the caveto refer to a snakelying straight with its head pointing into or out of a cave mouth. 5. Probably poorer as models are such other forms of agency as an Agent' s affecting some cognitive state that she herself has or somephysical object that she is already in contact with . 6. This mapping may be reinforced by the fact that the prospect path ascribedto an inanimate with an actual viewer located configuration , such as a cliff wall or a window , is often associated at that configuration and directing her or his visual path along the samepath as the prospect line. Thus, in (i ), one readily imagines a viewer standing at the cliff edge or in the bedroom looking out along the samepath as is associatedwith the cliff wall or the window. / looks out toward the butte. (i ) a. The cliff wall faces b. The bedroom window faces / looks out /opens out toward the butte/ onto the patio . 7. Colllparisons of language structure to the structure in visual perception appear in Talmy , 1983 , 1988a , and this chapter) and in Jackendoff ( 1987 ) . Comparisons of language ( 1978 structure to the structure of the reasoning systemappear in Talmy ( 1988a ); to the structure of kinesthetic perception, in Talmy ( 1988b , in ); to the structure of the cognitive culture system . And the most in 1995a this and to the attentional 1995 and , Talmy ( ) system chapter); Talmy ( extensiveidentification and analysis to date of the foundational structural properties common " to all the cognitive systemsappears in the " Parameters section of Talmy . In this work , the with reference to a putative cognitive subsystemunderlying the analysis is presentedprimarily structure of narrative, but the analysis is intended to be quite general across the range of . cognitive systems " 8. To note the correspondences , Jackendoff ( 1983 ) has abstracted a concept of pure di " " " " " rectedness with four particularizations : actual motion , extension (e.g., The road goes " " arrow from New York to L .A .), corresponding to our coveragepaths, orientation (e.g., The " e. . " and end location town to our demonstrative to toward the , ( g, ), corresponding paths points / The houseis over the hill ), corresponding to our access paths. 9. The term and perhaps the basic concept of ception derive from a short unpublished paper " " by StephenPalmer and Eleanor Rosch titled Ception : Per- and Con- . But the structuring of the ception concept found here, as well as the parametersnext posited to extend through it , belong to the present approach. Already in common usage are other terms that are neutral to any perception- conception . Such distinction , though perhaps without much recognition of conferring that advantage

Fictive Motion in Languageand " Ception"

273

tenns include representation , experience , cognize , and sometimescognition. All thesetenns have their particular applications and will be used in this chapter, but the new tenn ception is specifically intended to emphasizethe continuity acrossthe larger domain and the existenceof largely gradient parametersthat span it . 10. Perhaps alone out of the thirteen, the parameter of strength has an open-ended upper of intensity. Thus the point along this parameter region, allowing increasingly greater degrees that would tend to correlate with the high ends of the other parameters should be located within its upper region. II . The parameterof objectivity , like the others, is intended as a phenomenologicalparameter . An entity is assignedto the high end of this gradient becauseit is experienced as being " out there," not becauseit fits a category of a theoretical ontology according to whose tenets the " " entity is out there. Insofar as it is concluded in our scientific ontology that an entity is in fact located external to one' s body, note further the following . Once stimuli from the entity impinge on the body' s , the neural processing of the stimuli , including the portion that leads to sensory receptors consciousexperiencingof the entity, never again leavesthe body. Despite this fact, we experience the entity as external. We lack any direct consciousexperiencethat our processingof the entity is itself internal. In physiological tenns, we apparently lack brain-internal senseorgans or other neural mechanismsthat register the interior location of the processingand that transmit that infonnation to the neural consciousness . On the contrary, the processingis system ' specifically organized to generate the experienceof the entity s situatednessat a particular external location. 12. The adoption of the verb to senseas a tenn for this purpose is derived from its everyday , not from any other usesthis word may have beenput to in the psychological colloquial usage literature. 13. As treated extensivelyin Talmy ( 1988a ), open-classfonns are categoriesoffonns that are . , consisting primarily of the roots of nouns, verbs, and adjectives large and easily augmented Closed-classfonns are categoriesof fonns that are relatively small and difficult to augment. Included among them are bound fonns like inflectional and derivational affixes ; free fonns like prepositions, conjunctions, and detenniners; abstract fonns like grammatical categories(e.g., " nounhood" and " verbhood" per se ), grammatical relations (e.g., subject and direct object), and word order patterns; and complexes like grammatical constructions and syntactic structures. 14. Linguistic categories like the preceding have been presented only to help illustrate the abstract end of the palpability parameter, not becausethat parameter is relevant to general . It should be recalled that the palpability gradient has here been introduced fictivity in language to mainly help characterizegeneralfictivity in vision. Though linguistic referencecan be located along it , this parameter is not suitable for characterizing generalfictivity for language . As discussed , general fictivity in languageinvolves the discrepancybetweenthe representation ' s literal reference of one' s belief about a referent situation and the representationof a sentence . -related representationsinto the visual modality does tend The mapping of two such language to involve a palpability contrast, but the original two representationsdo not.

274

Leonard Talmy

15. Talmy ( 1978a , 1988a ) first observed the homology between vision and language as to a content/ structure distinction . These papers also present an expanded form of the linguistic demonstration synopsizedin the text below. References Babcock, M ., and Freyd, ) . Perception of dynamic infonnation in static handwritten ~ J. ( 1988 - 130 . . American fonDs Journalof Psychology , 101 , 111

and : Natural ontologies on cultural representations , P. ( 1994 ). Cognitiveconstraints Boyer .), Mappingthe mind : Domain . In L. A. Hirschfeldand S. A. Gelman(Eds religiousideas . New York: Cambridge . in cognition andculture UniversityPress specificity Bucher , N. M., and PalmerS. E. ( 1985 ). Effectsof motion on the perceived pointing of - 236 . . Perception andPsychophysics , 38, 227 ambiguous triangles . Cambridge . in childhood , MA : MIT Press , S. ( 1985 ). Conceptual Carey change between structuraland episodic , L. A., and Schacter , D. L. ( 1992 ). Dissociations Cooper - 146 in Psychological . . Current Directions Science of visualobjects , 1(5), 141 representations . In Proceedings visualmotion boundaries , S. A., and Rubin , J. M. ( 1986 ). Detecting of Engel on Motion: Representation andAnalysis IEEE the Workshop , Charleston , , ComputerSociety SC , 7- 9 May. . Cambridge Fodor, J. A. ( 1983 : An essay onfaculty psychology , MA.: ). Modularityof mind MIT Press . . Cognitive momentum , 19(3), , J. ( 1987 ). Explorationsof representational Psychology Freyd - 401 . 369 andspatialcognition : An interdisciplinary Herskovits , A. ( 1986 studyof the prepositions ). Language . in English . Cambridge : Cambridge Press University " and " " " Across . and the interface between Herskovits , A. ( 1994 ) along : Lexicalorganization . Unpublished . andspatialcognition manuscript language . . Cambridge Jackendoff andcognition , MA: MIT Press , R. ( 1983 ). Semantics . : The relation of linguisticand visual information Jackendoff , R. ( 1987 ). On beyondzebra - 114 . , 26 , 89 Cognition -TR-93 -43 . ? Technical , A., and Richards , W. ( 1993 ). Whatis a Percept report RBCV Jepson . Toronto: Universityof Toronto Department of ComputerScience . . Cambridge Keil, F. ( 1989 , kinds , andcognitive , MA: MIT Press ). Concepts development : Universityof Chicago welive by. Chicago Lakoff, G., and Johnson , M. ( 1980 ). Metaphors . Press " " " " and spatial Landau , B., and Jackendoff , R. ( 1993 ). What and where in spatiallanguage . . Behavioral andBrainSciences , 16 (2), 217 238 cognition . . Stanford : StanfordUniversityPress , R. ( 1987 ). Foundations of cognitive grammar Langacker

" Fictive Motion in Languageand Ceptinn

275

Levinson, S. ( 1996 ) . Relativity in spatial conception and description. In J. J. Gumperz and . S. C. Levinson (Eds.), Rethinking linguistic relativity . Cambridge: Cambridge University Press . ). Symmetry, causality, mind. Cambridge, MA : MIT Press Leyton , M . ( 1992 Linde , C., and Labov , W . ( 1975 ) . Spatial networks as a site for the study of language and 51 924 . 939. , , thought Language Marr , D . ( 1982 ) . Vision: A computationalinvestigationinto the human representationand processing of visual information. San Francisco: Freeman. Matsumoto , Y . (in prep.) . Subjective motion and English and Japaneseverbs. Cognitive Linguistics. Nadel, L ., and O' Keefe, J. ( 1978 ). The hippocampusas a cognitive map. Oxford : Clarendon . Press

Palmer in configurations of , S. E. ( 1980 ). What makes triangles point: Localandglobaleffects - 905 . Cognitive . , 12 , 285 ambiguous triangles Psychology Palmer in perceived effects , S. E., andBucher , N. M. ( 1981 ). Configural pointingof ambiguous . Journalof Experimental : HumanPerception andPerformance . , 7, 88- 114 triangles Psychology Pederson and manipulable in two Tamil linguisticsystems . In , E. ( 1993 ). Geographic space A. U. Frank and I. Campari(Eds .), Spatialinformation : Springer . , Berlin theory Pentland andthe representation of naturalform. Artificial , A. ( 1986 ). Perceptual organization - 331 28 293 . , Intelligence , Rubin of visual motion. PhiD. diss ., Massachusetts Institute of , J. M. ( 1986 ). Categories . Technology and syntax of motion. In J. P. Kimball (Ed.), Syntax and , L. ( 1975 ). Semantics Talmy - 238 . New York: Academic semantics Press . , vol. 4, 181 causative . In Syntaxand semantics . Vol. 6, M. Shibatani , L. ( 1976 ). Semantic Talmy types . NewYork: Academic constructions Press . , 43- 116 (Ed.), Thegrammar of causative : A synopsis . In D. Waltz (Ed.), , L. ( 1978a ). The relation of grammarto cognition Talmy TIN LAP 2 Issues in Natural : University Proceedings ( Theoretical ). Urbana of Language Processing of Illinois. . . In Universals sentences , L. ( 1978b ). Figureandgroundin complex Talmy of human language - 649 . Stanford . Vol. 4, J. H. Greenberg , 625 , CA: StanfordUniversityPress (Ed.), Syntax . In H. L. Pick, Jr., andLP . Acredolo(Eds .), structures , L. ( 1983 ). How language Talmy space - 282 : Theory . NewYork: Plenum . Press , research , andapplication , 225 Spatialorientation : Semantic structurein lexicalforms . In Language , L. ( 1985 Talmy ). Lexicalization patterns . Vol. 3, T. Shopen and the typologyand syntacticdescription (Ed.), Grammatical categories - 149 lexicon . Cambridge : Cambridge . , 57 UniversityPress -Ostyn(Ed.), Topics to cognition . In B. Rudzka in , L. ( 1988a ). Therelationof grammar Talmy . . 165 205 Amsterdam : , cognitive linguistics Benjamins in language andcognition . Cognitive Science . , L. ( 1988b , 12 , 49- 100 ). Forcedynamics Talmy

276

Leonard Talmy

) . Fictive motion and change in language and cognition . Plenary addressat Talmy, L . ( 1990 . Conferenceof the International PragmaticsAssociation, Barcelona. July, 1990 . . Monist , 78( 1), 81- 116 ) . The cognitive culture system Talmy, L . ( 1995 . In M . Shibatani and S. Thompson ) . The windowing of attention in language Talmy , L . ( 1995a . : Theirform and meaning , Oxford : Oxford University Press (Eds.), Grammatical constructions ) . Narrative structure in a cognitive framework. In G. Bruder, J. Duchan, and Talmy , L . ( 1995b L . Hewitt (Eds.), Deixis in narrative: A cognitive science , 421- 460. Hillsdale, NJ: perspective Erlbaum. . In D . J. Ingle, M . A . ) . Two cortical visual systems Ungerleider, L . G ., and Mishkin , M . ( 1982 Goodale, and R. H . W. Mansfield (Eds.), Analysis of visual behavior , Cambridge, MA : MIT . Press

7 Chapter
-

The Spatial Prepositionsin English, Vector Grammar, and the Cognitive Map Theory JohnO' Keefe

7.1

Introduction

In this chapter I wish to return to a subject that Lynn Nadel and I first addressedin our book The Hippocampusas a Cognitive Map ( 1978 ) nearly two decadesago. The gist of the argument presentedthere was as follows. Evidence from animal experiments proves strong evidencethat the hippocampus, a cortical area in the mammalian forebrain , is involved in the construction of an allocentric spatial representation " " of the environment, what Tolman ( 1948 ) called a cognitive map. Constructed and modified during exploration (a cognitive behavior), this map provides the animal with a representationcenteredon the environment and locatesit within that environment . During the initial exploration of an environment and subsequently , placesof use for future stored locations label and their in the and ; interest are labeled map to direct as and used theselocations can subsequentlybe retrieved into the map goals behavior. For example , if a satiated animal notices food in a location during its initial an environment of , it can on a subsequentoccasionuse that information exploration . need a to satisfy hunger Upon finding itself in the sameenvironment it can retrieve the location of the food and useit to direct its behavior toward that location. This theory can account for a substantial part of the experimental literature on the infrahuman hippocampus. In order to extend the theory to account for the human data, however, we neededto extend it in two ways. First , we had to incorporate a temporal senseinto the basic map to account for the ability of humans to process and store spatiotemporal, or episodic , we had to allow for the , information . Second demonstrated in the been has that of function lateralization repeatedly impressive that while much of the right human brain. Neuropsychological studieshad suggested " " cerebral hemisphereis specializedfor visuospatial processing , the left side has been given over to languagefunction . Following her dramatic demonstration with Scoville of a memory function for structures in the mesial temporal lobe (Scoville and Milner 1957 ), Milner showed that this memory function respectedthe generallateralization

278

John O' Keefe

of function: patients with damage restricted to the right mesial temporal lobe were amnesicfor visuospatial material, whereasthose with left -sideddamagewere amnesic for linguistic material. Evidence gathered since has strengthened this conclusion , 1989 ; Frisk and Milner 1990 ). (Smith and Milner 1981 that this lateralization of function might be due primarily to Nadel and I suggested differencesbetweenthe inputs to the hippocampal map on the two sidesof the human brain and not necessarilyto any fundamental differencesin principles of operation. The right human hippocampus would receive inputs about objects and stimuli derived from the sensoryanalyzersof the neocortex and attributable to inputs from the external physical world . It would operate in the same way as both right and left . In contrast, the left human hippocampus would receive infrahuman hippocampuses a new set of inputs, which would come primarily from the language centers of the neocortex and would consist of the names of objects and features and not of their " " sensoryattributes. In addition , this semanticmap would incorporate linear temporal would serveas the deepestlevel of the linguistic information and in consequence . narrative the basis for , providing comprehensionand narrative memory. system However, languageis clearly not reducible to the set of spatial sentences ; therefore we sought to create a more general framework by following the work of Gruber , 1976 ), who noted the similarity in structure between ( 1965 ) and Jackendoff ( 1976 " " went from New York to Los Angeles sentences such as " The message , The book " " The rock went from smooth to pitted ," " The went from Mary to the library , " librarian went from laughing to crying . They proposed that the parallels in surface structure reflected parallels in underlying meaning, in this case the substitution of for the positional sense , and a circumstantial sense , identificational sense possessional senseof the prototype . Nadel and I interpreted this to mean it might be possible to envisagenonphysical spacesthat located items, not according to their physicalloca . We suggestedone tion , but according to their location in these other dimensions suchdimension might be that of influencebut did not develop this notion any further . In this chapter I would like to develop further this idea of the semanticmap. In the years that have intervened since the first publication of the idea, we have learned a great deal about the working of the infrahuman cognitive map at the physiological level, and there are now severalcomputational models available. I intend to explore the adequacyof one of thesein particular (O ' Keefe 1990 ) as the basisfor a semantic
map . Before returning to the semantic map idea , it will be helpful if I elaborate some of the details of the basic theory as developed for physical space. In the cognitive map theory , entities are located by their spatial relationships to each other . Spatial relationships are specified in terms of three variables : places, directions , and distances (figure 7.1) . Places are patches of an environment that can vary in size and shape

The Spatial Prepositions

279

ELEMENTS FORA MAP

-AB

""z (~ ::::: =:: . ~~ ~ :::::::)


MAP = PLACES ABC L AC I Aci L CB I CBI DIRECTIONS LAB DISTANCES I ABI

Figure7.1 and the distances and directions Cognitivemapsconsistof a set of placerepresentations between them . Distances and di~ tions can be represented by v~ tors drawn from oneplace to another . In animalssuch as the rat, they are computedin real time on the basisof actualmovements in highermammals autonomous , whereas from actual they may become . movements n of features in that depending on the size of the environment and the distributi < ? environment. They are located in terms of the spatial relations among the invariant features of the environment; they can also be located by their direction and distance from other places . The place codeis carried by the pattern of firing of the place cells in the cortical region called the " hippo campus ." Directions are specifiedas a set of parallel vectors. As with places , thesecan be identified in one of severalways: either as the local gradient of a universal signal such as gravity, geomagnetism , or olfactory currents, as the vector originating at a place or object and passing through another place or object (or passing through two places ), or as having a specifiedangle to a

280

John O' Keefe

previously identified direction (e.g., through updating the current direction on the basisof angular head movements ) . For every direction there is an opposite direction , which can be marked by the negativeof that vector. The direction codeis carried by the pattern of firing of the head direction cells in the postsubiculum (see , , for example Taube, Muller , and Ranck 1990 ), another cortical region that neighbors on the hip pocampal region and is anatomically connected to it . Distances between objects or placesare given by a metric. The basic unit of this metric might be derived from : either there is a reafferencesignal from the motor systemwhich one of two sources estimates the distance that a given behavior should translate the animal or use is . made of environmental or interoceptive inputs which result from such movements An exampleof an environmental input would be a changein retinal location of visual stimuli , and an example of an interoceptive input would be a vestibular signal. In either case , the geodesicdistancebetweentwo objects or placesneedsto be computed , gating the metric signals arising from such sources by the headby, for example direction signals so that only movements when the animal is heading in the same direction are integrated. A path is an ordered sequence of placesand the translation vectors betweenthem. Paths can be identified by their end placesor by a distinct name. Conversely , places along the path can be identified and associatedwith the path . A path may be marked by a continuous feature such as an odor trail or a road but neednot be. Within this spatial framework, translations of position in an environment are spe whose tail at the of movement and whose cified as translation vectors begins origin head ends at the destination. Vector addition and subtraction allow journeys with one or more subgoalsto be representedand integrated. Furthermore, on a journey with more than one destination the optimal or minimal path can be calculated. It is still not clear whether the spatial coordinate framework is a rectilinear or a polar one and whether the metric is Euclidean or otherwise. In recent papers , I have explored Euclidean polar models (O' Keefe 1988 , 1990 , 1991 ). If the cognitive map theory is on the right track in its contention that the left human hippocampus is basically a spatial mapping systemthat has beenmodified to store linguistic as opposedto physical information , then it might be possible to learn , something about the structure of the systemby analyzing the way it representsspace in revived within case . A tradition , grammar linguistics recently linguistically long theory, postulatesthat the deepsemanticstructure of languageis intrinsically spatial and that other, nonspatial, propositions are in some way parasitical on theseprototypical formulas, perhaps by means of metaphorical extension of their core spatial . This is the contention of a group of linguists called " location ists" or meanings " localists" Anderson 1971 Bennett 1975 . Theselocalist theories seeCook 1989for ; ) ( ( consistsin a verb and its a recent review) suggestthat the basis for spatial sentences

The Spatial Prepositions

281

. Typical casesmight be agent, object, and locative, identifying the associatedcases initiator of the action, the thing acted on, and the place or places of the action, . In an uninflected languagesuch as English, many of the spatial relations respectively described in spatial sentencesare conveyed by the prepositions. As Landau and Jackendoff ( 1993 ) have pointed out in their recent article, there are only a limited If this be the case then it is possible that a description of the . these number of representationsset up by the spatial prepositions might provide the basis for a more general linguistics. Nadel and I speculatedthat the origin of language might have been the need to transmit information about the spatial layout of an area from one ' that at some , 4O1n ) . This view suggests person to another (O Keefe and Nadel 1978 the basic to elaborate hominids cognitive map by began point in their evolution substituting sounds for the elementsin the map or for someof the sensoryaspectsof . These maps were probably primarily transmitted as drawings in the theseelements sand or dirt with different icons standing for different environmental objects. In this way one group of a family could forage a patchy environment and report back the locations of foods to the rest of the family . Different grunts would enrich the detailed information in the map and might serve the additional purpose of acting as an encrypting . Over time, an increasein vocabulary would eventually obviate the device need for the externalized map entirely, but the neural substrate would retain the structure of the original mapping function . In this chapter I will set out the basic framework of vector grammar and show how it accounts for many of the spatial meaningsof the spatial prepositions. My thesisis that the primary role of the prepositions is to provide the spatial relationships among a set of places and objects and to specify movements and transformations in these relationships over time; these spatial relationships and their modifications can be representedby vectors. The location of an entity within this notation is given by a vector that consistsof a direction and a distance from a known location. Much of the work of the locative prepositions involves the identification of these two variables. In some cases(for example, with vertical prepositions; seebelow), the direction is given by an environmental , however, it needs to be signal such as the force of gravity . In most cases more objects or places or two between , calculated from the spatial relationships or vector of the head the and ) which specify the origin and termination (or the tail , ; in most cases a point along the vector. By contrast, distancesare lesswell specified the metric is an interval one. One of the roles of the preposition for is to supply the metric information . The spacecoded by the locative prepositions is a mixed necessary polar rectilinear one. In this chapter I will assume(following the location ists; seeabove) that the prepositions , temporal) senseas in English have a spatial (or in one or two instances

John O' Keefe

their basicmeaningand that the other meanings are derivedby metaphor . I will concentrate on the locativeprepositions and in particular thosedealingwith the verticaldimension otherswill alsobecovered . I will thenextend theanalysis , although to showhow the temporalprepositions codefor a fourth dimension , which differs in stateor locationcanbe , and how changes only slightlyfrom the threespatialones codedby the translationalmeanings of the same . If time can be coded prepositions a fourth dimension is it to other , by possible incorporate nonspatial relationships by axes aswell? As a preliminaryexplorationof this question , I will higherdimensional conclude with a discussion of the metaphorical usesof the verticalstativeprepositions to represent the nonphysical relationsof statusandcontrol. in this chapteris to setout the premise that a vectornotation My primaryconcern cancapturemanyof thebasicmeanings of the spatialprepositions in English . Consequently in anydetailthe role of syntaxin this kind of grammar . In , I will not address will consist of a setof rulesfor relatingthespatial , thesyntaxof sucha system general structureof the deepsemanticnarrativeto the temporalstructureof the surface informationtransmission . Thus motor programmer , just asthereis an associated system that translates informationfrom the spatialmap into instructions to the motor sothat the animalcanapproach desirable planningsystems places containing objects or avoid oneswith undesirable , so also there is a production systemfor objects sentences from the map narrative . The syntacticrules specify generating , among otherthings of the narrativeareto be readand how , the orderin whichtheelements the differentpartsof the vectorsystem areto be translated into surface elements asa functionof thewaythat theyareread . For example the difference between the active ,
and the passivevoice in the surfacesentence dependson the direction of travel along the underlying vector (head to tailor vice versa ) relating an agent and its actions. 7.2 Physical Spatial Meaningsof the Vertical Prepositions In this section I shall analyzethe spatial meaningsof four related prepositions: be/ow, down, under , and beneath(or underneath ) . Although thesehave antonyms (above , up, over, and on top of>, I shall refer to theselatter only when they contribute something extra to the discussion . The four prepositions have in common that they denote 1 spatial relationships betweenentities in one linear dimension, which I shall call the " Z -dimension." They differ from each other in interesting ways that will allow us to explore the properties of the spacethey depict. 7.2.1 Below Let us begin with what I believeto be the most basic of the four prepositions, be/ow. On my reading, be/ow relatestwo entities (A and B) in terms of their relative location along the Z -direction. Consider the simple deictic sentence

The Spatial Prepositions

283

( 1) John is below. Becausebe/ow is a bipolar preposition, there must be a second suppressedterm, which I shall argue is the place occupied by the speakeror the listener. John or his ' ' place is A , the speakers (or listener s) place is B, and the relationship betweenthem is as follows: the magnitude of the component of A ' s place in the - Z -direction is ' greater than the magnitude of B s place. In order to make the assertion in ( 1), or to assess its validity , we needa notation for specifying the Z -direction , a way of locating A and B along that direction , and means for assessing whether A or B has a larger component along that direction. The most convenient notation for accomplishing theseis vector algebra. In this notation a direction is designatedby a set of parallel vectors of unlimited magnitude and unspecifiedmetric. The location of each entity is specifiedby a vector drawn from an observer to the entity . This vector can be specifiedby a magnitude R and an angle; with the direction vector through the point observer (figure 7.2) . The component of the vector A along the Z -axis can be computed by calculating the inner product of A and Z given by the formula : Az = A cos; , where A is the magnitude of A and ; is the angle that A makes with the Z -direction vector at observer(obs) . In the deictic example of sentence I , A is be/ow the observer if Az < obs, and abovethe observerif Az > obs. The sameconsiderationsallow the observerto decide whether A is below B when neither is located at the observer (figure 7.3 shows this situation ). Again , the question of whether A is below or above B can be assessed by comparing their relative magnitudesalong the Z -axis. If Az - Bz > 0, A above B ; If Az - Bz < 0, A below B. Note that the relationship betweenA and B is perfectly symmetrical and that neither A nor B can be considereda referenceentity in the deep structural representationof the relationship. Choice of one or other as the referent in the surface sentencemay , which of the two , the previous sentences depend more on the topic of the discussion entities has already been located, which is easier to locate perceptually, and other . The be/ow relationship is a transitive one. By simple transitivity of considerations arithmetical relations on the Z -dimension, if Az > Bz and Bz > Dz, . . . Az > Dz .

284

John O' Keefe

0 .~ tk ~ Q ,- .
A BELOW Observer
. Figure 7.2 Vector location and the below relation. The location of an entity A can be representedby a vector drawn from the observerto that entity . The vector is characterizedby a distance R and an angle ; measuredwith respect to a direction Z . The projection of the vector onto the Z direction is shown as Az . Observer

" r '"",A
.

B,C,D Below A Fiaure7.3 Eachitem A, B, C, and D hasa projectiononto the Z -axis . The relativelengths of the projection onto this axis determine which itemsare belowwhich . In the example , Band C have identicalprojections andaretherefore both equallybelowA.

The Spatial Prepositions

285

In figures 7.2 and 7.3, I choseto represententities A - D in an allocentric framework ; that is, I assumedthat they existed in an environmental framework independent of the location of the observersand that their relationship within the framework could be assessed . Further , I assumedthat independently of the locations of the observers the distancesfrom each observerto the entities was known or could be computed, for , by movementparallax. Does this imply that the spatial relationship denoted example by be/ow can be computed only within an allocentric framework? Can we say anything about the constraints on frameworks that can be used ? In general , the use of be/ow relies on the availability of a direction vector shared between the speaker and listener; in the case of the allocentric framework, this is provided by the universal gravity signal. There are, however, other, more limited uses of be/ow that employ egocentric and object-centereddirectional vectors. Egocentric vectors are fixed to the body surface of the observer , and object-centeredvectors are fixed to the entity or entities related. Sentences 2- 5 are examples . 2 The new ( ) planet appearedbelow the moon. (3) Below this line on the page. (4) Hitting below the belt. (5) The label below the neck of the bottle. The egocentric use occurs under circumstances(a) where the entities are very far away from the observerand therefore do not changerelative locations with observer location ; or (b) where the entities are constrained to lie on the XZ plane, as on a page or a video display unit . In the fonner case , the conversantsmust ensurethat they are similarly aligned to each other relative to the entities or that there is a conventional orientation relative to the gravity signal that enables the Z -direction to be labeled conventionally . This is most obvious with the specializedcase of the parts of the human body, which are probably labeled by referenceto their canonical orientation relative to gravity (see Levelt, chapter 3, this volume) . The case of the bottle and similar manufactured objects that refer to body parts (back of a chair , leg of a stool) would seem to follow the same rule. In general then nonnal , conversation would , seemto require the use of an allocentric framework for most purposes , for the reasons pointed out by other contributors to the present volume ( Levelt, chapter 3; Levinson, chapter 4) . Even the ability to see things from another' s point of view would appear to involve computations basedon an underlying knowledge of the two ' observers locations in allocentric space . A secondconclusion can be drawn about the underlying framework on the basisof our discussionof be/ow. Where it is used to describean allocentric relationship, the framework cannot be a simple polar coordinate system , but must have at least one

286

John O' Keefe

rectilinear axis. This follows from the simple observation that in a polar coordinate system the below relation cannot be specified by one variable alone, but requires two variables: a distance and an angle (seefigure 7.2) . It follows therefore that the most parsimonious theory would specify the Z -direction by a single dimension in all . As we shall see , this does not necessarilyimply that the other two (non-Z ) usages dimensionsare also rectilinear. We have, then, evidencefor a single dimension along which entities can be located. Can we say anything more about the metric at this stage , and if so, how are distances . Roughly, in of the differ ? Scales2 this dimension type metric employed specifiedalong the to or measurements this describesthe relationship of the observations systemof real numbers. The usual categoriesof scalesare the nominal, ordinal , interval , ratio , and absolute; they differ in the number of properties of the real number systemthey . This is most easily characterizedby the types of transformations that can be respect applied to the assignedvalues without transforming the relationship of the scale to . Nominal scalesare simple classification scalesin which the labels the thing measured . For the purposesof the scaling, the elementswithin stand for the namesof classes each classare consideredequivalent and different from all the elementsin the other . No other relationship among the elementsis implied , and only transforms classes are allowed. Clearly, the below relationship equivalent to the relabeling of the classes . Ordinal scales consist of a series of numbers such that satisfies a nominal scale observations equal to each other are assignedthe samenumber and an observation larger than another is assigneda larger number, but no significanceis attached to the interval betweenthe numbers. The relationship betweennumbersis transitive because m > nand n > pimplies m > p , and all mathematical transformations that maintain . Becauseit is the monotonic ordering of the numerical assignmentsare permissible are dealing A we below C B below A and C B below that to , implies possible say in addition , that scales . Interval scales are ordinal , with at least an ordinal scale provide information about the differences between the scale values. In particular , are equal to eachother. For example , m - n = p - q. they assertthat somedifferences Transformations that preservethe differencesbetweenvalues as well as their ordering . Specifically are permissible , the valuesof one scalecan be multiplied by a positive to relationships. constant and added to another constant without consequence Z2 = aZ . + b, a > 0 In this linear transform , a changesthe gain of the metric, and b the origin . It would appear that the be/ow directional scalecomesclose to fulfilling the requirements for . One way of testing this is to ask whether it is possible to apply the an interval scale comparative operator more to the preposition and thus to derive equivalent intervals . The question is whether the comparative notion is an intrinsic part of of be/owness

The Spatial Prepositions

287

the meaning of below or merely an extension of it . I would argue that becauseit is always legitimate to ask for the relationships set out in (8), the scaleis an interval one. Indeed, it may not be possible to compute the vector calculations suggestedin this . chapter on material ordered on lessthan an interval scale (6) A and B are below C. (nominal) (7) A is more below C than B. (ordinal ) (8) A is as far below Bas C is below D . (interval) Compare theseto (6a) A and B are brighter than C. (7a) . A is more brighter compared to B than to C. (8a) . A is more brighter than B by the sameamount as C is more brighter than D. Ratio scalesare interval scalesthat do not have an arbitrary origin . Here the only permissibletransform is the gain of the scale . Z2 = aZl , a > O In absolutescales , the final category we shall consider, no transfers are allowed and the underlying assumption is that the real number system uniquely maps onto the observations

Z2= Zl .

As we have seenalready, the metric of the be/ow relationship is at least ordinal , and probably interval. But is it ratio? Here the fact that the be/ow relationship can be assessed from any arbitrary observation point and can use any origin suggests that it does not rely on a fixed origin but is invariant under arbitrary translations. Furthermore , it is intuitively obvious that changesin scale do not affect the relationship either. These suggestthat it falls short of a ratio scale . It can, however, be elevated into a ratio or even an absolute scaleby the provision of explicit metric information . (9) a. A is twice as far below B as Cis . b. A is three feet below the surface. 7.2.2 Do H '" (and Up) The locative meaning of down is related to that of be/ow in that it specifiesthe direction of the entity as lying in the - Z -direction. In addition , however, it requires a line or surface that is not orthogonal to the Z -direction and on which the entity is located. This line or surface is the object of the preposition down. As with be/ow, the directional component of down is relative to another entity, which in this case is

288

O' Keefe John

from identifiesthe from. In generalthe preposition governedby the preposition is not information . If this , it is or tail of a directionvector explicitly source supplied . is the deicticlocationhere that the referent assumed is down thehill (from here ). ( 10 ) The house ( 11 ) Justdownthe treefrom Samwasa largetiger. . ( 12 ) *The boat wasdownthe ocean

: a planeor line that I shallcall the " reference entities Thustherearetwo reference " " " entity. As long as the plane and a placeor objectthat I shallcall the reference to the Z axis reference extended ), it ) asin ( 12 entity is not horizontal(perpendicular this reference . surface dimensional a two line or dimensional , canbea one Intuitively functionof Z over the decreasing entity shouldbe a linearor at leastmonotonically 's of the person on the other side of the hill , regardless . Someone relevantrange , a localminimumon relative- Z-coordinate , is not downthehill from you. Similarly the reference and located the hill between the of theslope entitydisruptstheuse entity down can . To put it anotherway, the preposition of down only takeasdirectobjects in the nonhorizontal entitiesthat haveor canbe treatedashavingmonotonicslopes down the more to our , we find, as we . preposition comparative plane Applying of the the Z on to is sense its that did with below component operate , primitive . relationship ) down thehill than Jill. ( 13 ) Johnis more(farther John and Jill are both locatedon the hill , the hill has a projection onto the Z the dimension , and John hasa larger - Z than Jill. Thereis no interactionbetween . This can be tested of the preposition of the reference planeand the sense steepness more Who is 7.4 in three of the the ) down (farther people figure by asking question or Jim? the hill from Jill? John of downis that neitherJohnnor Jim is moredown from of the meaning My sense . However areirrelevant , the Jill thanthe other, indicatingthat the non-Z -dimensions that either or surface line a from Z the to extract suggests sloping component ability or that non Z Z and into two thesecan be decomposed ) ( orthogonalcomponents our of the basis then on It seems . onto the Z axiscanbecomputed , , their projections coordinate , that we are dealingwith at least a two dimensional analysisof down more dimensions one or other the and is vertical , one dimension in which system down between difference the above direction below with the As to this. , / orthogonal . of signand thereareno obviousasymmetries and its antonymupis merelya change axis would the Z scale of If A is downfrom B, thenB is upfrom A. Themeasurement of a true 0 or the absence of evidence is clear and there one interval to be an appear

The Spatial Prepositions

Jill

289

Farther Down the Hill

Figure7.4 Downmeasures in the Z-direction the relationship . JohnandJim areequallyfar downthe hill from Jill, despite differentlateraldisplacements . origin (this is relative to the referencepoint identified byfrom ), and therefore the scale is not a ratio one. The scale of the other two dimensions is not clear from the two prepositions below and down becausethe use of the comparative operator more in conjunction with theseonly operateson the Z - component of the meaning. Evidence about theseother dimensionscan, however, be garneredfrom an analysisof the third of our prepositions, under. 7.2.3 UlUler Under is similar to down and be/ow in that it also codes for the spatial relationship betweentwo entities in the Z -direction. In addition , however, it placesrestrictions on the location of theseentities in one or two directions orthogonal to the Z -direction. If B is under A , then it must have a more negative value in the Z -dimension. In addition , however, it must have one or more locations in common in at least one orthogonal dimension (let us call them X and Y for the moment without prejudice to the question of the best representationof relationships in this plane) . The projection of the entity onto the X -direction is determined in the same way as that onto the Z -direction by calculating the inner product of the vector drawn to the entity from . Figure 7.5 shows this relationship for three pointlike objects. The relation an observer depicted is conveyedby the sentences ( 14 ) C is under A but not under B ; B is not under A . When one or more of the entities is extended in one or more of the non-Z directions, the under relationship can be assessed by the samealgorithm . For example , if the entities are extendedin the XY plane, then an overlap in any location in the

290
X Direction
. .

John O' Keefe

Bx

Direction

0 B II Bz C
-

Cx Ax

C under A - but not under8 Figare 7.5 Under representsa spatial relationship in the XY -plane as well as the Z -direction. C is under A the Bx and because it has the sameX -length and a greater - Z -length. C is not underB because Cx lengths differ.

. Note that unlike be/ow, under is not transitive when applied to XY -plane suffices in the XY -plane. B under A and C under B does not mean extended entities that are that C is underA . Another interesting differencebetweenunder , on the one hand, and down and be/ow, on the other, ariseswhen we examine the locus of operation of the comparator more. Recall that when applied to be/ow and down, more acted to increase the length of the Z - componentof the vector to the entity . When applied to under , the effect of the comparator is not fixed but dependson the relative dimensionsof the two entities. Let us leave aside for the moment the small number of usagesthat seemto mean that there is no intervening entity betweenthe two relata: . ( 15 ) Under the canopy of the heavens ( 16 ) Under the widening sky. . In The comparator cannot be applied to theseusages , which I shall designateunder1 the more frequent usageof under, the comparator is more often found to operate on the orthogonal X -dimensionthan on the primary Z -dimension. Compare the following : two sentences

The Spatial Prepositions

291
IE

~ A
B more under than A

Figure7.6 StickB isfarther (more thetablethanstickA because thereis a greater ) under lengthof overlap with theprojectiononto the XY-plane . . ( 17 ) The wreck was farther under the water than expected . ( 18 ) The box was farther under the table than expected , ( 17 Ignoring the metonymic usesof table and water, it is clear that the first usage ), a or Z dimension , while the second , ( 18 implies greater depth ), implies a greater , which I shall designateunder2 length in the X -dimension. In the first usage , under acts as a synonym for be/ow, and the substitution can usually be made transparently. These usagesmay be confined to situations in which the upper entity is very long relative to the lower one and completely overlaps with it . It follows that any change in the lateral location of the lower one will not affect the amount of overlap, and there is no information contained in the preposition about the lateral variable. In contrast, where both relata have a limited extension in the XY -plane, under2is responsiveto thesedimensions . We can use this fact to explore the properties of the second and third dimensions of spatial language and the relations between these and the Z dimension. Consider sentence( 19 ) and related figure 7.6: ( 19 ) Stick A was under the table, but stick B was even farther under it . I read sentence( 19 ) to mean that both sticks A and B and the table (top) have onto the XY -plane and theseprojections overlap, that is, have locations projections in common. Further , the magnitude of some aspect of the projection of B onto the table is greater than that of A . In general , this magnitude will be a length along some vector (e.g., Y in figure 7.6) measuredfrom the edgeof the table to the farthest edge

292

John O'Keefe

of the object projection . Furthermore, any differencesin the projections of the objects in the Z -direction are irrelevant. Thus (20) Box A was farther below the shelf than box B and farther under it . Applying the comparative test to the preposition under revealsthat the metric is the sameas that for the - Z -direction , that is, an interval scale . (21) Chair A was as far under the table as chair B. Note that this sentence can be used even when the chairs are at right anglesto each other, in which caseeach distance is measuredfrom the edgeof the table intersected are on an interval by the chair. The sentencealso confirms that both measurements scaleand that the samemetric applies to each. This conclusion is strengthenedby the fact that it makes sense to say (22) Chair A was as far under the table as it was below it . This last sentencealso suggeststhat the meaning of under2 in the XY -plane is a distance and not an area. Evidence for this can be gained by imagining the sameor different objects of different projection sizesand exploring the meaning of (23) A farther under than B, as theseobjects are positioned in different ways undera constant-sizetable (seefigure 7.7) . Figure 7.7 shows that the judgment of which objects are more under (or more under2 ) does not depend on the relative proportion of the length that intersectswith the referenceobject (B more under than A ); the orientation of the objects need not necessarilybe the samebecausethe relevant length is taken from the intersection of the object with the edgeof the table or from the nearestedge( C is asfar underas B) . My claim that A more under2refers to the absolute length of A might appear to be contradicted by sentences such as (24) Mary got more under the umbrella than Jane and thus got lesswet. This clearly implies that Mary got more of herself (i.e., a greater proportion ) under " " " " the umbrella. In this usage , however, it is clear that more modifies Mary rather than " under," and does not constitute a refutation of the presentproposal. that when an object has two Finally , D more under2than C in figure 7.7 suggests dimensions either of which could be taken into consideration, the distance under2is taken from the longer length. It is interesting to note that , unlike the antonyms up . (for down ) and above(for be/ow), over does not show complete symmetry with under2 In somesubtle sense , the table is lessover the chair than the chair is under2the table. This slight asymmetry appears not to relate so much to size as to relative mobility . Consider (25) and (26) :

The Spatial Prepositions


D

293

. -I I I I I I I I ' I ' , 0

r -' I I I ' I I I , I I I I I I I I I I I

I I I I I I I I I I I I I I I- - - -

,- - - - - - - - ~- - - - - - - -

Figure7.7 The relationship moreunderis determined the two by the total lengthof the overlapbetween in the XY-planeand not by the proportionof the total objectwhichis under(B > A), objects or theorientationof theobject(C > A). Whentwo objects differ in morethan onedimension , is determined dimension of eachandnot by thetotal area(D > C). farther under by thelargest

red carwas under thestreet . (25 ) The lamp street was over theredcar . (26 ) The lamp Sentence in most contexts . Thereason forthis , butless (26 ) isnotincorrect likely , at least in part be that the in thecognitive arespecified , may places map primarily by theinvariant features of anenvironment andonlysecondarily andtransiently by which them . objects occupy
7.2.4 Belleatll (or Underlleatll ) Beneath (or underneath ) has a meaning that is close to that of under but differs in two ways. First , it has a more restricted sensein the XY -plane. Whereas under means an overlap between the projections of the reference entity and the target entity, beneathmeans that the target entity is wholly contained within the limits of the reference entity projection . It follows that the projection of the lower entity in the XY -plane must be smaller than the upper. Furthermore, and in part as a of this restriction, the application of the comparator more (or farther ) consequence to beneathoperateson the Z -direction and not on the XY -plane.

294

John O' Keefe

(27) The red tray was farther beneath the top of the stack than the blue one. Beneaththen meansthat the target element is contained within the volume of space defined by its XY -projection through a large (or infinite ) distance in the - Z of to have a slightly more restricted meaning in the sense seems direction. Underneath limiting the projection in the Z direction. More underneathsounds lessacceptable than more beneathand might indicate that underneathis a three-dimensional volume of spacerestricted to the immediate proximity of the - Z or undersurface of the reference element.

S 7.3 Distance PreJ ")Sitioll near ( to) and its antDistances are ~iven by the preposition for and the adverbials
onymfar (from ) as in (28) and (29) . (28) This road goeson for three miles. (29) The housewas near (far from ) the lake. For gives the length of a path; near and far from give relative distances that are . In somecases , one or more of the contextual referentshave contextually dependent been omitted. Let us begin by examining the meaning of near when points are being related. O ' Keefe and Nadel ( 1978 , 8) observedthat the meaning of near was context, and I will pursue that line here. It follows that , with only two points, dependent neither is near (or far from ) the other. Three points, A , B, and C, provide the necessary and sufficient condition for useof the comparativesnearerandfarther . Note that the directions of the points from each other are not confined to the samedimension but are free to vary across all three dimensions , and that the distance is measured metric. Near is not simply derived Euclidean the line determined the by geodesic along of the proportional distances a sense from nearer but contains in addition among the items in question. (30) A is not near B but it is nearer to B than Cis . to be calibrated relative to distances The distancemeasureincorporated in near seems betweenthe items with the smallest and largest Euclidean distance separation in the set. Theseitems act as anchor points that control the meaning of the terms for all the others. Changing the relations of other items in the set can alter whether two items are near to or far from each other. Thus, in figure 7.8a, Band E are near each other, but in figure 7.8b, they are not. Consideration of the near/far relationship of two- or three-dimensional entities shows it is the surface points that are important and not any other aspect of their

TheSpatialPrepositions (a) B F E D

295

AB C

Figure 7.8 Nearnessis context-dependent . In (a) A is not near B but nearer than CE is near B in (a) but not in (b) . In (c), B is nearer A than C is by virtue of point x .

shape (e.g., centroid) or mass (center of gravity) . If we inspect figure 7.8c and ask which is nearer to A , shapeB or shapeC, we will seethat B is, by virtue of point x . Finally , the presenceof barriers seemsnot to influence our judgment of near or far , because . (31) is permissible (31) The houseis nearby, but it will take a long time to get there sincewe have to go the long way around.

7.4 VerticalPrepositio . : Reprise These considerations of the meanings of the verticalprepositions the following suggest conclusions :

296

John O' Keefe

1. Prepositions identify relationships between places , directions, and distances , or . Static locative prepositions relate two entities; static directional combinations of these prepositions relate three entities becausethere is always an (often implied) origin of the directional vector; and static distance prepositions also relate three entities becausethis is the minimum required to give substanceto the comparative
judgment that they imply . 2. The space mapped by the prepositions is at least two -dimensional and rectilinear in the vertical direction . The nonvertical dimension ( if present) may be rectilinear , but there are also circumstances in which the two non vertical dimensions may be expressed in polar ( or other ) coordinates . 3. The metric of vertical and nonvertical axes is identical because it is possible to compare distances along orthogonal axes. Interestingly , the distance between objects is calculated from the nearest surface of each entity and not from some alternative derived location such as the geometric centroid or center of mass. 4. The scale is an interval scale with a relative origin detennined by one of the reference entities of the directional prepositions (usually the vector source or tail ) . 5. In the vertical dimension , direction can be given by the universal gravity signal , which is constant regardless of location . In the horizontal plane , nothing comparable to this signal is available and the direction vectors must be computed from the relative 3 positions of environmental cues.

7.S HorizontalPrepositions
The original cognitive map theory suggestedthat , in the horizontal plane, places could be located in severalways. Foremost among thesewas their relation to other placesas determined by vectors that coded for distance and direction (figure 7.1) . In that the direction component of this a recent paper (O ' Keefe 1990 ) I have suggested vector is carried by the head direction cells of the postsubiculum. These cells are selectivefor facing in specificdirections relative to the environmental frame, irrespective of the animal' s location in that environment. The direction vector originating in one placeor entity and running through a secondcan be computed by vector subtraction (seefigure 7.9) of the two vectors from the observer to each of the entities, and ' s location. The resultant direction this computation is independent of the observer vector functions in the sameway in the horizontal plane as the gravitational signal in the vertical direction. The primary differenceis that , whereasthe latter is a universal signal, the horizontal direction vectors are local and need to be coordinated relative . to each other. This is achievedby mapping them onto the global directional system Locative horizontal prepositions, in common with their vertical cousins, specify . The directions are given relative to the places in terms of directions and distances

The Spatial Prepositions

..

Observer '. . " .. .. . '. " ." " '. '" " c " W " " . '~ ~ irection Vector AB ~ i f ij" :~ : ,
Vector AB

297

Figure7.9 The directionvectorthroughtwo objectsA and B can be computed by taking the difference between thevectors A and B. direction vector, and distancesare given relative to the length of a standard vector drawn betweenthe two referenceentities along the referencedirection. 7.5.1 Beyond Let us begin with an analysis of the spatial meaning of the preposition beyond . As shown on the left side of figure 7.10, this specifiesa three-dimensional region located with--to a specificrelationship to the referencedirection and a pair by the set of vectors--to of referencevectors (AB , A C ) terminating on different parts of the referenceobject or place. The region beyondthe mound is specifiedby the set of vectors originating at A whose projection onto the direction vector (inner product) has a greater length than --to the larger of the two reference vectors coincident with the direction vector (AC ). According to this definition , it acts in a manner analogous to be/ow in the vertical dimension. No restriction is placed on the location of the entity in the vertical direction , as can be seenfrom sentence (32) : (32) Janecamped beyond and above the woods. Furthermore, the effect of the comparator more is to act on the length of the vector in the horizontal plane: (33) The tower was farther beyond the mound than the castle.

298

John O' Keefe

A Beyondthe Mound Behind the Mound

A Besidethe Mound

Figure7.10 detennined can be represented as places , and beside , behind by their relation to the Beyond -, entitiesand a setof reference vectors directionvectordrawnthroughtwo reference (AB, AC is a restricted subset with a lengthgreater than AC. Behind AD). Beyond is the setof all places with locationvectorsgreaterthan AC and anglewith of beyond and includes only the places thoseplaces the directionvectorsmallerthan AD. Beside represents havinga projectiononto In additionthe angle AB less than AC. than and the reference directionof magnitude greater that of AD. with thedirectionvectormustexceed -used behither means that the The opposite of beyondis the seldom , and this ---+ simply location vector has a length lessthan the referencevector AB . 7.5.2 Be1lill4 Behindfunctions in a manner analogous to underin that it placesgreater restrictions . An object behinda referenceentity on location than doesbeyond - - . is located by the set of vectors with a ---+ larger magnitude than the referencevector (A C ) but with an angle less than vector AD (figure 7.10 , an entity can be partially , center) . As with under behindthe referenceentity, and the test for this is an overlap in the projections of the two in the XZ -plane. This need for overlap accounts for the awkwardnessin using behindwith referents that are not extendedin the vertical dimension. (34) me tree was behindthe trench. (35) me cottage was behindthe lake. The application of the comparator test shows further similarities. In the sameway that farther under can refer to the amount of overlap in the XY -plane between two

TheSpatial Prepositions

299

entities separated in the vertical dimension, so farther behind can refer to greater overlap in the XZ -plane of entities separatedalong a horizontal referencedirection. (36) The red toy was pushed farther behind the box than the blue ball. The source of the direction vector can be specified explicitly as the object of the preposition from . (37) From where Jane stood, Jameswas hidden behind the boulder. More usually, the source is implicit , being inferable from the previous context. In sentence , it would be legitimate to omit the first clauseif the previous (37), for example narrative had establishedthat Jane had been looking for James . More often, the source of the direction vector is the implicit deictic here. In a pool game it might be the cue ball: (38) The last red was behind the eight ball. Familiar objects have " natural " behindsestablishedby a vector drawn from one differential part to another, as, for example , the front to the back of a car. However, this is easily overridden by the motion of the vehicle: (39) The car careeredbackward down the hill , scattering pedestriansin front of it and leaving a trail of destruction behind it . The opposite of behindis before , or more usually in front of 7.5.3 Bes ;de Besideidentifies a region at the end of the set of vectors whose projections onto the referencedirection fall between the referencevectors All and .-:iC ' but whose angle with the referencedirection is greater than that of referencevector AD (figure 7.10, right) .
7.5.4 By By is the generalized horizontal preposition and includes the meanings behind , beyond, and beside with a slight preference for the latter . 7.6 Omnidirectional Prepositio .

of before ,

At , about , around , between, among (amid ), along , across, opposite , against , from , to , via , and through locate entities in terms of their relationships to other entities irrespective of their direction in a coordinate reference framework and therefore can be used in any of the three directions . At is the general one - to - one substitution operator that locates the entity in the same place as the reference entity . About relaxes the precision

300

John O' Keefe

of the localization and introduces a small uncertainty into the substitution. About is . In the cognitive map theory the size of the equivalent to at plus contiguous places place fields is a function of the overall environment, and this would appear to apply to about as well. Therefore the area covered by about is relative to the distribution of the other distancesin the set under consideration in the sameway that the meaning of near dependson the distribution of the entities within the set. Around has at least two distinct meanings , both related to the underlying figure of a circle (i.e., the set of vectors of a constant R originating at an entity) with the referenceentity at its center. on that circle. If it is extended The first meaningis that the locatedentity is somewhere , if the circle it lies on several contiguous places along ; more compact, it lies at one place on the circle perhapsat the end of an arc of the circle. (40) The shop was around the comer. in almost all instancesthe radius of the circle is left undefined, except that it Because be small relative to the averageinterentity distancesof the other membersof the set, there is little to choosebetweenthe use of about and around when single entities are located. When multiple entities are located, there is the weak presumption that they all lie on the samecircle when around is used , but not when about is used. (41) Those who could not fit around the table sat scatteredabout the room. Betweenlocates the entity on the geodesicconnecting the two referenceentities. The computation is the sameas that for deriving a direction vector from the subtraction of two entity vectors (seeabove discussionin section 7.5), except that the order in which theseare taken is ignored. An equivalent definition of betweenis that the sum of the distances from each of the referenceentities to the target entity is not greater than the distance betweenthe two referenceentities. Alternatively , the angle made by the vectors joining the target to each of the referencesshould be 180 . . Among increasesthe number of referenceentities to greater than the two of between reference The interesting issuehere, as with many of theseprepositions that usemultiple entities, is how the referenceset is defined. Among roughly meansthat the target entity is within some imaginary boundary formed by the lines connecting the outermost items of the set. But clearly the membership of the referenceset itself is not immediately obvious. Consider a cluster of trees with an individual outlier pine tree somedistance from the main group . , but stood betweenthe thicket and the lone pine. (42) He was not among the trees This suggests that the application of the preposition amongdependson a prior clustering to determine the numbers of the referenceset. Amid operation that is necessary is a stronger version of amongthat conveysthe senseof a location near to the center

The Spatial Prepositions

301

of the referenceentities. One possibility is that the centroid or geometrical center of the cluster is computed, and amid denotes a location not too far from this. The centroid is a central concept in one computational version of the cognitive map ' ). theory (O Keefe 1990 Across , along, and oppositeare like down in that they situate an entity in terms of its relationship to a referenceentity and a one- or two-dimensional feature. Two dimensional features are usually more extended in one direction than the other. Across specifiesthat the vector from the referenceentity to the target intersects the reference line or plane an odd number of times. Along specifiesan even number . In addition , there is the weak presumption that the (including 0) of intersections distance from the target entity to the last intersection is roughly the sameas from the referenceentity to the first intersection; that is, both are roughly the same distance from the referenceline or plane. Oppositerestricts the number of intersectionsto one and the intersection angle to 90 . Against specifiesthat the entity is in contact with the surfaceof the referenceentity at at least one point . It is, however, not attached to it but is supported independently in the vertical dimension. In the present scheme , from and to mark places at the , and via and beginning and end of a path that consists of a set of connected places through specify someof the placesalong the way. (43) Oxford Street goesfrom Tottenham Court Road to Marble Arch via Bond 't Street but doesn passthrough Hyde Park.

_ ion - andtheFourthDime 7.7 Temporal Prepositio


The incorporation of time into the mapping systemis accomplishedthrough various . The primary grammatical featuresare tense , aspect , grammatical and lexical features and the temporal prepositions. Because my emphasisin this chapter is on the prepositional , , I will mention tense and aspect only in passing (see Comrie 1975 system . 1976 ) / 1985for detailed discussions In the presentsystem , time is representedas a set of vectors along a fourth dimension at right angles to the three spatial ones. Each event is representedas a vector that is oriented with its tail to the left and its head to the right , this constraint being due to the fact that changesin time can take place in only one direction (from past to future) . The location of these time events is also based on vectors and thesecan be oriented in either direction from a reference point , which can be the present moment of the utterance or someother time. Times future to the referencepoint have vectors of positive length, times past have vectors of negativelength, and the present , a vector of 0 length. These different times are representedby the tensesof the verb.

John O' Keefe


" The choice of the present time as a 0 referencepoint is traditionally called absolute " " " tense while that of a nonpresent referencepoint , relative tense (seeComrie 1985 the vectors representingtime are all unidimensional, for further discussion ) . Because of the temporal prepositions lying parallel to the fourth axis, we will expectthat the senses are also unidirectional . For example , most of the temporal prepositions are similar to (diachronically borrowed from?) their homophonic spatial counterparts, to be that but not all spatial prepositions can be so employed. The general rule seems only spatial prepositions that can operate in the single, nonvertical dimension of the line can be borrowed in this way (but seethe specialcasesaround and about) . As we shall see , this leavesthe nonphysical vertical prepositions free to representspecialized relationships betweenentities. The temporal prepositions, then, specify the location, order, and direction within . In my the fourth dimension of the entities and eventsof the other three dimensions brief summary I will classify them according to whether they use one or more reference points. Becausethe temporal dimension appearsto be confined to a single axis , in the latter casesthe two referencesare confined to orthogonal to the spatial axes that axis and are therefore collinear. My discussionof the meaningsof the temporal prepositions will be basedon the abstract eventsportrayed in figure 7.11. The upper event shows a state of affairs in which an entity occupies a vertical location before time A , then jumps to a new location and remains there for a short period AB , after which it returns to the previous location. The lower event shows a processof movement 44 and 45 as examples of the over a period of time. Let us use the sentences . the state AB CD and , respectively process . (44) Mary moved from an apartment on the top floor to one on the floor beneath ' (45) Sarah, Mary s roommate, dropped down to tidy up the new apartment for an hour during the move. of eventsonto the time axis is shown at the bottom The projection of thesesequences of the figure. The punctate events A and B, the beginning and end of the dropping down, are marked as points on the time axis. These points can be located in three , ways. First , they can be placedin isolation independentlyof any other representation as might occur at the beginning of a story . Second , they can be related to the present time of the speaker / listener or , third , to some other previously identified time. In instances theselatter , the location vector is drawn with the tail at the referencepoint and the head at the located time, that is, from right to left (with a negativemagnitude) if the event occurred prior to the reference point , and from left to right (with a positive magnitude) if it occurred later than the referencepoint . es(Mary ' s move) and The eventsthemselves are states(dropping down) or process are representedas vectors that must move from left to right (no time reversal ) . The

The Spatial Prepositions


.

303
8

-------- - N

-----

-----------

lJ -------~
.
.

c """ D ~ -----_ .- ----------.

C [) ~ AS TIME

C A B D

. Present

Figure7.11 as relationships in a fourth dimension . An event such as " Sarah Temporalprepositions " on the Z -axis that begins at time A, droppeddown is represented by a physicalmovement endsat time B, and is represented vectorAB on the time axis . A process suchas " Mary by " has a similar moved on the time axis . The representation assumes that the representation events occurred in the past could have been . , but other0 reference points adopted three eventsof the top sequence (the dropping down and the presuppositionsof being in and returning to the upstairs apartment, are representedon the T-axis by vectors --+ --+ --+ AB , - TA , and + BT , respectively . The tail of the secondand head of the third are left indeterminate. Here I am assuming that all events have some projection in the time domain, but that this can be ignored, for example , when the length of the event vector is short in comparison to the length of the location vectors. --+ The processof moving representedby vector CD has a similar representation on the time line, the difference betweena state and a processresiding in changesin the nontime dimensions . Referring to figure 7.11, I suggestthat the meaning of the temporal prepositions is as follows. The usual representationof a processsuch as CD is (46) The move took placefrom noon to 2 P.M. The event CD has a time vector which begins at Tc (noon) and ends at To (2 P.M.) . --+ T (CD) = To - Tc, where D and C are the respectivelocation vectors.

304

John O' Keefe

(47) The move lastedfor two hours - -+ setsthe length of vector CD. (48) Sarah dropped down after Mary beganmoving, before Mary finished moving, by the end of the move sets T.. > Tc, T.. < To, T.. :::::; To. (49) Sarah visited the new apartment during the move sets Tc < T.. :::::; TB < To. Sinceand until are two temporal prepositions that do not have spatial homologues . Until specifiesthe time at which a state or processended, whereassince specifiesthe time at which it began. Sincehas the additional restriction that the temporal reference point acting as the source of the location vectors for the event in question must be later than the event, that is, the location vectors must have negativemagnitudes . This is to account for the acceptability of (50) but not (51) . (50) Mary has (had) beenmoving sincenoon. (51) ?By 2 P.M. tomorrow Mary will have beenmoving sincenoon. The simple temporativesat , by, in locate an entity by referenceto a single place on the fourth axis. At operates in the same way as it does in the spatial domain by substituting the place of the referent for the entity . By fixes the location of the reference . In suggeststhat there is an point as the maximum of a set of possible places extent of time that is considered as the referent and that contains the entity . On is somewhatmore difficult ; it would seemto introduce the notion of a secondtemporal dimension, a vertical dimension that would place the entity at a location above or alongside of the time point . About and around also suggesta second dimension. In to be restricted to the days of the week , however, the temporal useof on seems general on and to dates on the is not usedin any general sense . It and ( Friday) ( first of April ) therefore be an use to these from the hours of may idiosyncratic distinguish pointlike ' the day (at 5 o clock) on the one hand and the extendedmonths of the year (in May) . Other simple temporal prepositions give the location of the event or duration of the condition by referenceto a time marker that fixes the beginning or end of the time vector. Whereasby and to set the head of the temporal vector at the referenceplace, before setsit to the first place to the left of that place. In neither caseis the origin or tail of the vector specified . This is given as the object of from . During specifiesboth the head and tail of the temporal vector. An event that occurs after one time and before another occurs during the interval. The length of the vector is given by the preposition for .

The Spatial Prepositions

305

As with the spatial prepositions, some of the temporal prepositions require two referencepoints for their meaning. These include between , beyond , past, since , and and the of the event later than the first time two times locates the start until. Between . The referent in beyonddenotesthe value that the end of the event before the second . Becausethe time axis is basically a unidimensional head of an event vector exceeds one, the important distinction betweenpast and beyondin the location of the entity in the orthogonal axis of the spatial domain does not apply, and the two prepositions . appear to be interchangeablein most expressions

7.8 Translation and Transfonnation Vectors


Once one has a temporal framework , it is possible to incorporate the notion of changes into the semantic map. These take two forms: changes in location and changesin state. The second of these relates to the circumstantial mode of Gruber ) and Jackendoff ( 1976 ) . Both changesare representedby vectors. Changesin ( 1976 location of an object are representedby a vector whose tail originates at the object in a place at a particular time and ends at the same object in a different place at a subsequenttime. Changesin state are representedby a vector drawn from an object at time 1 to itself in the same location at time 1 + I . The change is encoded in the attributes of the object. In both types of change , the origin or tail of the vector is the object of the locative preposition from , and the head or terminus of the vector . is the location identified by the locative preposition 10 (52) The icicle fell from the roof to the garden. . It consistsof a four -dimensional The representationof this is shown in figure 7.12 structure with time as the fourth dimension. In the figure, I have shown two spatial dimensionsand one temporal dimension. The left sideof the representationshowsthe unstated presupposition that the icicle was on the roof for some unstated time prior . As Nadel and I noted (O' Keefe and Nadel 1978 to the event of the sentence ), the is read as an and its location between object relationship (53) a. The icicle was on the roof (before time I ) . b. The roof had an icicle on it . The middle of the figure shows the translation vector that representsthe event of the sentence , and the right hand the postsupposition that the icicle continues in the garden for someduration after the event. (53) c. The icicle was in the garden (after time I) . The representation of the second type of change , the circumstantial change , also involves a vector, this time a transformation vector, where there is no change in the

306 TRANSLATION VECTOR

John O' Keefe

t +

in locationof an objectin the semantic by a mapat a particulartime 1is represented Change . The four. In addition to the time axis translationvector , one spatial axis (Z ) is shown " " " " dimensional , labeled icicle , is shownon the placelabeled roof at all timesprior to object " " 1 (1- ) and in the placelabeled garden at all timesafter 1 (1+ ). The vertical movement . thetwo places vectordrawnbetween between thetwo places at 1is represented by a translation location of the object, but a change in one of the attributes assignedto the object. Objects are formed from the collection of inputs that occupy the samelocation in the ' map and that translocate as a bundle (see O Keefe 1994 for a discussion of this Kantian notion of the relationship between objects and spatial frameworks) . Thus each object has associatedwith it a list of attributes. In a circumstantial change ,a vector representsthe changein one of theseattributes at a time t. Figure 7.13 shows the map representationof sentence54. (54) The icicle melted ( = changedfrom hard to soft at time t , or changedfrom solid to liquid ) . 7.9 Metaphorical Usesof Vertical Prepositions In the following sections , I shall explore the metaphorical usesof the vertical stative prepositions. I hope to show that they apply to two restricted domains: influence ) and social status. In the course of this discussion I shall (including social influence ask some of the same questions about these metaphorical uses as I did for their : what are the properties of the spacesrepresented , what type of scaleis physical uses used , and so on? Section 7.9.1 will explore the metaphorical meaningsof be/ow and beneathas used within the restricted domain of social status. Section 7.9.2 will deal with under , whose

7.12 Figure

The Spatial Prepositions

307 VECTOR TRANSFORMATION

---------

------------ICICLE long cold - - - - - - -J ( solid }


- - - - - - - - - - - - -

I CICLE long cold liquid)

'

- -------- ----

- - - - - - - - - - -

- - - - - - - - - - - - -

(
4

ROOF
t+ .

Figure7.13 vector in stateof an objectin the semantic by a transformation maparerepresented Changes in the old propertybeforet and whoseheadendsin the new property whosetail originates after t.
semantics is more complex , but appears to be restricted to the domain of influence or control . In general , the representation of ideas such as causation , force and influence

in the semanticmap presentsa problem. The basic mapping systemappearsto be a kinematic one which does not representforce relations. The closestone comesin the physical domain is the implicit notions that .an entity which is vertical to another and in contact with it might exert a gravitational force on it or that an entity inside another might be confined by it . This might explain why the prepositions that convey theserelationships, such as under and in, are used to representinfluence in the metaphorical domain. 7.9.1 Below, Beneath , and Dow" Contrast the following legitimate and illegitimate metaphorical uses of be/ow and under: ) her station. (55) Shewas acting below (beneath (56) Shewas acting under his orders. (57) . Shewas acting under her station. . ( 58) Shewas acting below his orders. When looking at be/ow and beneathwithin the domain of social status, the first thing to notice is that people are ranked or ordered in tenDs of their social status on a . One person has a higher or lower status than another, and that status vertical scale would appear to be transitive: if A has a higher status than Band B than C, it follows

308

John O' Keefe

that A has a higher status than C. I am ignoring here the possibility that status might be context-specific becauseI do not think this is reflected in the semanticsof the prepositions. Now within the vertical scale of status, one can have a disparity between the value assignedto an individual act and the longer-term status. This gives rise to sentences such as (59) John acted in a manner beneathhim. (60) That remark was below you. A sequence of such actions, however, will result in a status change , so that (61) Until recently that remark would have beenbeneathyou, but now it is quite in character. The antonym of be/ow/ beneathin this context is above . , although it is not much used (62) Sally was getting above her station, but not * (63) That remark was above you. The use of be/ow and beneathin this senseis restricted to reflexive status, and thus one could not say ' (64) John acted in a way beneath Sally (Sally s station) . Thus the best model (seefigure 7.14 ) seemsto be one in which each status token is confined to a vertical line in the status dimension, but these are free to vary in the other dimensions such that John can move so as to be beneath himself but not beneath Sally, but at the sametime can be compared in the vertical dimension with " " Sally, His status is below hers. Finally , note that there is no vantagepoint (egocentric point ) from which thesejudgments are made or which would change them (i .e., ' s status is not relevant . the speaker ) . The stative preposition down seems to have almost no usein the nonphysical sense The closestone comesare colloquial forms of verbal ranking such as (65) Put him down.

7.9.2 Under in the meta the most interesting useof the verticalprepositions Underhasperhaps . It to be of influence or control . In The domain seems confined to the domain phorical as a CognitiveMap ( 1978 that one of the Hippo ), Nadel and I suggested campus . HereI will pursue theideathat this domains wouldbethat of influence metaphorical " " is represented by an additional vertical dimension (figure7.15 ). relationship

The Spatial Prepositions


CI .

309

Sally

Status John

Tom

INFLUENCE

:# ' ~ " " ' : ~ ~ ~ ~ ~ :~ :! II


, ( ~

7.15 Fipre Influence of one entity, usually an agent, over another entity or an event is representedby a superior location of the first on the vertical influence axis.

TIME

~ o " , ' 0 " v

~ ~ ~ : : : : : : : :: : ~

310

John O' Keefe

There are two homophones (under! and under ), which follow different rules and which are derived from the two meaningsin the physical domain: (66) under a widening sky (67) under the table Compare (68) Under the aegisof with (66), and ' (69) a. under John s influence b. under Sally' s control with (67) . The first meaning of undercannot take a comparative form . * (70) More under the aegisof the King is not transitive, and has no antonym. * (71) He was above, outside of , free from the aegisof the King . In contrast, the secondmeaning follows all the rules for the secondphysical under . (72) More under her influence every day. But surprisingly the antonym of this under is not over in many examples , but varies with the direct object. . (73) Shewas free from stress (74) The car was out of control . (75) He was out from under the control of his boss. As the last examplessuggest , the referent in this meaning of underhas an extent in the vertical dimension, and to be more under a cloud than X has the same senseof a greater overlap in the projection onto (one or more) horizontal dimension as in the physical meaning. To increase or decreasethis influence requires a movement or expansion of one or the other entity in the horizontal plane, and this may require force in that direction. (76) John was more under control than Sam. (77) John was more under the influence of Mary than Sam. ' (78) She slowly extricated Sam from Harry s influence.

The Spatial Prepositions

311

There are two types of relationships that conform to this pattern, control and influence, and thesevary in the amount of freedom left to the referent object. (79) Jane increasedher influence over Harry until shehad complete control . The antonym of under2is over. ' (80) Jane s influence over John (81) Jane lords it over John. (82) Jane holds sway over John. ' (83) a. * The King s aegiswas over John. * b. The King held his aegisover John. Notice that the underrelationship is not transitive. John can be under Jane' s influence and Jane can be under Joe' s, but John is not necessarilyunder Joe' s. Finally , I wish to remark briefly on the fact that there appear to be two nonphysical vertical dimensionsthat are orthogonal to eachother and to the physical vertical one. On the face of it , it does not seemobvious how they could be reduced to a single dimension becauseone wishes to preserve the possibility of the following types of relationship. to act below his station in order to maintain control over (84) Jack felt it necessary Jane. Perhaps here oile should consider the possibility that overlapping representations symbolize a control or influence relationship while nonoverlapping ones stand for a status one in the same 2-D space . If this were the case , what would the Z -axis be? Perhapsthe higher the status, the more possibility for control? Finally , in terms of the scaling of the metaphorical vertical prepositions, they . Thus one can say: appearto havethe sameinterval scaleas their physicalcounterparts (85) Jane is as far below Mary in status as John is above ' (86) John is lessunder Sam s control than Jim is and it will be easierto extricate John. Note that , unlike the three dimensions of physical space , we cannot compare the Z -axis and the non-Z -axis directly . ' * (87) John is more under Sam s control than Sam acted below himself. Now we come to the most difficult part of the theory: the relationship betweencontrol and causation. Causation, on this reading, would be the occurrenceof an event underneath the control of an agent' s influence.

312
(88) The book went to the library . (89) John causedthe book to go to the library . 7.10 Causal Relatioll Sin the SemanticMap

O'Keefe John

Our analysisof the metaphorical useof be/ow and underhas led to the suggestionthat the causal influence of one item in the map over another might be representedby relationships in the fifth dimension. If the influence of an agent over another agent or , then it might object can be representedby the location of the first above the second be possibleto representthe influenceof an agent over an event such as that portrayed in (90) an (91) by an action or movement along the influencedimension. Consider the : closely related sentences (90) Mary made (caused ) the icicle fall from the roof to the garden. (91) Mary let (did not prevent) the icicle fall from the roof to the garden. , which differ , theseare five-dimensional sentences According to the present analysis in the control exerted by the agent over the event. As we saw in the previous section, influence is representedby an under relationship between the influencer and the influenced. The lateral overhang between the two representsthe amount of control exerted , and the distance between them on the vertical dimension, the amount of influence exerted. On the simplest reading, causation is representedas a pulsatile increasein influencecoincident with the physical spatial event. Figure 7.16showsthis ' as a momentary increasein Mary s influence to symbolize an active role in the event, while figure 7.17 shows a continuing influence but no change to symbolize a passive role in the event. The sentence (92) Mary did not causeX is ambiguous, with two possible underlying structures: one in which Mary has influence but the event did not happen; and the other in which the event did happen but the causalinfluencewas not exertedby Mary . This type of representationcan also it can show how capture someof the more subtle featuresof causalinfluence, because influence can selectivelyact on parts of the event as well as on the whole. For example , the sentence (93) Mary made John throw down the icicle ' meansthat both Mary and John had agentiverolesin the event , but that Mary s was the by placing Mary at a higher level than John in influence superior one. This can be represented in their locationsat the time and showingmomentarysynchronous changes space : of the event. the complex influencerelationship also allows for the following sentences

The Spatial Prepositions

313
THEEVENT MARYCAUSED

~ =

Figure 7.16 Causal influence is representedby a pulsatile changein the vertical inftuen ~ dimension at the sametime t as the physical event.

(94) Mary allowed John to throw down the icicle. (95) Mary allowed John to drop the icicle. (96) Mary made John drop the icicle. It also permits one to representrelative degrees of influenceover an event in a manner analogous to that over agentsor objects, as in (97) Mary had more influence over the course of eventsthan John, or the idea that an event of continuing duration can have variable amounts of control at different times, (98) Mary took over control of the event from John on Monday . 7.11 Syntactic Structuresin Vector Grammar Thus far , I have said very little about the way that surface sentences and paragraphs ' could be generatedfrom the static semantic map. Nadel and I (O Keefe and Nadel

TIME

~ o ~ & -------. /~"~ v ~ ]

INFLUENCE

314
EVENT THE ALLOWED MARY

John O' Keefe

OBJECf

-.~ ~ " ~ ~l
,

Figure7.17 in the verticalinfluence dimension Permissive influence is represented of change by theabsence . of theinfluencer during theevent 1978 ) likened this operation to the way in which an infinite number of routes between two placescould be read off a map. Recall that the cognitive map systemin animals includes a mechanism for reading information from the map as well as for writing information into the map. In particular , we postulated a system that extracts the distance and direction from the current location to the desired destination. This information can be sent to the motor programming circuits of the brain to generate spatial behaviors. The corresponding systemin the semanticmap would comprise the syntactic rules of the grammar. The syntactic rules operate on both the categoriesof the deep structures and the direction and order in which they are read. For example , reading the relationship between an influencer and the object or event influenced determines whether the active or passivevoice will be used. In an important sense there are no transformation rules for reordering the elementsof sentencesbecause theseare read directly from the deep structure. Given a particular semantic map, a large number of narrative strings can be generateddepending on the point of entry and the subsequentroute through the map. Economy of expressionis analogous to the optimal solution to the traveling salesmanproblem.

TIME

~ o ~ & v

MARY

INFLUENCE

.... :

- - - - - - -

OBJECf
~ ~~ ~ ~ ~ ~

( :::

TheSpatialPrepositions Acknowledgments

315

contributions Cartwright for herextensive I would like to thank Miss Maureen helpand substantive . The version an earlier on comments made . experimental to this chapter Neil Burgess that forms the basisfor the cognitivemap model was supportedby the Medical research Councilof Britain. Research
Notes I do not wish I . I have deliberately chosenthe tenD entities to refer to the relationships because , and so on. to limit my discussionto objects, but wish to include places , features 2. In what follows , I have relied heavily on the classicdiscussionby Torgerson ( 1958 ). is absentor so weak in humans that it is not available 3. I am assumingthe geomagneticsense for spatial coding. As far as I am aware, there is no evidencefor it in the prepositional system . of any language References : Towardsa localistic theory. Cambridge: Cambridge Anderson, J. M . ( 1971 ) . The grammar of case . University Press : An essay in stratificational BennettD . C. ( 1975 ). Spatial and temporal uses of English prepositions . : . London Longmans semantics . . Cambridge: Cambridge University Press . Comrie, B. ( 1976 ) Aspect . . Cambridge: Cambridge University Press Comrie, B. ( 1985 ) . Tense . Cook , W . A . ( 1989 ) . Casegrammar theory. Washington, DC : Georgetown University Press Frisk , V ., and Milner , B. ( 1990 ) . The role of the left hippocampal region in the acquisition and retention of story content. Neuropsychologia , 28, 349- 359. Institute of Technology. Gruber, J. ( 1965 ). Studiesin lexical relations. PhiD. diss., Massachusetts . Amsterdam: North Holland . Gruber , J. ( 1976 ) . Lexical structuresin syntax and semantics . Linguistic Inquiry , 7, Jackendoff, R. ( 1976 ) . Toward an explanatory semantic representation . 89- 150 " " " " Landau, B., and Jackendoff, R. ( 1993 ) . What and where in spatial language and spatial 217 265. , 16, cognition . Behavioraland Brain Sciences ' O Keefe, J. ( 1988 ) . Computations the hippocampus might perform . In L . Nadel, L . A . Cooper, , 225- 284. , mental computation P. Culicover, and R. M . Harnish (Eds.), Neural connections . Cambridge, MA : MIT Press ' O Keefe, J. ( 1990 ) . A computational theory of the hippocampal cognitive map. In O. P. , OUersen and J. Storm- Mathisen (Eds.), Understandingthe brain through the hippocampus 287- 300. Progressin Brain Research , vol. 83. Amsterdam: Elsevier.

316

John O' Keefe

O' Keefe, J. ( 1991 . In J. Paillard ) . The hippocampal cognitive map and navigational strategies . , 273- 295. Oxford : Oxford University Press (Ed.), Brain and space O' Keefe, J. ( 1994 ) . Cognitive maps, time and causality. Proceedings , 83, of the British Academy 35- 45. O' Keefe, J., and Nadel, L . ( 1978 ) . The hippocampusas a cognitive map. Oxford : Clarendon . Press Scoville, W. B., and Milner , B. ( 1957 ). Loss of recent memory after bilateral hippocampal . Journal of Neurology, Neurosurgery lesions , and Psychiatry, 20, 11- 21. Smith, M . L ., and Milner , B. ( 1981 ) . The role of the right hippo campusin the recall of spatial location. Neuropsychologia , 19, 781- 793.

- 81. location : Encoding deficitor rapid forgetting ? Neuropsychologia , 27, 71

Smith, M . L ., and Milner , B. ( 1989 ) . Right hippocampal impairment in the recall of spatial

Taube , J. S., Muller, R. U., and Ranck , J. B. ( 1990 ). Head direction cells recordedfrom the postsubiculum in freelymovingrats. I . Description and quantitativeanalysis . Journalof - 435 Neuroscience . , 10 , 420 - 208 Tolman .. Review . , E. C. ( 1948 ). Cognitive , 55 , 189 mapsin ratsandmen Psychological andmethods / ing. NewYork: Wiley. , W. ( 1958 ). Theory Torgerson ofsca

Chapter

8 Representations of Objects in Languages and

Geometric Multiple Language Learners

Barbara Landau

Central to our understanding of how young children learn to talk about spaceis the question of how they representobjects. Linguistically encoded spatial relationships most often representrelationships betweentwo objects, the one that is being located " " ' (the figure object, in Talmy s 1983 terms) and one that servesas the reference 's " " object (Talmy ground object) . Crucially, learning the language of even the plainest spatial preposition- say, in or on- requires that the child come to represent objects in terms of geometrical descriptions that are quite abstract and quite distinct from each other. Consider the still life arrangement in figure 8.1. If we were to describe this scene , we might say any of the following : ( I ) a. b. c. d. There is a bowl. The bowl has flowers painted on it . It has some fruit in it . There is a cup in front of the bowl and a vasenext to it .

What are the geometric representationsunderlying thesedifferent spatial descriptions " - we ? In calling each object by its name- " bowl ," " cup," " vase distinguish among three containers that have rather different shapes(and functions), suggesting that we are recruiting relatively detailed descriptions of the objects' shapes . Such descriptions could be captured within a volumetric framework such as that described by modem componential theoriesin which object parts and their spatial relationships are represented (e.g., Binford 1971 ; Lowe 1985 ; Marr 1982 ; Biederman 1987 ) . This is one kind of representation . However, in describing the spatial relationships between or among objects, we seemto recruit representationsof a quite diffe .rent sort. When we say, " The bowl has some fruit in it ," we recruit a relatively global representation of the object' s shape , in which its status as a volume- a " container" - is critical , but no further details are. When we say, " The bowl has flowers painted on it ," we seemto recruit a different representation , one in which the surface of the object is relevant,

318

Barbara Landau

Figure 8.1 Each object in this scenecan be representedas a number of different geometric types.

" " but nothing elseis. When we say, There is a cup infront of the bowl , we recruit yet - one in which the principal axesof the bowl are relevant. a different representation " " The region in front of the bowl spreadsout from one of its half axes(and whether these axes are object-centered or environment-centered depends on a variety of factors; seeLevelt, chapter 3, this volume) . These few examples show that learning the meanings of spatial terms requires learning the mapping betweenspatial terms and their corresponding regions where " the relevant regions are defined with referenceto geometrically idealized or sche " matized representationsof objects (Talmy 1983 ) . Therefore a crucial part of learning the mappings is properly representingobjects in terms of their distinct relevant , representingan object as a volume in the case geometricaldescriptions for example of the term in, as a surface in the case of the term on, and as a set of axes in the . In fact, learners must possessthese object representations case of in front of and behind the correct before learning mapping; if the objects cannot be represented properly , the terms cannot be learned.

: Geometric Representationsof Objects ultlplf

319

The brief analysisjust given suggeststhat there is a variety of object representations - the languageof objects and . Objects must underlying spatial language places be representedat a fairly detailed level of shape , they must also be representedat a - and they must be representedat a level that skeletal level- simply as a set of axes " " is quite coarse (as volumes, surfaces , or simply blobs ) . That we can talk easily about bowls, cups, and vases , and the kinds of spatial relationships into which they enter suggests that we possess a cognitive systemthat allows for flexible " schematiz " of ing objects (cf. Talmy 1983 ) . Central to the present discussion , the early acquisition of spatial terms among children suggeststhat thesemultiple representationsof objects may exist early in life and may be used to guide the learning of spatial . language The idea that very young children might possess such rich and flexible representations of objects is at odds with traditional theories of spatial development , which substantial in over the first six years of life. According posit changes spatial knowledge to Piaget' s theory, the first two years of life are devoted to constructing a system of knowledge that can support the general permanenceof objects in the face of continually changing perceptual and motor interactions betweenthe infant and objects in the world (Piaget and Inhelder 1948 ; Piaget 1954 ) . Once such knowledge has " " of the child is said to possess true representations , developed objects- representations ' that go beyond perception. However, the child s knowledge of spaceis still incomplete. Piaget hypothesizedthat from around age two , the developmentof spatial of stagesin which children would knowledge would proceed through a sequence - highly general properties such as first representonly top logical properties of space connectedness and openness versusclosedness . Although even infants might be capable of discriminating betweenobjects having different metric properties (e.g., a square vs. a triangle), Piaget proposed that the child possessing a topological representation of spacewould only be capable of representingthe difference betweena line and a closed loop, but not the differencebetweena square and a triangle. For Piaget, such impoverished representations were evidenced , for example , by the fact that twoand three-year-olds draw a variety of geometric figures as simple open versusclosed no specific metric properties. Later, projective properties would figures, possessing develop, such as the straight line, or a relationship specifiedby location along such a line; metric properties such as angles and distances would come to be represented even later, sometimeduring later childhood. ' Extending Piaget s view to the realm of spatial relationships, a topological representation could support understanding of a contact or attachment relationship between two objects, but could not support the representationof a distinction between contact with a vertical versusa horizontal surface . Similarly, relationships such as

320

Barbara Landau

that encoded by the terms in front of or behind would require at least projective , emergingduring late childhood. representationsof space While topological properties might seemcongenial to the analysis of spatialloca tional terms (Talmy 1983 ), a variety of evidencesuggeststhat a topological representation ' of objects and relationships is too weak to characterize young children s , the child who was limited to representingobjects topologi knowledge. For example cally would be incapable of using precise object shape for naming bowls or cups, would be unable to representobjects in terms of their axesin order to learn such basic , and would be unable to learn the distinction spatial terms as in front of or behind betweenGerman auf and an (attachment to vertical vs. horizontal surface ). In this chapter I review evidenceshowing that such nontoplogical representations . Further , are indeed accessibleto young children learning the language of space it appears that young children possess multiple representationsof objects that can of the spatial lexicon. I focus on three different of different parts support acquisition " : ( I ) " coarse kinds of representations , bloblike representations of objects, which " " eliminate all details of shape information ; (2) axial representations , which eliminate all details of shapeexcept the relative length and orientations of the three principal " " axes , which preserve a considerable ; and (3) fine-grained representations is primarily based on studies I will describe The evidence detail. of degree shape from children learning other evidence of young children learning English, although languagesis consistent. The evidenceindicates that both coarse and axial representations ' of objects can be elicited by engaging children s knowledge of known and novel spatial terms (in English, spatial prepositions) . The axial representations in particular illustrate that young children naturally representobjects in terms of skeletal ' descriptions in which the object s principal axes are the major components of ." The studies also indicate that , although the representationsunderlying its " shape " " ), by Talmy 1983 spatial terms appear to strip away details of shape(as suggested -based representations of objects are also accessibleto young fine-grained, shape children. Theserepresentationstend to emergewhen children are engagedin learning . object names In the following sections , I first outline how objects are representedwhen they are " " encodedby noun phrase argumentsof spatial prepositions in English (e.g., the cat " " " " or mat in the sentence The cat is on the mat ), and how theseobject descriptions . Pa~ icular emphasis differ from those relevant to similar spatial terms in other languages will be placed on comparing English to other languages whose locational terms appear to incorporate much more shape information than those in English. Next I present evidenceshowing that young children learning English show strong blasesto ignore fine-grained shapewhen learning novel spatial terms or when interpreting known English spatial terms, but that they show equally strong blases

Multiple Geometric Representationsof Objects

321

to attend to fine-grained shape when learning novel object names . This empirical evidence will raise a number of questions which I will outline , , including issues of possible structures and mechanisms underlying this gross difference in object . representation

8.1 Waysof Representing in Places Objects


How are objects representedwhen they serveas figure or referenceobject in a locational - are encoded canonically ? In English, spatial locations- places expression by prepositional phrasesheaded by spatial prepositions. In a simple sentencesuch " " " " " as " The flowers are on the vase , the flowers play the role of figure, the vase is the referenceobject, and the spatial preposition " on" maps a region of spaceonto the referenceobject. Although the upper surface of an object may be the preferred reading for on in English, the relevant region is actually any portion of the surface of the vase : The sentencewill be true regardlessof where in particular the flowers are located, as long as they are somewhere .! contiguous with the surfaceof the vase Note that spatial prepositions do not exhaust the possibilities for talking about spatial location , even in English, where placesare canonically encodedthis way. For , there exist verbs that describeposture, a kind of static spatial relationship: example stand representsthe vertical posture of an object; recline representshorizontal posture ; crouch and kneel other postures ; etc. However, becausespatial prepositions in English encode location only , they provide a well-defined domain within which to intensively examine the kinds of spatial relationships that languages encode . With that knowledge, one can compare these meanings to those encoded by other spatial terms in English (e.g., nouns such as top and bottom; adjectives such as long and wide; verbs such as stand and recline ) and to locational terms in other 2 . languages 8.1.1 English Spatial Prepositio18 The spatial prepositions in English form a relatively small closed class numbering somewhere aboveeighty (not consideringcompounds suchas right next to) . A sample list is given in table 8.1. Most of theseprepositions are two-place predicates , although there are some with a greater number of arguments , for example , among , amidst. Other languagescontain as few as one generallocational marker (e.g., ta in Tzeltal; Levinson 1992 ), and there is variability in the preciserelationships that are encoded in other languages terms : Considering prepositions only , some languages by spatial collapse several English distinctions into broader categories(e.g., Spanish en covers English in and on), while others split a single English distinction into several finer categories(e.g., German auf and an cover English on but distinguish betweenvertical

322

Barbara Landau

Temporal only during Intransitivies here there upward downward inward

since outward afterwards ) upstairs downstairs sideways

until

ago south east west left right

backwards )
away apart together north

and horizontal attachment, respectively ; Korean ahn and sok cover English in but " " " " " " ). distinguish between loose and deep or tight containment, respectively in and how to be universals however there this , , figure appear variability Despite referenceobjects are geometrically schematizedand in the kinds of spatial relationships . Theseuniversalscan be revealedby considering the geometric that are encoded restrictions imposed by a spatial term on its arguments (see , Miller and , for example Johnson- Laird 1976 ; Herskovits 1986 ; Jackendoff 1983 ) . As one example ; Talmy 1983 , the preposition in requires a referenceobject that can be construed as having an interior : If one object is in another, the latter must have some volume or area within " " " " which the object can be located. Phrasessuch as in the bowl or in the house are easily understood becausebowls and housesare easily construed as volumes. However , the abstract nature of thesegeometric descriptions can be seenthrough other ' cases , in which the preposition will coerceone s reading of the referenceobject. For

Geometric of Objects Multiple Representations

323

" " " " , in a phrase such as in the dot or in the mat the dot or mat will be example " construedas a 2-D area or evena 3-D volume (e.g., dirt in the mat" ). Thus, although the term in seems to expressstraightforward " containment" (with the referenceobject " somesort of container" ), we can useit equally well for " coffee in a cup" (where the reference object is a physical container), " birds in a tree" (a virtual volume), or " customersin a line" a virtual line . Such ( ) semanticallymotivated restrictions appear to restrictions verbs on their arguments . For example , the comparable imposed by verb to drink requires an argument construable as a continuous quantity (centrally, a liquid ), the verb eat requires an argument construable as an edible (hopefully, food), and so forth . Given coercion by the verb, we can interpret a sentencesuch as " John drank marbles," where marbles are taken as a continuous stream (cf. * " John drank a marble" ) . " This processof " schematizing objects has been describedby Talmy ( 1983 ) in his seminal work on the geometry of figure and reference object where he suggested strong universal constraints on the geometric properties relevant to the figure and referenceobject. Specifically , he proposed an asymmetry in the geometric descriptions of figure and referenceobject, with the figure often representedas a relatively blob , and the referenceobject represented more richly , often in terms of the shapeless ' s three axes . object principal orthogonal 8.1.2 Geometryof the Figure Object Taking examples from English, the prepositions listed in table 8.1 show very few constraints on the figure object. Terms such as in, on, above , below , and many others do not impose any specialgeometrical requirementson the figure object- any object of any shape , size , or type can play the role without violating the meanings of the majority of prepositions. There do exist, however, a few restrictions for certain terms. Terms such as acrossand along representrelationshipsof intersectionand parallelism , and these to a and reference ; respectively relationships appear require figure object that can be construed as a " linear" object.3 Thus sentences (2a, b) both are easily understood, whereassentence it is difficult to construe a ball (2c) is marginal because as a " linear" object. Note , however, that sentence(2d) is completely natural ; in this ' case , the ball s path (as it bounces ) becomesthe figure. (2) a. b. c. d; A snake lay along the road. Trees stood along the road. ?A ball lay along the road. A ball bounced along the road.

One further distinction mentioned by Talmy is the figure object' s distribution in space : through is used for nondistributed objects, while throughout express es

324

Barbara Landau

distribution of the object in the ground (compare " There were raisins throughout " the pudding" to " ' !fhere were raisins through the pudding ) . Aside from these few distinctions, there do not appear to be any other requirements on the geometry of the figure object for spatial prepositions in English. Nor do I know of any in the spatial prepositions of other languages , although other languages have locational verbsthat do impose shape restrictions on the figure object. For example , there is only one basic spatial preposition in Tzeltal (ta, a general relational marker), but information about an object' s axial structure (specifically, aspect ratio , or the ratio of height to width ) can appear as part of different spatial ; Levinson 1992 ) . Thus waxal- ta predicatesused for locating objects (seeBrown 1993 is predicated of objects whose opening is smaller than their height, pachal- ta of - ta of flexible bulging bags objects whose opening is larger than their height, chepel es a considerable number of , Atsugewi possess (Brown 1993 ) . As another example " figure object distinctions in locational verbs, including roots meaning small, shiny, " " " spherical object to move/ be located, slimy, lumpish object to move/ be located, " " " limp , linear object suspendedby one end to move/ be located, and runny , icky material to move/ be located" (Talmy 1985 ) . English makes similar distinctions in certain verbs (e.g., to rain, to spit), although this particular pattern of conflation is not dominant in English, according to Talmy . - in which a greater amount of geometric information is incorporated These examples into the figure object- are challenging becausethey raise the question of whether there are universal blasesin the kinds of information typically incorporated . At this point , it should be noted that into the figure object in locational expressions the degree of shape information exhibited in , say, Tzeltallocational predicates , is however . It remains to be determined than that shown , by English prepositions greater , exactly how fine-grained theseshapedescriptors are, and what role they play in . the overall systemof spatial language 8.1.3 Geometryof the Reference Object Like the figure, the referenceobject tends to be representedfairly coarsely. For certain terms, it is representedas a shapeless point or blob (e.g., terms such as near or at do not require that any specificgeometricinformation be preserved ) . For other terms, a surface as a volume (in, inside or as the referenceobject is represented (on), and for ) between for two still other terms, the number of referenceobjects is distinguished ( referenceobjects, amongor amid for more than two ) . In other languages , the orientation of the ground is distinguished (German aufvs . an), the opennessof the ground ), and direction toward or away (Korean has two separateterms for English through hin others. German her vs. from the speaker( ), among

Multiple Geometric Representationsof Objects

325

Most critically , however, a number of spatial prepositions require that the reference . The vertical axis object be construed in terms of its three principal axes (above / below ), and the two sets of horizontal axes (right/left or beside ; in front of/ behind) . These axes are also engaged by certain spatial nouns and adjectives in English: top/ bottom, front / back, and side expressregions defined by referenceto the axes /, long, thin, and wide expresssize differencesalong different axes . , and Tai The spatial nouns are marked not only for different axes , but also for different ends of the axes(top/ bottom, front / back, right/left, with the viewpoint varying application of the latter being quite difficult to learn) . " Thesespatial terms appear to be insensitiveto referencesystem . For example , The " star is above the flagpole can be used to describe a location with respect to an object-centered framework (the region near the top of thejlagpole, regardlessof its orientation ) or an environment-centeredframework (the region adjacent to the gravitational top). However, people do appear to have blasesto interpret theseterms with regard to different referencesystemsunder different conditions ( Levelt, chapter 3, this volume; Carlson- Radvansky and Irwin 1993 es different ) . At least one languagepossess setsof terms to refer to the object-centeredversusenvironment-centeredapplication of theseterms. The Tzeltal body-part systemutilizes one set of terms to refer to object parts, and another to refer to (environmentally determined) regions adjacent to the object (Levinson 1992 ). The axial representationsas a whole appear to be the richest geometric representations required by English spatial prepositions; they also playa major role in the . For example spatial terms of other languages , the Tzeltal body-part systemis massively ' , which specifiesan object s principal dependent on the object axial system dimensions , the endsof which are often labeledwith locational terms, such as " at the " " head of , at the butt of ," " at the noseof ," and so on ( Levinson 1992 ) . English also has such expressions(e.g., " at the head of the table," the foot of the bed," " the arm of the chair " ), but the Tzeltal systemis richer in its range of locational terms. Each of theseterms, however, dependson very much the samekind of analysis into principal . Levinson suggests that the assignmentof body-part terms dependson a object axes strict object-centeredalgorithmic assignmentthat analyzesthe object into its principal and secondaryaxes , and then decideson markednessusing detailed shapeinformation a clear (e.g., for top vs. bottom) . For example , a novel object might possess " head of " and " foot of " axis for which would be relevant if but one end principal , of the axis has a distinct protrusion , then that would be marked " head," or perhaps " nose " consistent with its . The rough shape parameters required for such , shape assignmentprovide a challengeto the generalization that ground objects are stripped of detailed shape elements , even though there is still quite a broad range of shape variation sufficient for assigning" nose of " to an object part .

326

Barbara Landau

The axial system thus appears to be critical to the representation of reference . Interestingly, this system has also been objects in English and in other languages posited to be developmentally complex, with children coming to representprojective geometric properties such as straight lines only during middle childhood (Piaget and Inhelder 1948 ; Piaget, Inhelder, and Szeminska1960 ) . Basedon this proposal for the axial of , a number of investigators have ) representations development nonlinguistic ( proposed that the spatial prepositions recruiting axial representationsmay be relatively difficult to learn (seeJohnston 1985for review). I return to this issuein section 8.2.2. 8.1.4 Summary The geometriesof both figure and referenceobject are relatively coarse , incorporating distinctions such as volume, surface , number, and most critically , principal axes (of either the figure object, the referenceobject, or both ) . As Talmy ( 1983 , ) suggested there appears to be an asymmetry between the figure and reference object, with the figure incorporating relatively less geometric specification than the reference object. If we consider the degreeof geometric specificationto be a dimension, English appearsto incorporate the least information in figure objects, disregarding almost all shapespecification of the figure object. At the other end of the dimension, languages such as Tzeltal appear to include more shape information , for example , grouping ' together objects by the relative proportions of the object s principal dimensions(e.g., pachal vs. waxal ) . However, evenTzeltal incorporates relatively little shapeinformation , when compared with the much richer information available to identify objects. As for the referenceobject, English again incorporates very little shapeinformation ; at the most, it engagesan axial representation of the referenceobject in order to also recruit the axial representation describethe relevant region. Other languages , but not much more. apparently, These geometric descriptions appear quite different from those which might be engagedduring object naming. The basic vocabulary for object names in English includes proper nouns (e.g., Fred, Mother ) and count nouns (a dog, a tree) . To the extent that these terms are linked with schemesfor object recognition, they would seemto require geometric representationsthat preservemuch more fine-grained spatial . information than the ones so far described How do young children appear to represent objects (both figure and reference ) when learning spatial terms, and how do they representthe sameobjects when learning ? Can they ? Are young children flexible in their representations object names , as axial, as fine grained? The following empirical evidence representobjects ascoarse . evidence for eachof thesetypes of representationin young learners positive provides

Multiple Geometric Representationsof Objects

327

8.2 EmpiricalEvidence for DifferentKindsof ObjectRepresentation among Young Learners


In order to determine whether young learners possess the different kinds of object representation underlying figure and reference object , we have conducted a variety of studies examining children ' s treatment of objects when learning novel spatial prepositions and when comprehending familiar prepositions . These studies have shown that children can ignore shape information altogether and that they can treat objects in terms of their axial representations . In addition , we have conducted a separate line of investigation to determine how children treat objects when they are learning a novel name for the object , independent of its location . These studies have shown that relatively fine -grained shape information can be used to assign objects to named categories .

8.2.1 CoarseRepresentatiol B : Scbematizingthe Figure Object - either as a Recall that in English, the figure object is generally treated quite coarsely shapeless point or blob , or (for terms such as along and across ) as a linear object, 's on the axis. Recall also that other focusing object principal languagesmay incorporate somewhatmore detailed shapeinformation into the figure object. Spatial predicates in Tzeltal include terms that incorporate information about the figure' s aspect ratio (height-to-width proportions), flexibility , and curvature, for example. Two questionsarise. One is whether young children learning a novel English spatial preposition ' will tend to ignore shape entirely (or perhaps , attend only to the object s principal axis) . If the answer to this question is positive, then one might wonder whether English-speaking children could readily learn to incorporate the somewhat more detailed (axial) information captured, for example . , in Tzeltal spatial predicates 8.2.1.1 Ignoring the Shapeof the Figure Object Landau and Stecker ( 1990 ) posed the first question by modeling a novel spatial preposition for young English-speaking children and then asking to what new figure objects and locations children would generalize this term. Three-year-olds and adults were shown a novel object " " (the standard ) being placed on the top of a box in the front right -hand comer (the " standard" location: see " figure 8.2). As the object was placed, subjects heard, See " this? This is acorp my box, using the novel term acorp in a syntactic and morphological context compatible with interpretation as a novel preposition. The entire display then was set aside , and subjects saw each of three different objects being placed in each of five different locations on and around a secondbox. One of the objects was identical to the standard, and the other two were different from it in shapeonly (see figure 8.2 for objects) . Each time subjectsviewedan object being placed on the second

328

Barbara Landau

. Figure 8.2 ) . Children and adults were shown a Objects and layout used by Landau and Stecker ( 1990 novel object being placed on the top of a box, as shown. They heard either " Seethis? This is " " " acorp my box (novel preposition) or Seethis? This is a corp (novel count noun) . Then they were shown the three different objects each being placed one at a time on and around the box in different locations. Each time, they were asked either " Is this acorp the box?" or " Is this a " ' corp? Subjectshearing the novel preposition ignored the object s shapeand generalizedon the basis of its location. Subjectshearing the novel count noun ignored the object' s location and . generalizedon the basisof its shape

" " box, they were asked , Is this acorp your box? The question was how children would generalizethe meaning of the novel term. Would they generalizeonly to the standard in its standard location? Or would they generalizethe term in a way consistent with the general pattern of English spatial prepositions- ignoring the particular shapeof the standard object, and generalizingto a range of locations? In this condition , both children and adults ignored the shape of the standard, acceptingall three objectsequally (summedover locations) . However, they did attend " to the object' s location. Having beentold that the object was " acorp the box (when placed on the top front right -hand corner of the box), children then generalizedto all locations on the top of the box, rejecting all locations off the box. Adults showed a similar pattern, also rejecting all locations that were off the box, although they were somewhatmore conservativethan the children. Someof them confined their general 4 . in the standard location front hand corner ization to any object ) right only (top One might wonder whether the context of the experiment- in which objects are being placed in various locations- might itself predisposesubjectsto ignore object . We found evidenceagainst this interpretation in a secondexperimentalcondishape

of Objects Multiple Geometric Representations

329

tion . In this condition , we followed the sameproceduresas above, with one critical , exception. This time, as the standard was being placed on the box, we told subjects " " Seethis? This is a a the same as for the novel , phonological sequence corp using preposition (acorp), but placing the new word in a syntactic and morphological context appropriate to a count noun interpretation . Subjectsthen were shown the same test objects placed in the same test locations as in the first condition , but each time , they observed a test object being placed in one of the locations, they were asked " With this " Is this a ? context as a mental to a count corp syntactic serving pointer noun reading, subjectsnow generalizedonly to the standard object, regardlessof its location , rejecting both of the objects that were not identical to the standard. That is, while subjects hearing a novel preposition (" acorp the box" ) ignored shape and attended to location , subjects hearing a novel count noun (" a corp" ) ignored location , and attended to the object itself. This pattern of findings shows that young children are capable of representingthe -grained level, completely ignoring shape . But they do figure object at a very coarse not show that children are incapable of incorporating any elements of the figure ' object s shape when learning a new spatial term. Even in English, certain terms ' , along requires a require attention to the figure object s principal axis- for example . And , as mentioned above, some Tzeltal roughly linear figure object as does across terms appear to require even more shapeinformation . Thus one might ask, How readily will young children incorporate shapeinformation into the figure object? We have approached this question through two sets of . In both , we have modeled novel spatial prepositions using figure objects experiments that possess a very clear principal axis. The question is whether such modeling might more strongly elicit at least an axial representationof the figure object. 8.2.1.2 Incorporating Axiallnformatio D into the Figure Object One experimentwas , except that different objects and locations were exactly like the one just described used( Landau and Stecker] 990; seefigure 8.3 for standard object and standard location ) . The standard object was now a 7-inch straight rod , and the test objects included a replica of the standard, a wavy rod of the same extent as the standard, and a 2" x 2" x ] " block. As subjectsheard, " Seethis? This is acorp my box," the standard ' object was placed perpendicular to the box s main axis. Test locations included this samelocation as well as one slightly to the left of it , one parallel to the box' s principal axis, and one diagonal to it . The results of this experiment again showed that subjectstended to ignore shape and generalize primarily on the basis of the object' s demonstrated location. In fact, many of the three-year-olds tested behavedjust as they had in the first experiment , ignoring object shapeand generalizingsolely on the basisof location. However,

330

Barbara Landau

.
F~ 8.3 Objectsand layout usedby Landauand Stecker( 1990 ) in a second study using the same ' in figure8.2. Subjects methodasdescribed hearingthe novelpreposition ignoredthe objects axis . the basis of its location and its detailedshape and on , principal Subjects generalized ' s exact . thenovelcountnoungeneralized on the basis of the object hearing shape sornethree-year-olds and rnost five-year-olds and adults acceptedboth the standard and the wavy object while rejecting the block. That is, they showedsorneattention to an abstract cornponent of shape , accepting objects that were s~ ciently long to intersect the box (when placed perpendicular to its rnain axis) . In doing so, these subjectstreated the two objects as sirnilar with respectto their principal axis, whereas . Thesesubjectsalso tended they disregardedthe details of their very different shapes to generalizeto the two locations in which the test object was at perpendicular intersection with the box; the horizontal and diagonal locations were considerably less favored (seenote 4) . a rnore salient principal Thus, when we rnodeledwith a standard object possessing axis, younger subjects (three-year-olds) still tended to cornpletely ignore detailed , although sornedid attend to the axis. Older children (five-year-olds) and adults shape - the principal axis. All subjects tended to attend to one skeletal cornponent of shape in this preposition condition also attended to location. This contrasts rnarkedly with . Thesesubjects the pattern shown by subjectsin a secondcondition of this experirnent were shown the sarneobjects and locations, but heard the novel term in the count " " noun context, that is, " See this? This is a corp ." When asked , Is this a corp?

Multiple Geometric Representationsof Objects

331

subjectsnow generalizedthe novel count noun to objects of exactly the same shape as the standard, regardlessof location. Thus the dissociation betweenshapeand location that we had found in the first set of experimentswas replicateo with entirely new objects and locations. This illustrates once more that children' s responses were not forced by salience(or lack thereof ) of either object shapeor location. Both children and adults were capableof generalizing on the basis of the object shown, ignoring its location. However, when learning a novel preposition, they tend to ignore the figure object' s shape , or , at best, to schematizeit in terms of its principal axis. In a relatively new approach to this issue , we have been modeling novel spatial terms using figure objects whose shape properties are representedin Tzeltal spatial . Figure 8.4 shows displays appropriate to the two terms waxal- ta and predicates lechel ta, each of which describes the location of an object. The locative ta is a relational marker and the predicates waxal and lechel each is used when locating a particular geometric figure type. Waxal is used for vertically oriented objects, for " " , a tall oblong-shapedcontainer or solid object canonically standing ; lechel example " " is used for wide flat objects lying flat (Brown 1993 ). Given that theseterms are found in a natural language , the conflation of specific with location must be All learnable. children geometry learning Tzeltal must learn the range of application of thesetwo terms, as well as quite a number of others that encodedifferent geometric distinctions. Our question, therefore, was not whether the terms are learnable, but rather, how difficult it would be for English speakersto infer such meaningsfrom a relevant modeling situation. In order to answer this question, we conducted an experiment quite similar to the studiesof novel spatial prepositions describedabove (Landau and Hseih in progress ). We introduced the experiment by telling subjects that we were interested in how , Tzeltal, might talk about locating objects, and people speakinga different language that we would use some words that Tzeltal speakersmight use . We then modeled two different locational situations. For one group of three-year-olds and adults, we modeled the meaning of waxal. As we placed a tall , oblong-shaped bottle on the " ' top right hand corner of a box, we said, See this? I m putting this waxal my " box (seetop left , figure 8.4) . For a secondgroup of three-year-olds and adults, we modeled the meaning of lechel . As we placed a wide, fla.t disk in the samelocation on " ' " a box, we told subjects , See this? I m putting this lechel my box (bottom left , figure 8.4) . The object on its box was then moved aside , and a second , identical box was placed in front of the subject. All subjects then saw a series of eight objects being placed in various locations on or around the box. Half of the objects were tall , oblong-shaped objects, and half of them were wide, flat objects (seeright column, " figure 8.4) . As each test object was placed in its location , subjectswere asked, What

332

Barbara Landau

Multiple Geometric Representationsof Objects

333

" about now? Am I putting this waxal (lechel ) the box? After the object was placed, " " they were asked again, Is this waxal (lechel ) the box? If subjectsattended to the overall shape(verticality or horizontality ) of the figure object as well as its location , then we should expectthem to generalizeto a compound of shape and position . If they had heard " waxal," they should generalize to all vertical objects in the relevant location ; if " lechel," then to all horizontal objects in that location. (And this region might be the top surface of the box, as it had been in the previous studies.) Alternatively , subjectsmight ignore the object' s overall shape , to all in located the relevant as had in done the generalizing objects region, subjects . previous studies The overall pattern of resultswas consistentwith previous findings. Subjectstended to generalize the novel term to new locations and to new objects, with children showing an overall tendency to say yes to novel object/position combinations more frequently than adults. Generalization to novel positions was consistent with previous results. Locations on the top of the box were acceptedmore frequently than those off the box, and adults tended to be more conservative than children, saying yes to the standard position and no to the position off the box more frequently than children . Most crucial to the design of the experiment, there was an interaction between the modeling condition subjectsobservedand the test objects to which they generalized . Subjects who saw the vertical standard (and heard, " This is waxal the box" ) generalized more often to other vertical test objects, while subjects who saw the horizontal standard (" This is lechel the box" ) generalizedmore often to other horizontal test objects. However, this effect was small, and there was no reliable interaction reflecting differential effectsof the standard in both object shapeand position . Examination of the individual responsepatterns shows that few subjectsactually generalizedon the compoundbasisof object shapeand position . Of the twenty adults tested , nine generalizedto all objects located in the standard position , and nine more generalized to all objects located on the top surface of the box. Only one subject
Figure 8.4 Objects and layout used by Landau and Hseih ( in progress ) . Subjects were shown either a vertical object or a horizontal object being placed on the top of a box, as shown in the left column. Subjectsshown the vertical object (upper left ) were told , " I ' m putting this waxal my box" (using the Tzeltal spatial predicate for tall oblong objects " sitting canonically" ) . Subjects shown the horizontal object (lower left ) were told , " I ' m putting this lechelmy box" (the Tzeltal predicate for flat objects lying on a surface ) . All subjects then were shown four vertical and four horizontal objects (right column) being placed on or around the box, and were asked whether each was waxal/ lechel the box. Adults entirely ignored the vertical/ horizontal aspect of the objectswhereasthree-year-olds tended to generalizeon the basisof the object' s principal axis, sometimesin combination with its location.

334

Barbara Landau

respondedin terms of both shapeand position , and this subject said yes to only the standard object in its standard position - that is, he did not generalizebeyond the modeled context. This overall pattern is quite different from that shown by the threeyear-olds. Removing from consideration the children who said yes to all queries left seventeenchildren. Of these , three children accepted all objects on the top of the ' box, and fourteen respondedon the basis of the standard object s axis. Of the latter , seven children accepted either vertical or horizontal objects (but not both ), four accepted the standard object (vertical or horizontal ) in the standard position , and three acceptedeither vertical or horizontal objects that were on the top surfaceof the ' box. Thus, while only 2 of 20 adults had consideredthe object s axis at all relevant to " " the novel spatial term, 14 of 17 children (who did not show a yes bias) did so. Not a single adult had actually generalizedon the basis of the compound axis-plusposition , while three children did so. While these results are only suggestive , the general pattern is intriguing . In this ' tended to conflate the direction of the object s axis but not adults children , , study with position . Why should children have been more likely to conflate axial information and location in this study when they had shown strong blasesin the other studies to ignore axial information ? At this point , we do not know , but it is possiblethat the contrast between vertical and horizontal objects in this experiment led to relatively strong weighting of this object property , while the contrast betweentwo long and one short object (all of which were horizontal ) in the previous study could have diminished attention to the axis. If so, this would suggestthat the parametersof the contrast set used in such studies might lead to different conjecturesabout which object dimensions are important . In real languagelearning, the linguistic contrast between such parametersmight readily serveto partition the geometric spaceso as to respect ' Tzeltal contrasts the verticality or horizontality of the object s axis. For example , because include vertical objects (waxal ), flat objects (lechel), flexible objects (pachal), and so forth , they might lead children to partition the geometric object descriptions in a different way from those invited by the partitioning of the object space in English. That even a small number of young English-speaking children are willing to conflate vertical/ horizontal axis together with location suggeststhat the learning processis not over by age three. English-speakingadults appear to be firmer in their conviction that object shapesimply should not be conflated with position for novel spatial terms. : SchematizingReference 8.2.2 Axial Representadorm Object That young English speakingchildren resist incorporating axial information into the figure object raises the question of whether they show similar limitations for the referenceobject. As describedin section 8.1, languagestend to incorporate a greater

Multiple Geometric Representationsof Objects

335

degreeof geometric detail in the referencethan the figure object. In English, terms such as in front of/ behind , and right/left representregions surrounding , above / below an object, with the particular region defined in terms of the object' s three principal . Identifying sucha region and mapping it to its respectiveterm might orthogonal axes seemsimple. The observer can derive the three axes , extend them outward from the . object, and establish regions centeredon thesevirtual axes In fact, establishingthe relevant regions for such terms requiresconsiderablestructure - that is, representationsand rules to ensure that on the part of the observer the correct axes are found and that they are extended in a linear fashion from the , reported in Jackendoff, chapter I , this volume, and object itself (seeNarissiman 1993 Levinson 1993for some rules of application) . The object axesare not given directly in the stimulus, although many theories of visual object representation suggestthat ' ' , hence to recovering an object s axes is critical to reconstructing an object s shape ; Leyton 1992 ) . The axial representations that must be recognizing it (Marr 1982 extendedoutward from the object are not directly given in the stimulus either; here it would seemcritical to acknowledgethe role of spatial representationinconstructing . theseextendedaxes Theserepresented axesmight be difficult for the learner to construct. According to ), the representation of axes does not emergeuntil Piaget (Piaget and Inhelder 1948 well into middle childhood. Moreover, a number of studies have shown that terms such as in front of and behind are not completely mastereduntil around age four or evenlater; this comparesto terms such as in or on, which appear much earlier and do . A prominent view of this difference not appear to undergo much developmentalchange in acquisition time is that object axes are difficult to represent ; in addition , the use of reference be difficult (compared to mastering changing systemsmight quite in or on which do not such see Levelt and , Tversky, , ; using engage systems Logan and Sadier, chapters 3, 12, and 13, respectively , this volume, for discussion of the com' ) . Consistent with this view is Piaget s argument plexities of referencesystem usage that representation of the straight line is not achieved until middle childhood , and that sensitivity to viewpoint differencesis not complete until this time (Piaget and Inhelder 1948 ; Piaget , Inhelder, and Szeminska1960 ) . Both of theselimitations would ' impose serious restrictions on the child s ability to learn terms requiring representation of the object axis, and in particular , terms that require extension of the axis outward into space(see , for example , Johnston and Slobin 1978 ). The empirical results from acquisition studies have indeed suggestedthat these . It is not obvious terms appear later than other terms not requiring axial representation , however, that this is the result of a representational problem in the child. They could be due to more data-driven causessuch as morphological complexity, form meaning transparency (e.g., the difference between in back of and behind) or even

336

Barbara Landau

input frequency. In English, in and on are ranked among the 20 most frequent words, while behindis ranked 450th (Francis and Kucera 1982 ). studies have shown that very young children two more to the separate point , Perhaps - neverthelessappear - who have not completely mastered in front of/ behind to possessrepresentationscongenial to the mature understanding of these terms. Levine and Carey ( 1982 ) gave two -year-olds a linguistic task in which they were to " in front of " another and a , nonlinguistic task in which they were place objects " " to place dolls and toy animals on a table such that they could either talk to each other or follow each other in a parade. Even the youngest children tended to orient the toys properly , suggestingthat they recognizedthe fronts and backs of the objects and knew how to align them with each other. In a separateset of experiments , Tanz in front of one children make errors showed that when placing object ) young ( 1980 " " or behind another, theseenrors tend to cluster around cardinal points, that is, the ' that very young children . This again suggests endpoints of the objects principal axes be accessed for learning spawhich can of axial representations objects may possess . tiallanguage These observations motivated us to investigate in detail the nature of young chil dren' s representationsunderlying the single spatial relationship encoded in English " " by in front of . We had two principal questions. First , we asked whether young children possessan axial representation of objects that could support learning of theseaxis-based spatial terms, and critically , whether this axial representation permitted extension of the object' s axes to the larger region surrounding the reference -based ) properties of the , we asked whether certain structural (shape object. Second reference object might more readily invite an axial interpretation . A number of studies have found that young children are especially poor at assigning fronts or backs to objects that themselveshave no intrinsic orientation (e.g., trees , balls, etc.; axial representations seeKucjaz and Maratsos 1975 ; Tanz 1980 ) . If children do possess for spatial term learning, then there still may be conditions that can be accessed under which this accessis impeded, for example , for objects whose principal axes . from a geometric analysisof their shape are not clearly accessible In the experiment, we showed two -, three-, and five-year-olds and adults one of three different referenceobjects placed flat and directly in front of them on a table ; seefigure 8.5) . One referenceobject was Ushaped , and because (Landau, in progress a clear principal axis; a second reference of its proportions and symmetry, it possessed no principal axis; and a third reference object was round and therefore possessed " " " " , but was marked with eyes and a tail (simple object was identical to the second ) . Theselatter properties might induce assignment piecesof fabric glued to the surface of a principal axis, and might therefore induce better performance than the round object. Subjects were tested on one of these reference objects; comparison across

Multiple Geometric Representationsof Objects

33

Figure 8.5 Three referenceobjects usedin study of the structure of regions. Objects were presentedin the horizontal plane in front of subjects . Subjectswere asked to place objects " in front of " each referenceobject and to judge what locations were acceptableinstancesof being " in front of " each. Referenceobjects varied in how clear an axis they exhibited. The U-shapedobject possessed a clear principal axis, the round object possessed no such axis, and the round object with " " " " cuesto indicate the probable location of the principal axis. Variation eyes and tail possessed in these properties affected the nature of young children' s and adults' judgments of the " " region in front of each.

referenceobjects would determine whether cues to the location of the principal axis " " (in the case of the Ushaped or eyes objects ) might induce better performance the children. among youngest Three- and five-year-olds and adults performed in a yes / no task in which they were shown a range of small novel objects (the figure) placed in a variety of locations around the referenceobject and were asked to judge whether the figure was " in front of " the referenceobject. Each of the figure objects were placed one at a time in each of the four cardinal locations plus a fifth , directly on top and in the center of the referenceobject (seefigure 8.6) . Each time the small object was placed in a location, " " , Is this (figure) in front of this (referenceobject)? (indicating subjectswere asked each object at the relevant moment) . Following ten such trials , the object was placed in (up to 16 ) additional locations in the region fanning out from the side of the object facing the subjects Figure 8.6 shows all 21 locations, separated into regions that ' correspond to (A ) the broad rectangular region following from the object s principal axis and ( B) the broad triangular region surrounding this. Locations were probed in a particular order, as indicated in figure 8.6, in order to ensure obtaining responses for critical areas such as the region closest to the object (locations 6, 7, 8, 9), the ' region extending directly from the object s principal axis (locations 10, II ), the , 18 , 19 regions surrounding the axis (locations 12, 13 ) and the regions farther away from the referenceobject (locations 14 15 16 17 20 , , , , , 21) . the entire yes Following / no procedure, subjectswere assigneda placement task in which they were given a seriesof four small objects with no distinguishing features

338

Barbara Landau

Figure 8.6 Layout of locations probed in regions task. Subjectsfirst were querled on locations 1 5, each time twice, followed by one query each on locations 6 20, in numerical order. The locations " " shadedwithin the block representthe proposed canonical region for the term in front of and adults. The locations shadedwithin olds and olds five three were widely acceptedby , year year " " the triangular area surrounding that block (the external region) tended to be lesspreferred " " by children and adults, except for the eyes referenceobject, which elicited a high proportion ). of acceptanceby adults (seefigure 8.8 for comparison of canonical and external regions

Multiple Geometric Representationsof Objects

339

and were asked to place each " in front of " the reference object. Three separate groups of two -year-olds, one group for each referenceobject, were also assignedthis placementtask. The results for the placementtask are shown in figure 8.7 for each of the reference ' objects. Individual dots representdifferent subjects placementof the first object they were given. The most frequent responseacrossall ageswas to place the object in line with the referenceobject at one of its cardinal points, that is, the point at which its front -back or side-side axis would have projected into the space surrounding the object. For easeof representation , this is shown in the figure as subjects lined up to each other. The adjacent pattern becomesstronger over age ; but the major developmental to occur between the ages of two and three. At age two change appears most children place the object at one of the cardinal locations, especially favoring both endsof the object' s front / back axis. The secondmost common pattern occursat both agestwo and three, and finds children locating objects at the end of the side / side axis. Athough there is some diffusenessin the responsesof the two-year-olds, this disappearsby the age of three. Note that both children and adults do vary somewhat in their preferred location for this initial placement . Even someadults consideredthe far end of the object to be their first choice for locating the figure in front of the referenceobject (a pattern that is the preferred one in Hausa; seeHill 1975 ) . This variability occurred only for the Ushaped and round referenceobjects, however. Adding eyesappearedto drive subjects of all ages to locate the figure object directly along the half axis extending outward from the eyes . This is consistentwith previous findings suggesting that young children more often correctly place objects in front of or behind objects with clear fronts and backs (Kuczaj and Maratsos 1975 ). The results of the yes / no method tell a similar story about the cardinal locations. When subjectswere asked to judge whether a small object was in front of the reference object, they tended to say yes to locations 1 and 3- the locations falling at the two ends of the front / back axis. This pattern of accepting both 1 and 3 (in " " " " English, the canonical locations for in front and behind ) occurred almost always with the Ushaped and the round reference objects. The round object with eyes " elicited " yes responses only to 1, the location directly adjacent to (i.e., in front of ) ' second the eyes . Trailing behind 1 and 3 as the subjects choices were locations 2 and 4 the locations falling at the ends of the side/ side axis. This pattern was most prominent among three-year-olds judging locations around the round referenceobject " reference , and leastprominent among adults judging locations around the " eyes object. The relatively high acceptanceof locations 2 and 4 among three-year-olds judging the round object suggeststhat lack of a clear object axis invited subjects to entertain more than one axis as the relevant one for determining when one object was

340

Barbara Landau

218

[il ] . . . . .[ijJ a . . . .1 1 .
0

: RF6TTFiAL ~ T TASK PLAC

()
. . .

.: .
315

..

.
I

I
51s

I .
Adults

8 0 81 8 I I . 6 I I

Multiple Geometric Representationsof Objects

341

in front of another. That is, the strong axis-based responsesfor the Ushaped and " " eyes referenceobjects suggestthat young children are quite capableof representing an object' s axis; their relatively poor performancewith the round object suggests that objects lacking a clear principal axis may be lesseffective in allowing young subjects to show their knowledge. Finally , the analysis of the regions surrounding the cardinal points showed that subjectsof all agesgeneralizedtheir interpretation of " in front of " to regions with a well-defined geometry that was basedon extension outward of the object' s principal " axis. Figure 8.8 showsthe proportions of " yes to the two different regions. responses " " The canonical region representsthose locations falling within the rectangular region extendingdirectly outward from the front edgeof the referenceobject (seefigure 8.6) . The " external" region representsthe triangular region extending outward from the front edgeof the referenceobject and surrounding the canonical region. Three noteworthy trends appear in figure 8.8. First , subjects at all ages and for all referenceobjects accept the canonical regions more frequently than the external regions. This suggeststhat even the youngest subjects represent the region directly adjacent to the referenceobject as the preferred region for the spatial relationship encoded by " in front of ." Second , there appears to be growth in the size of the canonical region over age. Adults accept a greater proportion of the locations as of " in front of " than do children. The causeof this differencecannot legitimate cases be ascertainedfrom the figure; but is due to expansionaround theprincipal axis rather than by a random increasein the acceptanceof locations or by expansion outward from the front edgeof the object. That is, younger children tend to accept positions directly along the axis while adults accept the entire canonical block. Overdevelopment " " , the region for in front of does not becomelarger by seepingoutward from ' the object s edge ; rather, it becomeslarger by expanding outward from the object' s extended(virtual ) axis. ' The third significant developmentconcernsthe subjects treatment of the reference with relative to the other . , object eyes objects Although the regions appear to be similar for the Ushaped object and the round object at all ages , the pattern differs for " ." Three-year-olds show the samepattern for eyesas they do for the other two eyes reference , although their preferencefor the canonical over the external regions objects is slightly more pronounced for eyes . Five-year-olds show an overall dampening Figure8.7 ' first " Can in response to thequestion Subjects placements you put X in front of the(reference " at all ages showed a preference for oneof the four cardinallocations )? Subjects object , each locationarisingfrom extension of axes on the object . mentallyimposed

342
Regions

Rarhara

Landau

[1]
3'5
0 . 1

402 1

46 1
-

Canonical External

Adult s

Figure8.8 " " " )? to the question" Is this is front of (the reference object Proportionsof yes responses " " " " reference each object surrounding arewithin the canonical and external regions Locations canonicalloca the at all . locations these of for details preferred 8.6 ages see ) Subjects ( figure " " . There , they object tions to the externalones , exceptfor adultsjudging the eyes reference

: . 5'8 F

Multiple Geometric Representationsof Objects

343

" of " yes , both for the canonical and external region. Inspection of individual responses responsesby the five-year-olds suggeststhat the critical change is in the canonical region, where a number of subject insist that the only acceptablelocations are those falling along the extensionof the axis. This conservatismcausesan overall " reduction in " yes . Finally , adults show an overall increasein the size of responses " reference the external region for the " eyes object. This appears to be due to their ' that the relevant is affected assumption region by the object s status as an animate. A number of subjectswere quite explicit on this issue , remarking that the object could " look over this " way (indicating a location in the external region) . The idea that the geometry of a region in front of an animate object might be different from that in front of an inanimate one is reminiscentof Michotte ' s ( 1963 ) notion . There may be a of wherein to themselves not only an object' s region reactivity perceiversrepresent region of influence when static, but also the region from which it potentially could influence another. Whether or not such regions are also part of young children' s representationsis an intriguing question. To summarize , the foregoing studies strongly suggestthat young children can and do representthe principal axesof referenceobjects by the age of two. The geometric structure of the referenceobject itself has some effect during the early years , but by and large, young children appear to be capableof setting up object axesevenin cases where the perceptual clues to the location of the axesare weak. By the age of three, theseaxial representationscan be extendedoutward from the object and can serveas organizing referenceframes for setting up regions relevant to basic spatial terms such as in front of or behind . Theseregions seemremarkably similar to those describedby and Tversky by Logan and Sadier (chapters 12 and 13 , this volume) among adults in and attention tasks see also participating imagery ( Hayward and Tarr 1994 ) . To the extent that they differ from those of adults, the children' s regions appear to be more narrowly defined with respectto the object' s axis. Thus, contrary to the pattern ' predicted by Piaget s theory, the development of regions appears to begin with the axis, and broaden with development . Although the geometric and conceptual nature of the referenceobject may modulate the geometric details of the relevant region, these effects seem to be imposed on a basic pattern in which any referenceobject can be representedin terms of its axes and surrounding regions. This basic pattern 8.8 (continued Figure ) all locations from the reference accepted , asif theyobjectwasmobile fanningoutwards object ' -year -olds and couldexplorethe environment . Five in acceptance of the canonical depression " to from their reluctance to say" yes those regionstems anylocations except alongtheextended axisitself(locationsI , I 0, and II in figure8.6).

344

Barbara Landau

appears to exist quite early in life and is mapped onto the corresponding spatial terms

betweenthe agesof two and three. . . : SchematizingObject Kinds 8.2.3 Fine-Grained Representatio ' focus of this the chapter so far has been the young child s ability to sche Although bloblike or coarse axis based either skeletal matize objects in terms of ) geometric ) ( ( descriptions, there is strong evidencethat children are not limited to these descriptions . However, these but can also representobjects in terms of rather detailed shape in the namesof children are to when tend learning engaged emerge representations than their locations. rather objects, Recall that in some of the experiments described in section 8.2.1, children and adults were shown an object being placed in a location on a box. In one condition , the scenewas describedusing a novel term in a syntactic and morphological context suitable to a preposition. In this context, neither children nor adults generalizedthe ' . However, in another condition , the novel term on the basis of the object s shape scenewas described using a novel term in a syntactic context suitable to a count noun, as if the object itself (and not its location ) was being named. In this context, both children and adults generalizedthe novel word only to objectsof the sameshape of its location. Attention to object shapeduring object as the modeled one, regardless . naming has beendemonstratedin a variety of other kinds of studies In many of these studies , young children (two -, three- , and five-year-olds) and " adults are shown a novel object and they hear it labeled, for example , Seethis? This " is a dax. Then, with the novel object still in view, subjects are shown a series of " " additional novel objects and asked for each , Is this a dax? In another version of this study, subjectsare shown the novel object, and hear it labeled, but then they are " " shown pairs of objects and are asked , Which one is a dax? The results from these two methods tend to convergeand suggestthat object shapeis a privileged perceptual . property that is engagedwhen young children are learning object names A variety of evidenceover the past twenty years has hinted at this pronounced ' . Clark ( 1973 role of object shape ) reported that children s early overgeneralizations " " tended to be based on shape , when a child calls the moon ball , , as, for example " or a dog " kitty -cat. In another context, Rosch et al. ( 1976 ) argued that our basic level categories setsof objects named by count nouns in responseto the question " What is that?" - are . organized in terms of certain key properties, including shape A number of developmental studies have shown that children find it easy to learn -basedcategories(Bomstein 1985 namesfor shape ). the role of object shapein the developmentof object A systematicattempt to asscss ' Landau Smith, and Jones( 1988 was , ), who compared children s reported by naming , and texture in generalizing the novel count noun to new , size weighting of shape

Multiple Geometric Representationsof Objects

345

objects. In the basic experiment, children were shown a novel object and heard it labeled, then they were shown objects varying from the standard in either shape , size , or texture. Each time, they were asked whether the test object was also a member of the named category (e.g., " Is this a dax?" ) Both children and adults tended to weight shapemore strongly than either sizeor texture. For example , when told a novel object was " a dax," subjects then generalized the word dax to other objects having the sameshapeas the original object, even if they were much larger than the original or a quite different surface texture (seefigure 8.9) . possessed In this study and severalfollow -ups, a developmentalpattern emerged : The " shape " bias appears to be weak among two-year-olds, moderate among three-year-olds, and quite strong among five-year-olds and adults (see , for example , Landau, Smith,

c Shape Changes NO :? r!lJ l !=


Sponge

Figure 8.9 When children and adults are shown a novel object and hear it labeled, they tend to generalize the object' s name to others of similar shape of sizeor texture. After Landau, Smith, , regardless and Jones( 1988 ).

Wire

Chicken

Cloth

Standard " (2 wooden)

"

'

"

"

"

Changes

Texture

1988

Bias Jones and , Shape Smith , t [ . 2

The ( Landau nges YES : ~

346

Landau Barbara

and Jones 1992 ) . For example, adults reject even a small change in shape from the but acceptan object of the sameshapethat is ten times as large. The younger original the child , the more willing he or she is to accept objects of different shape ; although by the age of about two , children show a reliable tendency to generalizeon the basis of sameshapein the naming context (Landau, Smith, and Jones 1988 ) . Recentstudies indicate that the growth in the shape bias is correlated with children' s productive vocabulary. The bias appears to begin around the time when children have fifty words in their vocabulary, suggestingthat the bias may becomesharper as children learn more about which properties are the best basis for generalization ( Joneset al. 1992 ). That is, because many words do indeed refer to objects sharing the sameshape . e. the basic level terms common in maternal input), an input bias may act in ( g, ' concert with the children s own representational blases ; the children may sharpen their conjectures as they learn that words for objects may safely be generalized to -shapeobjects are often of the samekind , other objects sharing the sameshape . Same hencesuch a generalization would in generalbe safe.6 , the preference for Although the shape bias emergesquite early in development sameshapeis highly context-dependentamong both children and adults. The particular pattern of context dependencesuggeststhat the bias is closely linked to the representation of objects and, in turn , object naming. By the age of two , the shape bias appears most robustly in the object-naming context, while in other contexts, . For example , Sola, Carey, and Spelke young children show different preferences ( 1991 ) found that two -year-olds showed a strong shape preferencewhen shown a rigid object, but a very weak shapepreferencewhen shown a massof gooey substance to Sola, Carey, and Spelkethat young (seealso Subrahmanyam 1993 ) . This suggested - in this case children bring to the language learning task certain a priori categories , object and substance whose existencemight constrain the range and type of inferences , children who they can project from a single exemplar. As another example have learned a bit more syntax can be guided by syntactic and morphological information to attend to properties other than object shape . Landau, Smith, and Jones ( 1992 ) showedthat somethree-year-olds and most five-year-olds tended to generalize on the basis of object shape when instructed with a count noun (" This is a dax" ), but tended to generalize on the basis of surface texture when instructed with a novel adjective (" This is a daxy one" ; see also Smith, Jones , and Landau 1992 ). Subrahmanyam ( 1993 ) showed similar effectsamong three- and five-year-olds using count nouns that guided attention to shapeand massnouns (" This is somedax" ) that ' . , often to substance guided children s attention away from shape -basedrepresentations What are the geometric properties of theseshape ? One striking fact is their level of detail, compared to those representationsrecruited forlocating -based representation appears objects. When an object is being named, its shape

ultlple Geometric Representationsof Objects

347

to preservea good deal of detailed infonnation about its part structure and arrangement (while such elementsseemto have only limited relevancefor locating objects) . For example , in the studies described above, adults who were shown aU -shaped object tended to reject objects with evena small defonnation in overall shapedcaused by bending one of the object parts slightly outward. In the studies described in section 8.2.1, children and adults who were shown a straight rod and heard it named tended to reject a roughly linear object of the samelinear extent; this object did not qualify as a member of the same named category, apparently becauseits overall contour was wavy (as compared to the straight contours of the modeled object) . The range and degreeof detail necessaryto include objects into the same named category is as yet unknown. However, many modem theories of object recognition propose a strong role for object parts as components of object shape(Hoffman and Richards 1984 ; Leyton 1992 ; Biedennan 1987 ); and it is the arrangement of these that would seem to what we call an object' s " shape ." Further , the parts capture specific arrangment of parts will be subject to some variability or range, as many objects possess parts that regularly undergo motion . Although little is known about the range of object-internal motions that must be captured by theories of shape , there do exist models for characterizing limited classes of motions (see for , , Marr example and Vaina 1982 . ) Because of the importance of both part articulation and part motion in the characterization of object shapeand in theoriesof object recognition, one might expectthat . Several young children would respect both in their generalization of object names recent studies suggestthey do. In one seriesof studies , Landau et al. ( 1992 ) sought to detennine whether children would make different predictions about the range of acceptableshape transfonnations relevant to a named object, depending on its part structure, especially as it interacts with imputed malleability . Three-year-olds and adults were shown a novel line-drawn object, heard it named, and then were asked which of a set of shapeand sizetransfonnations were also membersof the namedcategory. In a first experiment, subjectswere shown a rigid -looking object with sharply delineated part boundaries more extremeshapechanges (figure 8.10 ) . They were testedon three successively , and three successivelylarger size changes . As in most of the studies on object shape and object naming, both children and adults extended the novel label to objects of different sizesfrom the standard, but did not extend the label to objects of different . shape In subsequentexperiments , subjects were shown standard objects comparable to thosefrom the first st:udy , but whosepart structure and suggested rigidity was altered. For example , subjectsin one experiment saw an object identical to the standard of the ' first experiment, except that it possessed curved edges , which weakenedthe object s

348

Barbara Landau

Standard

No

No

Yes Yes No
Yes Yes Figure8.10 ' Children and adults judgmentsof which objectsbelongin the samenamedcategoryare . When angularobjectswere shownand named affectedby subtledetailsof object shape , was tended to even small , aspart structure (top"panel ). However reject shape changes subjects moreshape or adding" wrinkle, subjects tended to accept weakened , eitherby curvingedges wereadded , subjects shape ). Wheneyes accepted quitesubstantial (middletwo panels changes that theyobjectcaneasilybedeformed , asif theynow assume (Landauet al. 1992 ). changes

Geometric Representationsof Objects Multiple

349

part boundaries and suggestedmalleability (figure 8.10 ) . In another experiment, a of saw an object identical to that of the second separategroup subjects , except that it possessed massively textured edges , further suggestingnonrigidity . And in a fourth " experiment, different subjectssaw a curved and " wrinkled " object with " eyes placed at one end. This last type of object was meant to test whether certain powerful ' cues to object kind (in this case , a cue to animacy) would affect subjects judgments -changedobjects could still be membersof the named of which shape category. The results of the four experiments showed massiveeffects of part structure and suggested rigidity . Although subjectshad generalizedsolely to sizechangesin the first experiment, progressiveweakening of the part boundaries (and correlated destruction of cues to rigidity ) led them to generalizeto shape changesas well. Both the curved and curved/ wrinkled objects led subjects to accept a moderate number of . When eyeswere added to the objects, subjectsgeneralizedto all shape shapechanges as if , changes they now assumed that the object was a tubelike, nonrigid object of internal motion . Thus, as rigidity and strong part boundaries were successively capable , subjectsbecamemore and more willing to accept a larger range of destroyed " -shape . This suggests that a bias for " same shapechanges objects must engageobject representationsthat admit of flexibility in the face of varying rigidity and changing part structure (seealso Becker and Ward 1991 ). In a different seriesof experiments , we have beeninvestigating children' s inferences about the kinds of shape changesthat might obtain under mechanical transformations . In one of these experiments , we showed children a novel object that was of distinct composed parts arranged in a particular configuration (seefigure 8.11 ). One group of subjectswas shown each of the three standard objects, heard it labeled " " (e.g., This is a dax ), and then was shown a set of new objects whoseconfiguration would obtain if the standard object' s parts were capableof motion . Another group of subjects was shown objects of the same configuration, but this time, subjects also saw one part of eachobject undergoing a small motion . (Objects undergoing rotation now had hinges at their joints ) . All subjects were then shown the same set of test objects, which were possible motion -based shapechangesof the standard. Children and adults who saw the standard object with no motion generalized to few shape . However, those who saw the standard undergoing a small amount of part changes motion generalizedquite freely to the novel configurations, eachof which was consistent with a more extensiverange of object part motion . These studies begin to suggestthat the spatial system underlying object naming . In particular, it must incorporate a incorporates information about object shape relatively detailed (possibly hierarchical) representationof object shape , in which the ' object s parts, their spatial relationships, and their ranges of relative motion are . Theserepresentationscould provide a powerful systemthat would allow the present

350

Barbara Landau

No Motion

Motion

Figure 8.11 Children and adults are sensitiveto the range of shapetransfonnations likely when an object has pennanently fixed versus moving parts. Subjects who viewed an object with no motion (left) and heard it named then tended to reject objects with even small changesin configuration as an instance of the named category. In contrast, subjectswho viewed an object undergoing that would be the product of more extensive a small motion then accepted shape . changes motions.

young child to link up an object name with its shapeand to generalizeto classesof transformed shapesconsistentwith certain principles of object constancy. Theserepresentations seemquite different from those engagedwhen young children are learning ' that describean object s location. terms or using

: TzeltalmaybeException Cha Notesanda Possible 8.3 Some Uenge Croatinguistie


that Proves Rule

different kinds that young children possess The empirical evidencereviewedsuggests are engaged children of object representations , each of which is selectively active when in tasks involving different parts of the vocabulary. The detailed representations of object shapethat seemto be engagedwhen young children are learning object

Geometric Representationsof Objects Multiple

351

names do not appear to be engagedwhen children are learning words for object locations. But one might object that all of the experimental evidencereviewed thus far has concernedchildren speaking English. Considering the variation in how locations are over languages , one should be suspiciousof conclusionsbasedonly on one expressed . Moreover evidence on the structure and acquisition of other languages , language - well before the age of two years- have begun to that children suggests very young form spatial linguistic categoriesconsistentwith those found in their native language . For example , Choi and Bowerman (Bowerman 1991 and chapter 10, this volume; Choi and Bowerman 1991 ) have found that children learning Korean are likely to " " " " distinctions between respect tight fit and loose fit contact and containment relations ignored by children learning English spatial prepositions (though of course English-speaking children must respectsuch distinctions when they learn adjectives such as tight and looseor verbs such as to fit ) . Suchcross linguistic differencespoint to a strong role for early learning, but they do not invalidate the search for the universals that underlie the expression of spatial . Continuing with the examples provided by Bowerman (chapter 10 language , this volume), children learning Korean , Dutch , and English all differ somewhat in the range of object types that are included in basic spatial notions expressedin English " " " " by the action of putting in compared to putting on. Korean distinguishes between " " " " degreeof fit and among actions involving putting on different types of clothing . Dutch distinguishes between various types of attachment all covered by the English preposition on. Other languagesmake yet further distinctions that are not found in English. For example , as noted earlier, German distinguishes two types " " of contact (on) relationships by the orientation of the reference object (auf for gravitational contact, usually horizontal , and an for nongravitational contact, such as attachment to a wall ) . A number of languagescollapse the distinction between locational and directional terms that is made in English by in versus into ; for example , Russian has a single term, v ( vo before certain consonant clusters ), covering both , as does Italian . In other cases , it is English that collapseslocational and directional " meanings (e.g., English over can be locational , as in The plane was over " " " whereas the house or directional , as in The plane flew over the house other ), the two languagessplit (e.g., German " ber can be either directional or locational. but obencan be locational only) .7 Yet none of thesedifferencesseemto provide major counterexamplesto the claim that the figure object tends to be representedas a point , blob , or line, that the reference object tends to be representedas those or as a set of orthogonal axes , " and that the geometriesof both figure and referenceobject are considerably " sparser (in terms of shapedetail) than the representationsof thesesameobjects as members

352

Barbara Landau

by Talmy of categories , named by nouns. Could this be a universal, as first suggested ? 1983 ) ( Some recent evidence from Tzeltal might appear to provide counterexamplesto such a universal. This language has often been described by investigators as particularly " visual" in that it appears to encode a large range of shape distinctions in -class items, including locational terms.8 For example , there are predicates its closed " and others that describe" horizontal " that apparently describe bulging bags , sitting " ) . Tzeltal has therefore been offered as a counterexample things, lying ( Brown 1993 to the notion that very little shapeinformation is encoded in the figure and ground for purely locational terms (Levinson 1992 ). The evidencecomes primarily from the Tzeltal body-part system , which usesanimal to the names and human body part terms to assign spatial regions of objects, regions that are encoded in English by terms top, bottom, front , back, and side. " " For example , the term for head is usedto describethe tops of objects and the term " " for bottom is used for the bottoms of objects. So much is actually quite similar to " " " " " " " " English. We often refer to the head or foot of the table, or the arm or leg of a chair. However, Levinson claims that much more so than in English, the Tzeltal to particular body-part systemusesspecificelementsof shapeto assignparticular terms " would be used " nose for the term Levinson to locations. For example , , according to locate something at an object part with a particularly sharp protrusion , whereas " the term for " mouth would be used to locate at an object part with an edge or " orifice, and the term for " tail would be used for long thin protrusions. Does this mean that fine-grained shapeinformation is part of the spatial meaning of the locational term; and that this therefore erodes the shapedistinction between objects-as-named and objects-as-located? I believenot. The shapedistinctions do not appear to be part of the spatial meaning of the term, but rather, are distinctions used ' to identify particular regions relevant for the term s meaning. To put it another way, the body-part terms do not appear to refer to the distinctive shapesof , say, nose or tail poisedon someobject (though they would if they were usedas nominals) . Rather, when usedas locatives, the terms refer to spatial regions whose locations are defined with respectto some salient geometric property . The meaning of the term (i.e., what used to region of the object it maps onto ) is separatefrom the geometric algorithms " head" of the from English, assign the term to the object. To take an example ' the table is a region at the end of the table s principal axis; which end is usually decided by a variety of criteria (e.g., where the Queen sits) . Just becausethe term " " headis used to name the region does not mean that each and every head of a table . must be similar to a real head in shape . According to Levinson ( 1992 The caseof Tzeltal seems ), Tzeltal assigns analogous most body-part terms not by a metaphorical extension , but rather, by a strictly

Geometric : Representationsof Objects Multiple

353

geometric algorithm that analyzes the object into its major components and their relative orientations. Thus the location of the region " head of " an object is defined with respectto the object' s principal axis; the axis is found by using properties such as elongation, protrusion , flatness , and symmetry- properties that are likely to be universally important in such assignments(seeJackendoff, chapter I , this volume; ) . As a (perhapsnecessary ) bonus, suchproperties are in generallikely to Leyton , 1992 remain robust over a variety of viewing conditions (e.g., blurring , rapid exposure ), thereby supporting the assignment of axes and directions to objects during a wide 9 variety of spatial tasks that humans normally perform . What kinds of counterexampleswould disconfirm the hypothesis that both figure and referenceobject do not contain any particular shapeinformation necessaryfor describing the region? As suggested by Landau and Jackendoff ( 1993 ), one should not to find terms that to expect any spatial correspond spatial relationships holding between specific volumetric components or specific arrangements of components . Such examplesmight be found , of course, in some languages , and this would neces , possibly suggestinga restricted set of sarily lead to modification of the hypothesis , however, the evidence shapeproperties that is relevant to spatial terms. As it stands from Tzeltal doesnot suggestthat spatial terms map onto specificcomponent shapes . Rather, it suggests that using spatial terms requires being able to locate the relevant ' ). This is as true of Tzeltal as it is of region (usually dependent on the object s axes and of all natural . Thus Tzeltal, rather than providing , English presumably languages a striking counterexampleto the general claim that the specificsof shapeare absent from the figure or referenceobject, may provide a particularly compelling exampleof how vast apparent cross linguistic differencesmay ultimately rest on deep similarities in how languagemaps onto spatial representation .

8.4 Structure : Some . , Function , andMechanism Possibilities , More Questio


What causesthe differencesin geometric representationbetweenfigure and reference object on the one hand, and objects as named on the other hand? Severalkinds of . explanation suggestthemselves One possibility is that this difference reflectsnothing particularly interesting about either language or spatial cognition, but rather, is a direct consequence of how the world is. Objects in the world actually do come in an astounding variety of shapes , and objects in particular named categorieshappen to share greater overlap in shape than they do with objects in different categories( Rosch et al. 1976 ). Locations in the world do not possess a three-dimensionalstructure , but they do possess shapethemselves that demands encoding in terms of three principal axes . Perhaps object shape doesnot matter to location because locations do not demand suchinformation . spatial

354

Barbara Landau

. Although it is certainly true that As stated, this possibility seemswrongheaded that locations come specifiedmetrically true and also in , objects come many shapes in three dimensions , it is not a foregone conclusion that all organisms will encode objects and spatial relationships in just this way. Why not encodeobjects in terms of ? Why not encodelocation in terms of generalproximity relative size , rather than shape - things closeenough to reach, far enough not to , without regard to the to oneself ? Given that there are different possibilities for how we represent objects three axes and places , the question is, What gives rise to the particular way in which we do ? Why do we attend these aspectsof space for the purposes or language represent to shape when naming objects, but (basically) ignore it when locating those same ? The structure of the world surely imposes some constraints on our representational objects ; but these systemsare not direct reflections of some objective system " the world out there." More of plausible is the possibility that our representational description in have evolved responseto constraints on both the physical systems . achieve we must world and on the tasks How , then, do we repre ~ent the world? The study of spatial language can tell us how we representthe world linguistically ; but does this have any bearing on how we representthe world nonlinguistically? Are there any communalities betweenthe representations , and their nonlinguistic underlying the language of objects and places counterparts? Is the structure of spatial language driven at all by the structure of spatial cognition? There appear to be severalintriguing parallels betweenspatial languageand spatial cognition that suggestpossible relationships. One parallel concerns the separation between object and place in language , on the one hand, and that found through " " " " , on the other neurological and cognitive studies of the what and where systems fuller discussion 1993 for Jackendoff Landau and 1982 see Mishkin and ; (Ungerleider the existenceof two systemsin monkeys of this parallel) . A variety of evidencesuggests " " and in humans, one specializedfor the task of object identification ( what ) and " " the other for object localization ( where ) . For example, experiments on monkeys have shown selectivedeficits in the two tasks. Damage to the inferior temporal cortex appearsto disrupt object identification (but not object localization), whereasdamage to the posterior parietal cortex disrupts various localization tasks (but not object recognition) . Thesecortical areascontain neurons with quite different receptive field properties. Those in the inferior temporal lobe have a large receptive field falling within the fovea and are driven by complex sets of features ; those in the posterior fovea and are insensitive include the does not field that a lobe have receptive parietal 1982for review) . Mishkin to such features(seeSchneider 1960 ; and Ungerleider and two streamsof Converging evidencefrom human psychophysicalstudies suggests " " is specialized . The bifurcation a similar reflect that system parvo may processing

Multiple Geometric Representationsof Objects

355

" " for color and shape , whereasthe magno systemis insensitiveto color but is special ized for properties relevant to localization- motion , depth, and location (Living stone , Anderson and Felleman 1992for evidencethat and Hube11989 ; but seeVan Essen ) . Human clinical the systemsare coordinated at relatively early stagesof processing evidenceindicates that object recognition functions can be spared without localiza) . Recently, tion , and vice versa ( Fara et al. 1988 ; Levine, Warach, and Farah 1985 color and between naming functional for a object evidencehas appeared separation Saffran Breedin other the on , and , ( on the one hand, and spatial (locational) language Coslett 1994 ). ? Landau and evidence relevant to the structure of spatial language is this Why these of different the that 1993 systemsmight serve Jackendoff ( properties ) suggested . For , the fact that object example as one pressurein the design of spatial language " " , whereas shape and color (but not location) are representedin the what" system " object location (but not shape or color ) is representedin the where system is reminiscent of the distinctions uncovered by linguistic analysis and documented through experimentation among young children. It is possiblethat the relative lack of shape information in locational terms across languagesis due to the lack of shape information in the cognitive and neurological systemsunderlying object location. Similarly , the lack of locational information in object namesmay be due to the lack of such information in the systemsunderlying object recognition. While intruiging , this parallel betweenspatial languageand spatial cognition will undoubtedly undergo " " " " . revision as we learn more about the coordination of the what and where systems For example, a variety of evidencepoints to the necessityof coordinating information at levels likely to precedelinguistic encoding. Objects must be assembledfrom parts (and this requires assignment of relative location), certain named locations must be supported by quite specific and detailed perceptual representations (e.g., " " " " Dodger Stadium, Lincoln Center ), and perception of certain kinds of motion (a " " where systemproblem) may be constrained by the specificsof object identification ). (Shiffrar 1994 A secondintriguing parallel, not inconsistent with the first , is that there are different for the tasks of object identification and object location , functional consequences differences functional and that these give rise to differencesin the kinds of properties ) most readily processedin the two tasks. A recent study by Schynsand Oliva ( 1994 illustrates how this might occur. Subjects were shown a target scenefollowed by a mask and a rapidly presentedimage that was a hybrid of two of different kinds of , a combination of a city sceneand a scenes(each a possible target), for example the conditions . In different , hybrids were createdfrom a low passfilter highway scene the scene highway) . (say, of one scene (say, the city) and a high passfilter of the other " " ; for example The low-passfilter preservedonly coarse information about the scene

356

Barbara Landau

' s overall " " it preservedthe scene geometry but eliminated all fine-grained boundary and edge infonnation such as would be required for identifying particular buildings or vehicles . The high-pass filter preservedfine-grained infonnation . Thus one city highway hybrid might contain the overall geometry of the city with the fine details of the highway vehicles ; the reversehybrid would contain the overall geometry of the with the fine details of city buildings. The question was whether subjects highway would identify the hybrids on the basis of coarse or fine-grained infonnation , and how this would vary with exposuretime. The resultsshowedthat at the fastestpresentation times (30 ms), subjectstended to identify the scene representedwith low -pass filter (coarse ) infonnation ; at slower times 150 ms tended to the scenerepresentedby highpresentation ( ), they identify pass infonnation . Schyns and Oliva ( 1994 ) interpreted this pattern as evidencefor two different processing schemesthat operate in sequence . One scheme operates earlier by extracting only coarse infonnation about scenegeometry, while the other operateslater by extracting the finer infonnation . While both might be used to identify scenes , sequential operation would allow the perceiver to extract infonnation about general geometric composition first , followed by focused attention to the details of an identified scene was unknown ; this would be most beneficialwhen the scene and the perceiver had to categorize it quickly . Schyns (personal communication) -grained infonnation is indeed processedmore rapidly than comments that if coarse " fine grained, then the where" system might be incapable of doing anything but selectingcoarseinfonnation about objects and their generalgeometric relationships. These two parallels betweenlinguistic and nonlinguistic systemsplace the burden of explanation on the design of systems that presumably evolved independent of . Does it make senseto attribute the design of spatial language to such language causes ? And certain facts about spatial languagemust surely be learned(or ignored)" or " flatness " children learning Tzeltal must learn to attend to an object' s " bulginess when describing its location , while children learning English must learn to ignore theseattributes. What are we to make of this? Note that none of thesepossibilities is inconsistent with the others. Any learning device that beginswith some broad set of distinctions is likely to converge on a solution more quickly than an unconstrained device- as long as the set of universalsis correct. Indeed, it is highly likely that universal predispositions in object representation interact with learning quite early in life. Consider object shape and object name. It is a fact that the human visual systemcan distinguish among an enonnous variety of object shapes . It is also a fact that sameness in object shapeis strongly correlated with sameness in name; this is most likely becauseobject shape is an excellent predictor of other properties held in common by members of many object " kinds"

Multiple Geometric Representationsof Objects

357

) . Becauseobject names often do apply to (though clearly not all ; see Bloom 1994 in children similar that are , learning all languagesshould learn terms shape objects -shapeobjects. In this way, they with these same kinds that are correlated for ) ( object to could learn that shape is important object naming. Similarly, becauselocational terms such as spatial prepositions tend to apply acrossobjects that vary enormously in shape , children should learn to discount the particular shape of an object when learning those terms. A role for learning would seemto be crucial, given that some languagesdo incorporate somewhat more object information than English in their stock of basic spatial terms. For example , the child learning Korean will have to learn the difference between ahn and sok, corresponding roughly to loose- and tight -fit versionsof the English term in. It is possible , of course, that the distinctions betweenfigure and ground geometry, and the kinds of distinctions that appear relevant acrossall languagesare completely . unrelated to the facts about structure and processingof objects compared to places external It is also possiblethat the facts about spatial languagederive not from causes to language , but from the requirementsof a communication systemthat must rapidly . But if this is true, we are still left with a puzzle of why convey complex meanings comparatively little fine-grained detail, while the same figure and ground do possess objects obviously can be and are representedin detail when they are recognizedor named as object kinds. From the perspectiveof learning, it would be reasonableto assumethat the possibilities outlined above are all mutually reinforcing . That is, there , basedon structure or function , that differentially select may exist different systems information relevant to naming objects and to locating them; the differential representation -based information in these systems may propagate up to the of shape highest level, appearing as differencesin the coding of objects in linguistic representations of " what" and " where."

Puzzles 8.S Concluding Comments , Remaining


clear that objects can be represented More puzzlesthan answersremain. Although it seems in terms of very different geometric descriptions (for different purposes ), it remains unclear just what the status of thesedescriptions is, with respectto at least . four different issues First , what is the status of these descriptions with respect to dividing up spatial " " ? If detailed shapeis really a function of the what system , whereascoarse language " " in then we see direct of the where is a function , repercussions system might shape . Objects (usually named by count nouns) preserve different portions of spatial language -functions, usually named by detailed shape , place , and places (more precisely

358

Barbara Landau

spatial prepositions) preserveonly coarse or axial descriptions. So far , so good. But ? Even can we really connect the object/place representationsto different form classes within English, precise shape is encoded in certain verbs (posture verbs such as to kneel and to crouch , and perhaps manner verbs such as to undulate and to spin) , and axial representation are encoded in spatial adjectives(e.g., long, wide, thin ; see Bierwisch, chapter 2, this volume) . In other languages , relatively detailed object ) and positional verbs; seeSinha et al. 1993 shapecan be encodedin verbs (Japanese , Allan 1977 coarse or axial shape can be encoded in classifiers (see , for example ). Should we expect the different shape descriptions to cleave neatly along lines of " " " " form class , or along lines of some other distinction such as what and where ? " " And if so, what do we do with the persistentappearanceof the same coarse shape " - that show " " " " " " " , verbs, and up in classifers descriptors- round , thin , long, flat ? spatial predicates A secondpuzzleconcernsthe status of thesedifferent object descriptions relative to . Is the three-part division (detailed, coarse visual representations , axial) to be found ? Or does that system give rise in any principled sensewithin the visual system " " to a variety of different descriptions, some of which are selectedas special by ? languages Third , what is the status of object descriptions relative to representation in the " " brain? Do the different object descriptions enjoy different status in the what and " where" ? Can we find evidencefor the existenceof axial and , for example systems coarse descriptions in one system but not the other? A recent study by Breedin, Saffran, and Coslett ( 1994 ; Breedin and Saffran in preparation) may shed somelight . One of their patients sustaineddamage on this issue , at least with respectto language a severeobject naming deficit. The deficit to the infero temporal lobeand possessed was specific to object naming- the patient could recognize objects. Despite the naming deficit, this patient showed no impairment on spatial prepositions, nor on . Thus the axisobject-part terms, which require labeling the ends of the object axes based terms are functionally separatefrom object names , supporting the functional axial coarse and the detailed between / descriptions outlined in this chapter. separation these status of Fourth and finally , what is the descriptions as they articulate with ? In this chapter, I have presentedevidencesuggestingthat learning and development , probably prior to multiple representations of objects exist early in development these different of . The existence , and the object representations language learning for cornerstone as a critical serve in life flexible accessto them early learning. may different with coordinated Discovering precisely how these representationsbecome parts of vocabulary and how they becomemodified by learning remains a challenge . for future research

Multiple Geometric Representationsof Objects


Acknowledgment

359

-FY93 -0723from the Award 12 Sciences This work wassupported by Socialand Behavioral . I thank Paul March of Dimesand by National Institutesfor Health grant ROI HD-28675 of thechapter versions on previous for thoughtfulcomments ; Jennifer BloomandManishSingh . Vim for helppreparing Nolan andJessie figures
Notes I . If the flowers are real (rather than painted), then pragmatic constraints would force the ) fordiscussion interpretation that they are on the upper surfaceof the bowl. SeeHerskovits ( 1986 of many other contextual constraints. 2. This chapter will focus on spatial prepositions in English. This focus does not entail that , even for English. spatial infonnation is coded only in thesetenDs. This is clearly not the case -class However, following Talmy ( 1983 , grammaticizedportion of the ), I assumethat the closed " " , while the open class languageis likely to representthe fine semanticstructure of a language . this assumption Should of wider a verbs meanings range ) may represent (including spatial prove wrong, the analysis of English spatial prepositions can still provide a framework within . which we can build richer theories of the kinds of spatial meaningsencodedin languages " " 3. The tenD acrossis described by Talmy ( 1983 ) as requiring a ribbonal figure and ground that people judging the acceptability of a 1991 showed Williams . An ( ) experiment by object circles across found instance of as an rectanglesmuch lessacceptablethan intersecting display . This suggeststhat the figure must have a clear principal axis ellipses intersecting"rectangles " (making it a linear figure) in order to best satisfy the requirementsfor this tenD. 4. It is worth noting that neither children nor adults were simply translating known prepositions ' . A separateseriesof questions probed subjects generalization patterns for known tenDS such as across ; the patterns were not the sameas those found in the learning study (seeLandau and Stecker 1990for details) . " " 5. This procedure was modified for the few children who said yes only to locations other than the one directly in front of them. Probe trials were conducted using the same span of locations, but with eachsurrounding the single location most frequently acceptedby the child. 6. There are severalpossibleexplanations for the sharpeningin the shapebias with vocabulary growth . One possibility (described in the text) is that children begin with a representational in tenDSof shape bias in which objects are represented , and another bias in which object names bejust to connect up the two pairs of would of The function kinds. are linked to object learning in noise with expandedcomputaa decrease either reflect could the ; sharpening representations tional resources(seeLandau 1994for discussion ) or an enhancementdue to input that reinforces . A secondpossibility is that both vocabulary growth and the the importance of shape of a third factor , such as the ability to detect sharpening of the shapebias are a consequence which words are count nouns (hence object names ) . Syntactic growth (with which the child could detennine which words are count nouns) has long been thought to be a possiblecauseof ) . A third the so-called vocabulary explosion (for discussion , seeLandau and Gleitman 1985 's the child of reflection is a bias of the the is that learning genuine shape sharpening possibility . . Thesepossibilities are currently being tested that shapematters for object names

Barbara Landau 7. I thank Misha Becker for helping collect data on thesedistinctions. " unmotivated; most of the shape 8. The characterization of Tzeltal as especially" visual seems distinctions it carries can also be representedby other spatial systems , most notably, haptics. I thank Paul Bloom for reminding me of this fact. . 9. I thank Manish Singh for illuminating discussionof this issue References

- 311 . . Language Allan , 53 , K. ( 1977 ). Classifers (2), 285


' in extending novellabelsto Becker , A. H., and Ward, T. B. ( 1991 ). Childrens useof shape . . Cognitive : Identity versus animate , 6, 3- 16 Development objects posturalchange . : A theoryof humanimageunderstanding Biederman , I. ( 1987 ). Recognition by-components - 147 . Review , 94 , 115 Psychological . Paper at IEEE Systems Binford, O. ( 1971 , Science , ). Visualperception by computer presented andCybernetics Conference , Miami. -semantics in the acquisition of : The role of syntax Bloom names , P. ( 1994 mappings ). Possible . Lingua .), Lexicalacquisition . Special volume . In L. R. Gleitmanand B. Landau(Eds nominals - 332 . , 92, 297 -namelearningin youngchildren . Journalof Bomstein , M. ( 1985 ). Color-nameversus shape . ChildLinguage , 12 , 387 393 ' . : Cognitive vs Bowerman , M. ( 1991 ). The origins of childrens spatial semantic categories Eds . . In and S. C. Levinson determinants J. J. ( ), Rethinking linguistic Gumperz linguistic . . Cambridge , MA: Cambridge UniversityPress relativity in the faceof semantic Breedin , EM . (in preparation , S., and Saffran ). Sentence processing . . Manuscript loss : A case , TempleUniversity study effectwith of theconcreteness Breedin , H. B. ( 1994 , S., Saffran , E. M., and Coslett ). Reversal - 660 . . Cognitive semantic dementia , 11 , 617 Neuropsychology . Paper Brown , P. ( 1993 ) locatives ). The role of shapein the acquisitionof Tzeltal (Mayan Stanford Research Forum. at the 25th Annual Child , , University April Language presented Stanford , CA. -Radvansky : in visionand language of reference Carlson , L. A., and Irwin, D. ( 1993 ). Frames - 244 . Whereis above ? Cognition , 46(3), 223 : in English andKorean motionevents to express Choi, S., andBowerman , M. ( 1991 ). Learning - 122 -specific . . Cognition lexicalization Theinfluence of language , 41, 83 patterns ' ' in his first Clark, E. V. ( 1973 ). What s in a word? On the child s acquisitionof semantics . and . In TE . Moore Ed. , 65- 110 of language ( ), Cognitive development acquisition language . Press NewYork: Academic Farah , R. ( 1988 , K ., Levine , D., and Calvanio , M., Hammond ). Visual and spatialmental . . Cognitive - 462 : Dissociable of representation , 20, 439 Psychology imagery systems

Multiple Geometric Representationsof Objects

361

Francis, W. N ., and Kucera, H . ( 1982 : Lexicon and ) . Frequencyanalysis of English usage grammar. Boston: Houghton Mifftin . . Cognition. Hayward , W ., and Tarr , M . ( 1994 ). Spatial languageand spatial representation Herskovits, A . ( 1986 ) . Languageand spatial cognition: An interdisciplinary study of theprepositions in English. Cambridge: Cambridge University Press . Hill , C. ( 1975 . In Proceedings ) . Variation in the use of front and back in bilingual speakers of the First Annual Meeting of the Berkeley Linguistics Society. Berkeley: University of California . Hoffman , D . and Richards, W . ( 1984 ) . Parts of recognition. Cognition, 18, 65- 96. Jackendoff, R. ( 1983 . ) . Semanticsand cognition. Cambridge, MA : MIT Press Johnston, J. R. ( 1985 : The evidencefrom children learning English. ) . Cognitive prerequisites In D . globin (Ed.), The cross . Vol . 2, Theoreticalissues , linguistic study of languageacquisition 961- 1004 . Hillsdale, NJ: Erlbaum. Johnston, J. R., and globin , D . I . ( 1978 ). The development of locative expressionsin English, -Croatian , and Turkish . Journal of Child Language Serbo , 6, 529- 545. JonesS ., Smith, L ., Landau, B., and Gershkoff-Stowe, L . ( 1992 ) . On the origins of the shape bias in young children' s novel word extensions . Paper presented at the Boston University , Boston, October. Langauge Development Conference Kuczaj, S. and Maratsos, M . ( 1975 ) . On the acquisition of front , back, and side. Child Development , 46, 202- 210. Landau, B. ( 1994 , object name, and object kind . In D . Medin ( Ed.), Vol . 31, ) . Object shape . Psychologyof learning and motivation, 253- 304. New York : Academic Press Landau, B., and Gleitman , L . ( 1985 . Cambridge, MA : Harvard ) . Languageand experience . University Press " " " " Landau, B., and Jackendoff, R. ( 1993 ) . What and where in spatial languageand spatial ;oral and Brain Sciences , 16, 217 238, 255 265. cognition . Behav Landau, B., Leyton , M ., Lynch, E., and Moore , C. ( 1992 kind ) . Rigidity , malleability , object ,
and object naming . Paper presented at the Psychonomics Society , St . Louis , Mo . Landau , B . , Smith , L . , and JonesS . 1988 . The of in lexical .

importance

shape

early

learning

Cognitive

Development

299

321

' Landau , adults B ' and lexical learning . Journal of Memory and Language , 31 , 807 . , Smith , L . , and JonesS . ( 1992 ) . Syntactic context and the shape 825 . bias in children s

Landau

, in

. ,

and

Stecker

( .

1990

Objects

and

places , 5

Syntactic 312 .

and

geometric

representations

early

lexical

learning

Cognitive

Development

287

Levine

, of

D "

. , what

Warach " and

J "

. ,

and

Farah "

, in

( 1985

) disorders

Two

visual

systems to bilateral

in

mental

imagery cerebral

Dissociation

where -

imagery

due

posterior

lesions

Neurology

35

1010

1018

362

Barbara lAndau

: Theacquisition of a concept and a word . Journal Levine , S. (1982 ). Up front , S. C., andCarey - 657 . , 9, 645 Language of Child
Levinson, S. ( 1992 , and linguistic description: Tzeltal body-part tenninology ) . Vision , shape . and object description Working paper no. 12 , Cognitive Anthropology ResearchGroup , Max Planck Institute for Psycho linguistics, Nijmegen. . ) . Symmetry, causality, mind. Cambridge, MA : MIT Press Leyton, M . ( 1992 ) . Segregation of form , color , movement, and depth: Living stone, M ., and Hubel , D . ( 1989 , 240, 740- 749. Anatomy , physiology, and perception. Science LoweD . ( 1985 ) . Perceptualorganizationand visual recognition. Dordrecht : Kluwer . Marr , D . ( 1982 ) . Vision. New York : Freeman. . Marr , D ., and VainaL . ( 1982 ) . Representationand recognition of the movement of shapes o Proceedings / the Royal Society, London, 2/ 4, 501 524. Michotte , A . ( 1963 / causality. London : Methuen. ) . Theperception0 . Cambridge, MA : Harvard Miller , G ., and Johnson-Laird , P. ( 1976 ) . Languageandperception . Press University " " " " " " Narissiman, B. ( 1993 ) . The lexical semanticsof length, width , and height. Unpublished manuscript. Boston University . / reality in the child. New York : Basic Books. ). The constructiono Piaget, J. ( 1954 ' . Reprint, New York : Norton , / space , J., and Inhelder, B. ( 1948 ) . The child s conceptiono Piaget 1967 . ' . Reprint, , A . ( 1960 / geometry ) . The child s conceptiono Piaget, J., Inhelder, B., and Szeminska . New York : Norton , 1981 -Braem, P. ( 1976 Rosch, E., Mervis, C., Gray , W ., Johnson, D ., and Boyes ) . Basic objects in . Cognitive Psychology natural categories , 8, 382- 439. . Science , / 63, 895- 902. Schneider , G. E. ( 1969 ) . Two visual systems : Evidencefor time and spatial ) . From blobs to boundary edges , P., and Oliva , A . ( 1994 Schyns - 200. , 5, 195 scaledependentscenerecognition. PsychologicalScience , 3, Shiffrar , M . ( 1994 ) . When what meets where. Current Directions in PsychologicalScience . 96- 100 Sinha, C., Thorseng, L ., Hayashi, M ., and Plunkett , K . ( 1993 ) . Comparative spatial semantics . Paper presentedat and Danish Evidence from : and language acquisition , English, Japanese the International Conferenceon the Psychologyof Languageand Communication , Glasgow. ' , E. ( 1991 ) . Onto logical categoriesguide young children s inductions Sola, N ., Carey, S., and Spelke of word meaning: Object terms and substanceterms. Cognition, 38, 179 211. , and perceptualproperties Smith, L ., Jones ) . Count nouns, adjectives , S., and Landau, B. ( 1992 , 28, 273- 286. in children' s novel word interpretations. DevelopmentalPsychology

Geometric Representationsof Objects Multiple

363

of count contextin the learning esandsyntactic process , K. ( 1993 ). Perceptual Subrahrnanyam . Los ., Universityof California . PhiD. Diss , nouns Angeles andmass .), Spatial Pick and L. Acredolo(Eds . In H. structures space ). How language Talmy, L. ( 1983 . Press Plenum York: . New 225 282 and research : , orientationTheory , application . In T. Shopen structurein lexicalforms : Semantic . Lexicalization 1985 patterns , L. ( ) Talmy and the 3 Grammatical . Vol. and , categories Ed. description syntactic typology ( ), Language - 149 . : Cambridge . Cambridge lexicon UniversityPress , 57 : Cambridge . Cambridge University in theacquisition Tanz of deicticterms , C. ( 1980 ). Studies . Press . In D. J. Ingle , M. A. ). Two corticalvisualsystems , L. G., and Mishkin, M. ( 1982 Ungerleider . 549 586 behavior visual , MA: Eds . W. Mansfield , Cambridge R. J. and Goodale of ), Analysis ( , . MIT Press in the primate , D. ( 1992 ). Informationprocessing Van Essen , C., andFelleman , D., Anderson - 423 . . Science , 255 , 419 : An integrated visualsystem perspective systems ' ' . Honorsthesis of across , Columbia Williams , P. ( 1991 ). Childrens and adults understanding . University

Chapter PreverbalRepresentation and Language


Jean M . Mandler

Although my interests lie in the character of the preverbal conceptual system rather than of language itself, the preverbal system forms the foundation on which language rests , and it constrains what is learnable. I shall argue that preverbal conceptual representationis largely spatial in nature and that the relationship betweenspace and languageis therefore far -reaching and pervasive . It is not just that spatial terms tell us something about spatial meanings or that , spatial meaningsplace constraints on spatial terms. It is that many of the most basicmeaningsthat languageexpresses both semantic and syntactic are basedon spatial representations . Such a point of view will hardly be newsto cognitive linguists such as Ronald Langacker or Leonard as to why languageshould be Talmy . What I hope to contribute are a few suggestions so structured. I will suggestthat language is structured in spatially relevant ways becausethe meaning systemof the preverbal languagelearner is spatially structured. So with apologiesto Leonard Talmy for twisting his words, the subtitle of this chapter should read: " How SpaceStructures Language ." One further introductory comment. To say that the preverbal meaning system is spatially structured is not to say that it is the same as spatial perception. Rather, " " spatial information has been redescribedinto meaning packages , and thesemeaning . I will argue that someof the categorical packagesretain somespatial characteristics or packaging characteristicsoften ascribed to languageitself are actually due to the prepackaging that is accomplished during the preverbal period. Babies do not wait until the onset of languageto start thinking ; the problem of packaging meanings into workable units is thus a prelinguistic one.
9.1 Sensorimotor Schemas Are Not Concepts

The more I delve into cognition in the first year of life the more it becomesapparent that many of the most basic foundations on which adult conceptsrest are laid down during this period. Pace Piaget, the first year of life is far from being an exclusively

366

Jean M . Mandler

. Instead, the higher cognitive functions that (among other things) sensorimotor stage will support languageacquisition are being formed in parallel with the sensorimotor learning that is going on. The researchthat Laraine McDonough and I have been conducting indicates that the foundations of the major conceptual domains are being laid down during this period (Mandler and McDonough 1993 ) . Fundamental concepts months (perhaps to seven of animals and vehicles are learned by around six resting on an even earlier conceptual distinction between animate and inanimate things), and the domains of plants, furniture , and utensils follow soon after. These es :ning process conceptual domains in turn are used to control inferential reaso has In addition the . in , episodic memory system ) press (Mandler and McDonough es have begun (Mandler and become operational and long-term recall process ) . All this is happening before children learn how to speak. McDonough in press . Where is the familiar sensorimotor infant we Such findings should give us pause who has not yet achievedconceptual representation the creature are usedto hearing about, . In its place we find a baby that has already ? It seemsto have disappeared life. For a rich many people working in language acquisition conceptual developed this will come as no surprise, if for no other reason than the need to account for the . But the complexity of the concepts that newly verbal children expressin language in literature the current researchdoes make evident a tension that has been lurking for many years. According to Piaget ( 1951 ), babies are not supposedto have a conceptual , yet according to linguists, to learn languagerequires representationalsystem base . As a result, we pay lip service to the idea that to onto a conceptual mapping learn languagerequires a preexisting conceptual system , but have avoided specifying what that systemis like. The neglect seemsto be due in part to a conflict within Piagetian theory . On the , but one hand, Piaget ( 1967 ) said that conceptual thought is not created by language . On the in it various transforms instead thought precedes ways , which then language other hand, becauselanguage begins before the sensorimotor period ends , Piaget tended to characterize early verbalizations as just another kind of sensorimotor . He did devote a good deal of effort to describing how sensorimotor schema schemasmight be transformed into conceptual (symbolic) representation , but he said little about how the new type of representationdiffered from the old. The result are said to be transformed into concepts is a gap in his theory . Sensorimotor schemas little is said about what the concepts but and concepts are mapped into language , like. are themselves As best as I can tell , this dilemma was handled in different ways by people studying . Workers in language languageacquisition and those studying cognitive development for learning language the various notions to necessary acquisition attempted specify it to the left reason and then, developmental psychologiststo explicate ably enough,

and Language Preverbal Representation

367

the representational status of these notions. For example (with the exception of the nativist position that grammatical categories are innately given) there seemsto be widespread agreement that the underlying concepts needed to learn grammatical " " " " " " " " categoriesare notions such as actionality , objecthood, agent, location , and " Maratsos 1983 . But where the " ) developmental psychologists were to possession ( take over, until the recent work on objects and agencybegan to appear (Baillargeon 1993 ; Spelke et al. 1992 ; Leslie 1984 ), there was largely a blank. BecausePiagetian theory was silent about conceptual representation at the end of the sensorimotor period, it seemsto have been assumedby default that the relevant conceptual categories . Thus in many accounts themselves were the sameas the sensorimotor schemas the sensorimotor achievementswere assumedto be the baseonto which languageis . Typical examples of this approachwerethe various attemptsto relatelanguage mapped , but , such as object permanence acquisition to stage6 sensorimotor accomplishments thesewere not very successful(seeNelson and Lucariello 1985for discussion ). For the most part , sensorimotor schemasare not the right sort of representation . Piaget provided some of the reasonswhy procedural forms of for learning language representation such as sensorimotor schemascannot in themselvesserve a semiotic function . A sensorimotor schemaprovides something like meaning in that it enables recognition of previously seenobjects to take place, and thus for the world to seem familiar . It also allows each component of a familiar event to signal the next component to come. This kind of reaction is indexical; a conditioned stimulus predicts or " that some other event will follow . But a sensorimotor schema does not " means to its parts for purposesof denotation or to enablethe baby allow independentaccess to think independently of the activation of the schemaitself (Karmiloff -Smith 1986 ). In short, sensorimotor schemas are neither conceptsnor symbols, which Piaget considered to be the sine qua non for both the development of the higher cognitive functions and languageacquisition. There are other ways in which sensorimotor knowledge also appears to be the structure perception . Sensorimotor schemas wrong sort of basefor learning language and control action. These schemasconsist of a large number of parameters that monitor continuously varying movementsand rapidly shifting perceptual views. How ? Some kind of are such schemasto be mapped into a discrete propositional system interface betweenperception (or action) and languageis needed , something that will allow an analog-digital transformation . For example , consider putting a spoon into a bowl. This requires an intricate sequenceof movements , but the conceptual system event that constitutes its meaning. In it a of the greatly simplifies , fonning summary this case , the meaning might be a representationof one object containing another. It is this conceptual simplification onto which propositional languageis mapped, rather . than onto the sensorimotor schemasthemselves

368 9.2 Differences between Perceptual and Conceptual Categories

Jean M . Mandler

In addition to Piaget' s view that at the end of infancy conceptsare constructed out of sensorimotor schemas , there is an even older view of the onset of concept formation , the traditional doctrine of the British empiricists, espousedin modern times namely, ) . In this view, which Keil ( 1991 ) has called the by philosophers such as Quine ( 1977 doctrine of " original sim," before children develop abstract conceptsabout the world they categorizeobjectson the basisof their physical appearance according to the laws of perceptual similarity . Once theseperceptual categoriesare formed, various types of information becomeassociatedwith them, and in so doing theseperceptual categories becomeconceptual in nature. This associativedoctrine of the creation of conceptsis exemplified in current theory by the view that the first conceptsto be formed are at the basic level ( Mervis and Rosch 1981 ) . In this view, babiesfirst form conceptssuch as dog and cat on the basis of the similarity of the exemplarsto each other, and only much later generalizefrom theseconceptsto form a superordinateconcept of animal. The details of this process have never been worked out , but it would seemto be a processalong the lines of the doctrine of original sim. This view is given support by the recent findings of Elmas and Quinn and their colleagues(Elmas and Quinn 1994 ; Quinn, Elmas, and Rosenkrantz 1993 ) showing that as young as three months, babies form perceptual . categories of animals after a very few exposuresto pictures of contrasting classes For example both three olds and six month olds learn to horses , quickly distinguish from zebras , dogs from cats, and cats from both dogs and lions. It is agreedthat these are purely perceptual accomplishments , as I , but Quinn and Elmas (in press ) believe assumedo many others, that these perceptual categories form the kernel around which the first conceptswill develop. Nevertheless , there are both theoretical and empirical difficulties with this view . Theoretically, it does not specify in what form the that have never been resolved information to be associatedwith the perceptualcategoriesis itself couched. A property such as barking might be a perceptual category in its own right , and one could imagine how it might becomeassociatedwith the perceptual category of dog. But it is difficult to understand how properties that are less clearly perceptual are represented " " " " , such as animate or interacts with me. More importantly , in my opinion , this approach does not explain how the transition from perceptually basedcategorization . Indeed, Quinn to more abstract or theory-laden concept formation takes place and Elmas ( 1986 ), among others (e.g., Keill99 I ), have pointed out that no one taking the traditional empiricist view hasever satisfactorily explained how abstract or superordinate conceptsare derived from the perceptualconceptsof infants, or how theorybasedassociationsbegin to supplant perceptually basedones(seealso Fodor 1981 ).

Preverbal Representationand Language

369

As long as it was assumedthat superordinate concepts such as animal, vehicle, and plant were late acquisitions, this difficulty might be finessed . For example , perhaps languageacquisition itself contributes to superordinateconcept formation (e.g., Nelson 1985 ) . However, research in our laboratory has shown that infants have formed conceptsof animal and vehicle as early as sevenmonths of age (Mandler and McDonough 1993 ), and other global concepts such as plant are in place at least eleven months by (we have not yet tested younger children on this concept) . This researchshows that on some tasks infants distinguish global categoriesbefore they distinguish the basic-level categoriesnestedwithin the animal class . ! For example , on our tasks infants differentiate animals and vehiclesfrom sevenmonths onward. But even by elevenmonths, they do not differentiate dogs and rabbits or dogs and fish.2 Furthermore, differentiation among various basic-level classesof mammals, such as -level classes of land vehicles dogs and rabbits (and also basic , such as cars and trucks) is still not well establishedat eighteen months (Mandler , Bauer, and McDonough 1991 ). The details of the development on theseconceptual domains is not my main concern here. Rather, I want to emphasizethat the developmentof perceptual categories (which are sensorimotor accomplishments ) does not look like the development of ones. Because most conceptual aspectsof thesetwo developmentshave not yet been , specifying the differencesbetween them is still problematic. Nevertheless investigated , several reasonsto make the distinction are already known. First , if there were only perceptually based categories in infancy, it would be difficult to explain how infants could manageon any kind of task to categorizetwo superordinate domains, whose exemplarsdo not look alike, while failing to categorizethe basic -level classes within them, whose exemplars do look alike. The quintessential example of this dilemma is shown by infants in our experimentsdistinguishing betweenlittle models of birds and airplanes, all of which have outstretchedwings and therefore very similar overall shapes , while at the same time not distinguishing between dogs and fish or 3 dogs and rabbits, whoseshapesare quite different ( Mandler and McDonough 1993 ). Second , a purely perceptual account of categorization cannot explain why three- to six-month -old infants are apparently so much more advancedthan seven - to eleven month -olds, in particular , why the younger infants make fine discriminations among -level classesthat the older infants do not. basic McDonough and I have suggested that the infants at these different ages are actually engaged in different kinds of , even though superficially there seemto be similar task demands in the processing various experimentsthat have beenconducted. The experimentsfor both age ranges have used a habituation -dishabituation paradigm. However, the studies of categorization in young infants have measuredtimes to look at pictures, whereasin our work we have measured times to manually explore objects. Apparently , the traditional

370

Jean M . Mandler

looking -time habituation -dishabituation experiments do not engage infants very , there is often high subject ); for example deeply (Mandler and McDonough 1993 loss in these experiments even when the infants are given something to suck on to keep them awake and happy. On the other hand, when infants are given objects to explore, they show intenseinterest and concentration and subject loss is virtually nil . Although this issueneedsfurther study, our findings suggestthat very young infants begin to perceptually categorizethe world in the absenceof meaning, but that when they are older and are given a task that engagestheir interest, a different processis brought to bear. This different processconsistsof treating objects as kinds of things, . that is, as having meaning, not just as things of differing appearance This early conceptual processing is crude in comparison to the fine perceptual discriminations that infants make. They appear not yet to havedivided the world into very many different kinds, although the kinds they have conceptualized are fundamental cuts that give meaning to the perceptualcategoriesthey are also making. That is, the primary meaning to accrue to a basic-level category such as dog is that it is an animal; it is secondary(not only for infants, but adults as well) that dogs are four ' 4 leggedor bark , or are man s best friend. I am suggestingthat the babiesin our experimentscan seethat dogs look different from fish or rabbits, but do not find thesedifferencesimportant enough to treat them differentially . This situation is essentially the same as when an older child or adult seesthe differencesin the appearanceof poodles and collies, but for most purposes treats them as the samekind of thing , namely, dogs. Babiesseethe differencesin the appearanceof dogs and rabbits, but having constructed fewer concepts about the world , for most purposestreat them as the samekind of thing , namely, animals. The , exactly what doesthis initial concept of animal consist of and question then becomes how is it learned ? Unless one wants to posit that the concept of animal consists of a set of innate ideas , the meaningsthat make up this concept need to be derived from information that babies can learn from observation alone. By sevenmonths, babies are not yet independently locomoting; they have just begun to handle objects and are still unskilled -month -olds have held any kind at doing so. It is also unlikely that most seven ? The of real animal in their hands. So what kind of information is at their disposal first that seems ) has shown likely to be relevant is biological motion . Bertenthal ( 1993 that three-month -olds already differentiate biological from nonbiological motion , . It seems insofar as the parametersof people' s motion are concerned likely that they do the same for other animals as well becausethe parameters governing animate motion are quite general. Thus perception of biological versusnonbiological motion of is one early sourceof knowledge that could be usedto divide the world into classes in and inanimate . that move animate ways things

Preverbal Representationand Language

371

Once these categoriesof motion are formed they must be characterized in some way, if the differenceis not just to remain a sensorimotor distinction but to represent a meaning. One of the ways to do this is to notice that the things that move in the biological way start up on their own, whereasthe things that move in the mechanical way start only when another object contacts them. Another characteristic to be noticed is that the things that move in the biological way and start on their own also interact with other objects from a distance , whereas those things that move mechanically and get pushednever interact from a distance. Notice that each of these . Indeed, these are some of the properties is available even to very young babies that babies can when their . major properties pick up acuity is still not well developed to Responsivity thesecharacteristics of motion can explain why babies as young as two months of age respond differentially to people and to dolls (Legerstee 1992 ). Peopleinteract with them; dolls do not. Similarly, it can explain why, by four months, babiesdifferentiate causedmotion from self-motion (Leslie 1984 ). There are, of course , many other properties of objects that babiesobserveas well. By four months, babies know that objects are solid, that other objects cannot pass through them, and that objects still exist when they move out of sight (Baillargeon 1993 ) . By six months, babieshave learned something about containment; they know that containers must have bottoms if they are to hold things (Kolstad 1991 ) . As young as three months, infants have begun to learn about the properties of object support . They expect an object that losescontact with a surface to fall , unless it is supported by a hand (Baillargeon in press ) . Slightly older infants expect that any contact implies support, so that various insubstantial objects such as a horizontal finger touching a large box are expected to be sufficient to provide support. By sevenmonths, babieshave learned enough about contact and support to predict that something seento overlap its supporting surface by only about 15% of its basewill fall . There are undoubtedly other properties babies learn about before they begin to handle objects themselves , but theseare someof the main onesthat have beenstudied to date.
9.3 How Meanings Are Created

-toSelf-starting, biologically moving, mechanically moving, interactive, causing -to-be-moved, contacting a surface move, caused , containing these are all observable and or kinetic . This is one of the reasonswhy I have proposed spatial / properties it that is spatial properties (including motion ) babies analyze and abstract from perceptual . I have suggestedthat as infants are learning to displays to form meanings parse the world into objects, a processof perceptual analysis begins to take place , 1992 (Mandler 1988 ) . This is an attentive processthat occurs when an object is being

372

Jean M . Mandler

, unlike the usual thoroughly examinedand/ or is being compared with somethingelse sensorimotor processing which occurs and is , automatically typically not under the attentive control of the perceiver . This attentive analysis results in a redescription of the perceptual information being processed . Thus babies have a mechanism that enablesthem to abstract spatial regularities and to usetheseabstractions to form the . The contents of this new conceptual systemare beginnings of a conceptual system setsof simplified spatial invariants. It is theseinvariants that form the earliest represented . I claim that thesespatial abstractions are sufficient in themselves to meanings the initial of such as animate inanimate represent meanings concepts thing, thing , cause to interact with objects (pick , agent, support, and container. It is not necessary them up, hold them, move them around, or move around them) for meaning to begin to be created , although as infants mature thesenewfound skills will provide different kinds of information than they received before. But to begin the process , it may take no more than an intelligent eye and a mechanism to transform what the eye observes .s I want to add an asidehere, which I hope will clarify the position I have taken with respect to the creation of meaning (Mandler 1992 ) . It is not a nativist position ; on the contrary, it is a constructivist account. The mechanismof perceptual analysis I have describedmakes it unnecessary to posit inmate ideas or concepts ; perceptual analysisalone can build up meaningsand can do so continuously throughout infancy (and for that matter, throughout life) . The mechanism itself must be innate, and presumably also the basic aspects of the spatial representations that result from the analysis , but the concepts our minds conceive do not have to be carried on our . Thus babiescan create a beginning concept of animal even though it is crude genes compared to the biological theory they will eventually espouse(Carey 1985 ) . New can new information at time and of course with the onset of , analyses , provide any a whole new source of information arrives on the , language accumulating conceptual scene . Even if we agree that the earliest meanings , such as animal or container, are derived from spatial information , their representational format need not be spatial. After all , I havejust describedthem using language . On the other hand, becausethe result from spatial analyses , there doesnot seemto be any good meaningsthemselves reason to translate them into propositional form . Language will be coming along shortly and babies may not need propositional representationsin the interim . Once language is learned, they will be in the advantageousposition of having two kinds of representation , one of which is useful for representingcontinuous and dynamic analog information and the other which provides a way of representinginformation in a discretecompositional system . Is there any advantagein the meantime to translate of spatial representations something starting up on its own or interacting with

Preverbal Representationand Language

373

something else from a distance into a list of propositions such as [selfmove (thing)] or [afar (thingl , thing2) + interact (thingl , thing2)]? And. how would this be accomplished ? Is there a list of empty slots waiting in the mind to be appointed to each successive , so that , say, slot 32a becomesa symbol meaning selfmoving spatial analysis becomes a symbol meaning distant interaction? This is what and slot 32b , the Hamad ( 1990 ) called symbol-grounding problem. Peopleusually try to solve this problem by saying that the external world provides the meaning for symbols. But neither the external world nor perception of it can provide meaning in and of themselves . The three-month -old who categorizesdog patterns or horse patterns can do so in the absenceof meaning, just as an industrial robot can categorize nuts and bolts on the assemblyline without meaning entering into its programs at all. Substituting perception for meaning is no different from substituting sensorimotor schemasfor . Nothing . Instead, meaningmust come from an analysisof what is perceived concepts about such analysis suggestsit need consist of propositions composed of discrete symbols. One reason to translate spatial representationsinto another format would be if it . If existing spatial representationswere themselves were needed to learn language this then a preverbal propositional representational system for purpose adequate . first would be superfluous At , spatial representationsseem unlikely candidates glance . Their continuous analog character for the baseon which to construct language I described for sensorimotor difficulties to some of the same to be subject appears . How do they get broken down into components that allow languageto be schemas mapped onto them? Here is where image-schemascome in. These are the type of spatial representations that I have described as resulting from perceptual analysis ; Mandler ). They are spatial abstractionsof a specialkind (Lakoff 1987 (Mandler 1992 the same character while at retain their continuous 1992 . schemas analog ) Image . of desirable characteristics some of the time providing propositional representations form discretemeaning packages Although they are not unitary symbols, image schemas . In addition , they can be combined both sequentiallyand recursively with other . Thus they provide an excellentmedium to bridge the transition from image-schemas . to prelinguistic linguistic representation

in theFormof Image -Scbemas 9.4 SpatialRepresentation


-schemas Because of the attention that babies give to moving objects, the first image that can be The simplest meaning they form are apt to be those involving movement. taken from such movement is the image schemapath . This schemarepresentsany , without regard to detail either of the object moving on any trajectory through space be analyzed But can themselves of movement. or , and as I discussed paths object type

374

Jean M . Mandler

, focus on the shape earlier, theseanalyseslead to the concept of animal. For example of the path itself leads to schemasof animate and inanimate motion . Focus on ways -motion , -schemasof self-motion and caused that trajectories begin leads to image . (This is an example of associatedwith animate and inanimate objects respectively and end-of -path are emof : beginning path the embedding nature of image schemas " " bedded in path itself ) . Although I originally called theseimage schemas dynamic becausethey can representcontinuous change in location , it would have been more " " accurateto call them kinetic. As I have defined them, path and its parts are spatial, rather than forceful. ' Other types of paths that attract babies attention are those that go into or out of -schemasof containment, contact, , leading to image things, and onto or off surfaces that perception of contingent motion , or interactions and support . I have also suggested , can be representedby the notion of coupled paths, among objects at a distance are interesting, not only because . The link schemas or a family of link image schemas they capture one of the ways in which animate objects behavebut also becausethey to be a nonspatial meaning (if A , then B) has illustrate how what at first glanceseems . In Mandler 1992 , I discussedhow the link schemathat an underlying spatial base in its representsthe meaning of one animal following another can, by a slight change a structure, also representtwo objects taking turns. This is an exampleof how spatial which representationcan also representtime. It requires mentally following a path, It is time. of an not takes time but which does independent representation require terms time by borrowing spatial known , of course, that languagestend to represent is that it is easier to think reason the I think . 1978 1982 Fillmore . e. ) ; Traugott ( g, about objects moving along paths than to think about time without any spatial aids. babies are slow information processorsand becausethey probably needa Because , analyzing lot of comparisons to carry out any single piece of perceptual analysis One can relations. than them for be easier should temporal relations analyzing spatial look back and forth at the various parts of an object or look back at the place where , and it may be difficult an object beganto move. Temporal information is evanescent ' . If the infant s initial to analyze without the help of previously acquired meanings conceptual vocabulary is spatial, the easiestway to handle more difficult conceptual In izations would be to use the spatial conceptions that have already been formed. , to say this view the conceptof time is not a primitive notion but derived. Of course not does than difficult imply conceptualizing space that conceptualizing time is more discussion This are. that babies are not sensitive to temporal relations; they obviously , however, is concerned with the ability to think about time and spaceand the relations, representationswe use to do so. All organisms are sensitive to temporal time about think do we When , we may them. but most get by without conceptualizing

Preverbal Representationand Language

375

always do so in terms of following a path . Part of path following may include some ineffable sense of duration , but that in itself does not seemto qualify as conceptual. It is not just time that is more difficult to analyze than space ; so are dynamics -schemasare derived and internal feelings . Talmy ( 1985 ) has suggestedthat image from analyzing the forces acting on objects, and Johnson ( 1987 ) claims that they are ' derived from one s bodily experiences . For developmental reasons , however, I have stressedspatial analysesas their source . If image schemasare to representpreverbal , they must reflect the processinglimitations of very young infants. Babies meanings begin their perceptual analysesbefore they have yet learned to pick up and examine objects; thus many of the action schemasthat might be used for purposesof image-schemaanalysis schematicanalysishave not yet beenformed. The process esof image must be already well advancedby the time babieshave becomeadept at manipulating the world , and long before they can move around in it . In addition , humansare strongly visual creatures , and it should be easierfor babies to analyzevisual displays (or even for blind babiesto analyzedisplays via touch) than to analyze their internal sensations . There is no evidenceon this issue , but it may be noted that we are notoriously bad at introspection evenas adults. It is not that babies are unaware of feelingsof force or happeningswithin the container that is their body . But in terms of analysis , one can seethe movements of objects, whereasone must typically infer the forces operating on them- and of course one cannot seeiqternal activity at all. It simply has to be more likely that a baby will learn about containers from watching objects go in and out of other objects than from introspecting about the act of eating. This point of view is supported by the widespreadphenomenonthat the vocabulariesof internal statesare derived from the vocabulariesused to describe external phenomena(e.g., Sweetser1990 ). It may be that even as adults the concepts " are at heart " " we call " internal states , given their internal flavor by spatial analyses the gut sensationsassociatedwith them. Again , I am talking about conceptionsof . internal states , not the statesthemselves

? 9.5 Whatis theEvidence That SpatialAnalyses Structure Language Learning


The spatial analysesI have beendiscussingare particularly important in learning the relational aspectsof language , such as the meaning of verbs and grammatical relations . Object labels can and do get mapped ostensively onto the shapesof things, although that does not in itself give them meaning. But young children do have the , furniture , kitchen utensils (and global preverbal meaningsof animal, plant , vehicle perhaps many more) at the time they begin to learn object names(Mandler , Bauer, and McDonough 1991 ) . A good deal of what parents teach young children by the

376

M. Mandler Jean

. For way they name things is to carve thesedomains into smaller meaning packages , children have the preverbal meaning of animal, and as discussedearlier, example they also seethe perceptual difference between dogs and cats. Now they hear that this-shapedanimal has a different name from that-shapedanimal, and, at least in our culture, much is made of the fact that the two kinds of animals make different sounds as well. All this must suggestto children that the difference betweencats and dogs may matter. In this way language can help the processof subdividing the initially es that carry meaning above and beyond their global concept of animal into subclass . It is interesting in this regard that in the initial stagesof noun learning, animalness . But as differential labeling increasesover children do not particularly rely on shape the next few months, they increasingly rely on shape to determine the referenceof that children are making new nouns ( Jonesand Smith 1993 ). Such a finding suggests -shapecategoriesthey have learned the connection betweennouns and the perceptual over the course of the first two years. -based perceptual categories such as " dog" and " cat" On the other hand, shape cannot be used for learning grammar becauserelations cannot be pointed to in the way that objectscan. But the global domain-level conceptssuch as animal and vehicle that were used to give meaning to these perceptual categoriescan be used instead. -schemasthat give the meaning " animate thing " to dog and cat can Thus the image also be used to frame languageoverall, to provide the relational notions that allow , once the meaningsare formed for animate propositions to be built up. For example move themselves and cause other things to move, one has that as things objects arrived at a simple concept of agent (Mandler 1991 ). Similarly, once the meaningsare but are caused formed for inanimate objectsas things that do not move by themselves the earliest to move, one has arrived at a simple concept of patient. It may be because notions and relational that abstract abstract and relational are themselves meanings such as agent and patient can be formed so easily. -schematicunderVerb acquisition provides concreteexamplesof this kind of image -schemas ) discuss in detail how the kinds of image pinning . Golinkoff et al. ( 1995 I have outlined underlie verb learning. The first verbs that children learn all describe " of these . The " shapes paths are represented paths of various sorts rather than states . These specific path schemasare more particular than the paths by image schemas that differentiate animate from inanimate motion , but are otherwise similar in kind . A typical example is the verb to fall , which specifiesthe direction of the path of motion , but leavesother details aside. This kind of image-schemaallows children to ignore the details of a given event and so to generalize from one instance to the next- in short to categorizetypes of motion . -to -move, agent, At a more general level, notions such as animate object, cause of the kind be moved are and caused to inanimate object, meanings needed exactly

Preverbal Representation and Language

377

. As Slobin to master the distinction betweentransitive and intransitive verb phrases , this distinction , abstract though it may be and marked in a ) has discussed ( 1985 in different of , is universally one of the earliest grammatical languages ways variety in the distinction is that the ideasexpressed for this . The reason forms to be acquired are among those which preverbal children have universally mastered by the time . English does not mark this distinction with grammatical morphemes language begins but , many languagesdo and these should be easy for children to learn. For example ) point out that Korean usesdifferent forms , Choi and Bowerman ( 1992 for intransitive verbs of self motion and transitive verbs of causedmotion (for example " " " " , a causativeinflection must be added to roll in He rolled the ball into the box, " whereasit is not neededin " The ball rolled into the box ) . Korean children respect this distinction as soon as they begin to use these verbs and do not make cross category errors. When errors are made in thesekinds of grammatical morphemes , they often consist . For example of underextensions , Slobin ( 1985 ) found that children first use the morphemes marking transitive verb clausesin the prototypical transitive situation in which an animate agent physically acts on an inanimate object. Only later do they extend the marking to the lessprototypical casesin which the agent is inanimate or that children may try a the patient is animate. This kind of underextensionsuggests their hear onto of the direct already-formed conceptulanguagethey mapping fairly alizations. Of course , and somedistinctions seem , languagesdo not always cooperate likely to give languagelearnerstrouble. This raisesthe old Whorfian issueof the extent to which languageis mapped onto preexisting conceptsor by its own structure leadschildren to create new ones. I will illustrate this issuewith the caseof learning spatial prepositions. Let me say at the we all agreethat languageis to somedegreemapped onto existing outset that because , we are only haggling over the details. But one of those important details is concepts the following . Have preverbal children learned all the major spatial relations that ? Or have they learnedonly a subsetand do languagesteach various languages express them to attend to new ones they have not analyzedon their own? Melissa Bowerman and I have discussedthis issuequite a bit , although I am not . The particular issue sure whether we have agreed , or merely agreed to disagree involves the notions of containment, contact, and support. As Bowerman ( 1989 ) has in various relations these divide discussed ways, and , the languagesof the world up , makesa single furthermore do so by a variety of constructions. English, for example of the means and containment between distinction prepositions by support general in and on, with contact being ignored. I have claimed that containment and support -schemasto be formed; becausethey match the English are among the first image prepositional system in a straightforward fashion, it is not surprising they are

378
the earliest grammatical morphemesto be learned, and are learned virtually without 6 error ( Mandler 1992 , they ) . These morphemes are very frequent in adult speech capture a well understood conceptual distinction , they are easyto say, and so forth . Although containment and support sound like universal spatial primitive , Bowerman ( 1989 ) suggeststhat this may be a somewhat provincial view. Some languages make no distinction at all (as in Spanishen), and others make a three-way distinction . Furthermore, various languagesmake the distinctions they do make by cutting the , German divides support relations into spatial pie up in different ways. For example two , depending on whether the support is horizontal or vertical. Dutch makes a similar split but apparently usesthe method of attachment to categorizethe support relation , rather than the horizontal and vertical. In either language can , difficult cases such as how to that a is on the . down is appear, express fly ceiling Upside support an unusual support relation , and one might predict that it would give young language learnerstrouble.7 Developmental psychologists have only recently begun to explore in depth the development of conceptsof containment, contact, and support in preverbal infants, but the work of Baillargeon and her colleaguesdescribed earlier (e.g., Baillargeon, 1995 ) tells us that a great deal of detailed knowledge is accumulating in the first but rapidly learn conceptual year. Babiesapparently start with quite simple image-schemas variations on these , including containment with and without contact, horizontal versusvertical support, and so forth . The data suggestthat a wide variety of . What remains theseconceptual notions are well establishedbefore languagebegins to be done is to repackagethesemeaningslinguistically. Perhapsbecausethe conceptual notions are meaningsand cannot be pointed to , or perhapsjust becauseof their abstractness , different languagesrepackage them in various ways (Gentner 1982 ), babies must learn by listening to their native tongue. ways If the native tongue is a prepositional one, it will expressa quite limited subsetof ), typically making binary or trinary spatial distinctions (Landau and Jackendoff 1993 distinctions in relations such as containment, contact, and support. The distinctions are few enough that they should pose few problems to the language learner who . There are ways to expressspace comesequipped with many such preverbal meanings that are limited by other principles, however. One way is to use body parts, as in " " Mixtec ; for example , instead of saying, The cat is under the table, in Mixtec " " one would say, The cat be-located belly-table (Brugman 1988 ) . The systemis still one set of such as containment and insteadexpress es ) spatial but ignores relationships( a different set (relative locations vis a vis a human or animal body) . Of course , body parts are well known to the young languagelearner; indeed, naming body parts is a common gameamong parents and newly verbal children, at least in our culture. This method of linguistically partitioning spaceshould therefore not give children trouble.

Preverbal Representationand Language

379

Other languagesuse verbs to expresssome of the relationships that English describes , entirely different morphemes by means of prepositions. In Korean , for example are used to expressrelationships of put into, take out, put onto, and take off. , Furthermore, the morphemesare different for put into tightly versusput into loosely and for putting clothes on the trunk , putting clothes on limbs, and so forth . Essentially what Korean does is to distinguish between containment and support when these relations involve loose contact, but override containment and support when tight -fitting contact occurs. It is as if the language says that if the relationship is tight -fitting both containment and support apply in equal measureso that only the . type of contact will be specified This set of semantic categories , combined with their expressionin separateverb cannot get by in the early stagesof communication children Korean forms means that by widespreaduse of a few all purpose prepositions such as in or out to express theserelations. On the other hand, they learn the morphemesjust describedearly and effortlessly, just as English-speaking children learn a small set of prepositions to . English-speakingchildren, of course , do not say./it together expresssimilar meanings those ideasare not expressed by single morphemesin tightly or put in looselybecause English. The question is whether English speakingchildren already understand these particular spatial distinctions and are silent about them becauseof a lack in their -schematicmeaningsuntil , or whether they do not form the relevant image language the languagedirects them to do so. We are back to our Whorfian issue , but we have turned it into a manageable Choi Bowerman and , McDonough , and I are engaged in an , , empirical question sure if we have different predictions or . I am not it to answer experimental attempt have had not. I believethat babies ample experienceof clothes fitting tightly or of the difficulty of separatingpop beadsto have formed a concept of tight fitting . Therefore, I predict we will be able to show this distinction in preverbal children. The fact that Korean children sometimes overgeneralizethe tight -fitting relation to the case of clothing (Korean usesa different word for putting on clothing), indicates to me the of a preverbal notion (as does the more general fact that the common errors presence of another) . children make in learning one languageare often the correct expressions We still know relatively little about the age at which thesevarious spatial analyses begin to be made. In addition , we do not yet have good estimatesof the amount of -specific learning that takes place before word production begins . If these language two factors interact, it may be difficult to disentangletheir relative importance. Nevertheless . First , if a languagedoes not make , a few simple principles can be surmised a given distinction that a preverbal baby has conceptualized , this will not cause a this lack of sensitivity. to overlook languagelearning problem. Babieswill be willing Second , if the language makes a distinction that the baby has already learned, that

380

Jean M . Mandler

will also not causea problem, whether the distinction is expressed by a preposition or verb (given equal saliencein the speechstream) . Third , difficulty will occur only when the languagemakes a distinction that the baby has not made prelinguistically. If the baby has no conception at all of the meaning of such a morpheme or construction, it should be a very late acquisition indeed. A more common situation is likely to be one in which the morpheme excludes one of the possible and likely meanings in question. A possibleexampleis an error Korean children sometimesmake inexpressing the tight -fittingness of a flat magnet on a refrigerator door (the verb for fitting tightly has to do with three-dimensional objects, and the status of a flat magnet is not entirely clear) . The presenceof such errors does not necessarilymean that the language is teaching a new relationship, only that the situations describedare unusual or atypical vis a vis the particular semanticcut that the languagehas made. One of the points I have made about image-schemarepresentatonsof spaceis that they have already been simplified and schematized ; they have already filtered out a deal of the information the great perceptual systemtakes in. Languagemay do some of this kind of work , as Landau and Jackendoff ( 1993 ) have hypothesized , but it seemslikely to me that much of it has already beendone before languageis learned. Infants have been analyzing spatial relations for many months. If thesespatial relations are representedin terms of image-schemasa lot of the analog-to -digital transformation neededfor languagelearning has already beendone. The result is a set of meaning packagesthat languagecan put together in a variety of ways, ignoring some , emphasizingothers. At the same time, no matter what the language , the number of distinctions neededto learn the spatial pronouns and/ or verbs children acquire in their first year of language is quite small, involving such notions as inside-outside, contact- no contact, horizontal -vertical, up- down, tight -loose. The language itself can help children learn the more complex relationships they master at later stagesby directing perceptual analysis to aspectsof stimuli they may not yet have noticed. I will close by reiterating the importance of the conceptual level of representation to understanding languageacquisition. I worry that in too many accounts language is talked about as if it were mapped onto actions or onto perception. This is a common approach in connectionist paradigms, for example. Instead, language is mapped onto a meaning system that forms an interface betweenanalog and digital forms. This interface, which shares some of the properties of both forms, is what enablesa propositional representationalsystemto be added to the baby' s repertoire.

Ackoowledgment
of thischapter wassupported in partby National Preparation Science Foundation research -21867 . grant08892

Preverbal Representationand Language Notes

381

I . We use the tenD global for theseconcepts becauseit does not seemcorrect to speak of a superordinate concept if it is not yet differentiated into subconcepts(Mandler , Bauer, and McDonough 1991 ). 2. Infants in our experimentsdo make more distinctions within the vehicle domain during this age range. 3. Domain - level categorization raisesthe issueof how infants identify as animals little models they have never seen before, such as a model elephant. We do not yet know which features -month-olds are using to identify the correct domain. We have seven that once infants suggested have begun to analyze object movement, it directs their attention to the parts associatedwith motion (Mandler and McDonough 1993 ). This may be why infants are sensitiveto what seem (to us) like very small differencesbetween the outstretched wings of the birds and airplanes in our experiments . They do not appear to be using face information becausesome of our planes are Flying Tigers with facespainted on them, and some of the bird facesdo not show . They might be using textural information , although texture cues are minimized in our eyes plastic models. Whether shapeor texture, however, a solely perceptual account has difficulty in explaining the shifts in use of one kind of perceptual cue to another when categorizing at the basic or global level. 4. It may be of interest that in various forms of meaning breakdown (semantic dementia), the most resilient aspect of knowledge about an object such as a dog is that it is an animal. Even when patients can no longer recognize the word dog or a picture of a dog or say anything specific about dog, they can often still say that it is an animal (Saffron and Schwartz 1994 ). S. In the caseof blind infants, an exploring hand is required instead (Landau 1988 ). 6. Only the presentprogressiveing , which express -schema esanother preverbal image , traversal of a path, is learned earlier; seeBrown ( 1973 ). 7. We also must not forget the arbitrary aspectsof languagethat arise from historical accident or for other reasons . These are more frequent than we sometimes realize. For example , in London one seessigns in the Underground saying " No Smoking Anywhere on This Station," which sounds distinctly odd to American ears, but of course perfectly fine to the British. I assumethat the British expressioncan be traced to the fact that railway stations originally consisted of raised platforms, but the example is typical of the many arbitrary aspects of languagethat children must learn. References Baillargeon, R. ( 1993 ). The object concept revisited: New directions in the investigation of infants' physical knowledge. In C. Granrud ( Ed.), Visualperception and cognition in infancy, 265- 315. Hillsdale, NJ: Erlbaum. -Collier and Baillargeon, R. ( 1995 ). A model of physical reasoning in infancy. In C. Rovee L . Lipsitt (Eds.), Advancesin infancy research , vol. 9. Norwood , NJ: Ablex.

382

Jean M . Mandler

' : Intrinsic imageand motions Bertenthal , B. ( 1993 ). Infants perceptionof biomechanical in infancy and Visual Ed. , . In C. Granrud constraints cognition based ), perception ( knowledge - 214 . . Hillsdale 175 , NJ: Erlbaum : What rolesdo cognitivepredispositions Bowerman system , M. ( 1989 ). Learninga semantic - 169 . .), The teachability , 133 of language ? In M. L. Rice and R. L. Schiefelbusch (Eds play . P. H. Brookes : Baltimore . Cambridge : Theearly stages , MA : Harvard University Brown , R. ( 1973 ). A first language Press . . : Polysemy , andthestructure , semantics of thelexicon , C. M. ( 1988 ). Thestoryof over Brugman . NewYork: Garland . . Cambridge in childhood , MA: MIT Press change , S. ( 1985 ). Conceptual Carey : andKorean in English motionevents to express Choi, S., andBowerman , M. ( 1992 ). Learning 121 . J 83 4 . lexicalization , , of language Theinfluence patterns Cognition specific basic based the formationof perceptually on . Studies 1994 P. C. Elmas ) , ( , P. D., and Quinn - 917 . . ChildDevelopment in younginfants , 65, 903 levelcategories . In R. J. Jarvellaand frameworkfor spatialdeixis Fillmore , C. ( 1982 ). Toward a descriptive . NewYork: Wiley. .), Speech W. Klein (Eds , place , andactions . . Cambridge Fodor, J. ( 1981 , MA: MIT Press ). Representations natural : Linguisticrelativity versus beforeverbs Gentner , D. ( 1982 ). Why nounsare learned . Vol. 2, Language , and , thought . In S. A. KuczajII (Ed.), Language development partitioning . Erlbaum NJ: Hillsdale culture , , ). R. M. , W. B., and Parillo, M. ( 1995 ' Golinkoft, , C. B., Frawley , K., Mervis , Hirsh-Pasek andW. Merri. In M. Tomasello of verbs to theacquisition canbeextended Lexicalprinciples 's - 221 . Hillsdale , : Young children , 185 names .), Beyond man(Eds of verbs acquisition for things . NJ: Erlbaum - 346 -grounding . a D, 42, 335 . Physic Harnard problem , S. ( 1990 ). The symbol , and , imagination : The bodily basisof meaning Johnson , M. ( 1987 ). The body in the mind . Press of : . ChicagoUniversity Chicago reasoning 's . Cognitive in children concepts JonesS. S., and Smith , LB . ( 1993 ). Theplaceof perception . 139 8 113 , , Development 's from children access esto conscious ; Evidence '-Smith Karmiloft , A. ( 1986 ). Frommetaprocess . and repairdata. Cognition , 23, 95 147 metalinguistic . In S. Carey on concepts asconstraints beliefs of theoretical Keil, F. C. ( 1991 ). Theemergence . Erlbaum NJ: . Hillsdale 256 237 mind The Eds . , , and R. Gelman( of ), epigenesis -old infants . Posterpresented in 5.5-month of containment . 1991 V. T. Kolstad , ) Understanding ( in Child Development , , Seattle at the BiennialMeetingof the Societyfor Research . April . Press : Universityof Chicago . Chicago ', G. ( 1987 Lakoft ,fire , anddangerous things ). Women

Preverbal Representationand Language

383

Landau . (1988 ,D anduse ). Theconstruction of spatial in blindandsighted knowledge -Davis children . In J. Stiles , M. Kritchevsky , andU. Bellugi Eds .), Spatial : Brain ( cognition - 371 bases and . Hillsdale , 343 development : Erlbaum . , NJ " and" where " in Landau ., and Jackendoff ,D , R. (1993 ). " What and spatial language spatial . - 265 Behavioral and Brain cognition Sciences . , /6, 217
-inanimate , M. (1992 Legerstee of theanimate ). A review distinction in infancy : Implications for models of social andcognitive . EarlyDevelopment - 67 andParenting knowing . , 1, 59 Leslie . , A. (1984 Infant of a manual ) perception . BritishJournal pickup event of Developmental - 32 . , 2, 19 Psychology

Mandler , J. M. (1988 to build a baby : Onthe ). How oranaccessible development representational - 136 . Cognitive system . , 3, 113 Development Mandler , J. M. (1991 . In L. A. Sutton ). Prelinguistic primitives andC. Johnson .), Proceedings (Eds Annual of theSeventeenth - 425 Meeting of theBerkeley 414 . Linguistics , Society : Berkeley , CA Berkeley . Linguistics Society Mandler to build , J. M. (1992 a baby : II. Conceptual ). How . Psychological primitives Review , - 604 99 . , 587
Mandler , J. M ., Bauer, P. J., and McDonough , L . ( 1991 ) . Separatingthe sheepfrom the goats: . Cognitive Psychology Differentiating global categories , 23, 263- 298. Mandler , J. M ., and McDonough , L . ( 1993 ) . Concept formation in infancy. Cognitive Development , 8, 291- 318. Mandler , J. Mo ., and McDonough , L . (in press ) . Drinking and driving don ' t mix: Inductive generalization in infancy . Cognition. Mandler, J. M ., and McDonough, L . (in press ) . Nonverbal recall. In N . L . Stein, P. O. Ornstein, B. Tversky, and C. Brainerd ( Eds.), Memory for everydayand emotional events , Hillsdale, NJ: Erlbaum.

Maratsos , M. ( 1983 current issues in thestudy ). Some of theacquisition of grammar . In J. H. FlavellandEM . Markman .), Cognitive (Eds , Vol. 3 of P. H. Mussen development .), (Ed Handbook . NewYork: Wiley of child . psychology
Mervis , C. B., and Rosch , E. ( 1981 of natural objects ). Categorization . AnnualReview of . , 32, 89- 115 Psychology Nelson , K. ( 1985 : Theacquisition ). Makingsense . SanDiego of shared meaning , CA: Academic Press . Nelson , K., and Lucariello J. 1985 of meaningin first words ). The development . In M. 's , - ( Barrett(Ed.), Children singlewordspeech , NewYork: Wiley. , J. ( 1951 Piaget ). Play, dreams , andimitationin childhood . New York: Norton. , J. ( 1967 Piaget studies . NewYork: RandomHouse ). Six psychological . . , W. V. ( 1977 Quine Natural kinds . In S. P. ) Schwartz (Ed.), Naming , necessity , andnatural - 177 kinds . Ithaca , 155 , NY: CornellUniversityPress .

384

Jean M . Mandler

) . On categorization in early infancy . Merrill - Palmer Quinn, P. C., and Elmas, P. D . ( 1986 Quarterly, 32, 331 363. ) . Evidence for representationsof Quinn, P. C., Elmas, P. D ., and Rosenkrantz, S. L. ( 1993 , perceptual similar natural categories by 3-month -old and 4-month old infants. Perception 22, 463- 475. ) . Perceptualorganization and categorization in young Quinn, P. C., and Elmas, P. D . (in press -Collier and L . Lipsitt ( Eds.), Advancesin Infancy Research , Vol . 11. infants. In C. Rovee Norwood , NJ : Ablex. Saffron, E. M ., and Schwartz, M . F. ( 1994 ). Of cabbagesand things: Semantic memory from - A tutorial review. In C. Umilta and M . Moscovitch ( Eds.), a neuropsychologicalperspective and unconscious Attention and performancexv : Conscious , Cambridge, information processing . MA : MIT Press -making capacity. In D . I . Slobin Slobin, D . I . ( 1985 ) . Crosslinguistic evidencefor the language - 1256 . , 1157 , Vol . 2, Theoretical issues linguistic study of languageacquisition (Ed.), The cross Hillsdale, NJ: Erlbaum. , K . ( 1992 ) . Origins of knowledge. , E. S., Breinlinger, K ., Macomber, J., and Jacobson Spelke 632. 99 605 Review , , Psychological Sweetser of semantic , E. ( 1990 ) . From etymology to pragmatics: Metaphorical and cultural aspects . structure. Cambridge: Cambridge University Press ) . Force dynamics in languageand thought . In W. H . Eilfort , P. D . Kroeber, Talmy , L . ( 1985 on Causativesand Agentivity at the and K . L . Peterson (Eds.), Papersfrom the Parasession : Twenty -first RegionalMeeting, Chicago Chicago Linguistic Society. . In J. H . ). On the expressionof spatiotemporal relations in language Traugott , E. C. ( 1978 CA : Stanford Stanford Word structure . Vol . 3 human Universals Ed. , , , language ), of Greenberg( . . University Press

Chapter10 : A Cross for Language linguistic LearningHow to StructureSpace Perspective


Melissa Bowennan

Space is an important preoccupation of young children. From birth on, infants explore the spatial properties of their environment, at first visually and proprioceptively , and then through action. With improved motor control during the secondyear of life , their spatial explorations becomemore complex, and they also begin to talk . Early commentson spacerevolve mostly around motions, with remarks about space about static position also beginning to appear in the secondhalf of the secondyear. -month -old girlleaming English are typical : The following utterancesfrom a nineteen ( I ) a. In. (About to climb from the grocery compartment of a shopping cart into the child seat.) b. Monies. In. (Looking under couch cushions in searchof coins shehasjust put down the crack betweenthe cushions.) c. Balls. Out. ( Trying to push round porthole piecesout of a foam boat puzzled . Books. Out. Books. Back. (Taking tiny books out of a fitted caseand putting them back in.) e. Monkey up. (After seeinga live monkey on TV jump up on a couch.) f . Down. Drop! (After a toy falls off the couch where sheis sitting .) g. On. (Fingering a pieceof cellophane tape that she finds stuck on the back of her highchair.) h. Off. (Pushing her mother' s hand off the paper sheis drawing on.) i. Openmommy. ( Wantsadult to straighten out a tiny flexible mommy doll whose legsare bent up) . 1 Remarks like theseattract little attention - the view of spacethey reflect is obvious : on closer to adult speakersof English. But their seeming simplicity is deceptive inspection, theselittle utterancesraise fundamental and difficult questions about the relationship betweenthe nonlinguistic developmentof spatial understanding and the . How do children come to analyzecomplex eventsand acquisition of spatial language often relationships, involving novel objects in novel configurations, into a set of

386

Melissa Bowennan

discretespatial categoriessuitable for labeling? How do they decidewhich situations are similar enough to be referred to by the sameword (e.g., the two ins above, and the two outs)? Why is their choice of spatial word occasionally odd from the adult point of view (e.g., openfor unbending a doll ) and yet, at the sametime, why is it so often appropriate? For many years it has been widely assumedthat the meaningschildren assign to spatial words reflect spatial concepts that arise in the infant independently of language , under the guidance of both built -in perceptual sensitivities and explorations ; McCunewith the spatial properties of objects (e.g., Johnston and Slobin 1979 Nicholich 1981 , the words in and out in the examples ) . For example ; Slobin 1973 to do with containment, on and off, notions above might label preverbally compiled notions of contact and support; and up and down, notions of motion oriented with respectto the vertical dimension. This view is buttressed by an impressive array of researchfindings with infants: for instance , toddlers clearly know a lot about spatial relationships before they begin to talk about them. It also draws support from studies that stressthe existenceof perceptual and environmental constraints on spatial cognition and that postulate a close correspondencebetween the nonlinguistic and linguistic structuring of space ; Olson and ; Miller and Johnson- Laird 1976 ; H . H . Clark 1973 (e.g., Bierwisch 1967 use of spatial and adult ) . In this view the similarity between child Bialystok 1983 morphemesis not surprising: the properties of human perception and cognition mold both the meaningsthat languagesencodeand the spatial notions that speakersof all agesentertain. I will argue that the path from a nonlinguistic understanding of spatial situations to knowledge of the meanings of spatial morphemes in any particular language is . The meaningsspatial morphemescan express far lessdirect than this view suggests and Jackendoff 1993 . Landau e. ; Talmy 1983 ) , but are undoubtedly constrained ( g , is in the differences way space structured recent researchis beginning to uncover striking for purposesof linguistic expression(seealso Levinson, chapter 4, this volume) . differ , nonlinguistic spatial developmentalone cannot be To the extent that languages with the conceptual children counted on to provide packaging of spacethey need for ' s children . Whatever form their native language nonlinguistic spatial understanding may take, this understanding must be applied to the task of discovering how spaceis . Although the interaction in development between organized in the local language of spatial structuring is still poorly understood, sources and linguistic nonlinguistic recent cross linguistic work suggeststhat the linguistic input begins to influence the , the child whose utterancesare shown child at a remarkably young age: for instance above is barely more than a year and a half old , but her utterancesalready reflect a

Howto Structure for Language Learning Space

387

-specific spatial organization (Bowennan 1994 , 1996 ; Choi and profoundly language Bowennan 1991 ). I first review studies suggestingthat nonlinguistic spatial development indeed lays an important foundation for the child ' s acquisition of spatial words. But this is not enough: Next I discuss the problem created for learners by the existence of cross , and review linguistic differencesin the way spaceis carved up into categories some other aspectsof spatial structuring that clearly must be learnedon the basis of . After this stagesetting, I describetwo studiesI have conducted, linguistic experience with together Soonia Choi , to explore how children who are learning languagesthat classify spacein interestingly different ways arrive at the spatial categoriesof their . Finally , I consider what thesestudiessuggestabout the interaction between language , nonlinguistic and linguistic factors in the acquisition of spatial semantic categories and about the kinds of hypotheseschildren may bring to the acquisition of spatial words.

10 .1 Cognitive of SpatialSemantic Underpinnings Development


-independent perceptual and If any domain has a plausible claim to strong language . The ability to perceiveand interpret spatial relationships cognitive organization, it is space is clearly fundamental to human activity , and it is supported by vision and other highly structured biological systems(e.g., DeValois and DeValois 1990 ; von der Peterhans and 1984 . Our mental of , Heydt , ) Baumgartner representations spaceare constrained not only by our biology but also by their fit to the world " out there" : if we try to set an object down in midair , it falls, and if we misrepresentthe location of something, we cannot find it later. Little wonder it has seemed likely to many investigators that the languageof spaceclosely mirrors the contours of nonlinguistic spatial understanding. Several kinds of empirical evidenceindeed support the assumption that children know a great deal about spacebefore they can talk about it , and that they draw on this knowledge in acquiring spatial words.

10.1.1.1 Piagetian Theory: Building Spatial Representatio . . through Action The for the modern that children original impetus day hypothesis map spatial words onto ' came from the fit between preestablished spatial concepts striking Piaget s arguments about the construction of spatial knowledge in young children and the course of 2 acquisition of spatial words. According to Piaget and Inhelder ( 1956 ), spatial do not reflect the of but are built on the level of concepts directly perception space up

388

Melissa Bowerman

' representation through the child s locomotion and actions upon objects during the first eighteenmonths or so of life. " The earliest spatial notions are thus closely bound to object functions such as containment or support, and to the child ' s concern with . Recall here the toddler ' s pleasurewith pots and pans, towers and object permanence . In the next phase , children construct the spatial notions of proximity , hiding games " Johnston 1985 969 . After the and order , ( ) separation, surrounding emergenceof " thesenotions- often called topological" because they do not involve perspectiveor - projective and Euclidean spatial notions are gradually constructed. measurement This order is closely mirrored by the sequencein which children acquire locative morphemessuch as the English prepositions. Locatives begin to come in during the second year of life , but their acquisition is a drawn-out affair. Within and across , they are acquired in a similar order: first come words for functional and languages topological notions of containment (in), support and contiguity (on), and occlusion , between (under ); then for notions of proximity (next to, beside ), and finally for relationships involving projective order (in front of and in back of/ behind) . This protracted and consistent order of acquisition of locatives, coupled with its correspondenceto Piaget' s claims about the course of development of spatial knowledge, has beentaken as strong evidencethat the learning of locatives is guided and pacedby the maturation of the relevant spatial notions (Johnston 1985 ; Johnston and Slobin 1979 ; Parisi and Antinucci 1970 ; Slobin 1973 ). 10.1.1.2 Infant Spatial Perception With the explosion over the last decade of research on infant perception, the evidence for prelinguistic spatial concepts has becomesteadily more impressive . Challenging Piaget' s emphasison the critical role of action in the construction of spatial concepts , studies show that even very young infants are sensitive to many spatial and other physical properties of their environment . For example , habituation studies of infant perception have established that within the first few days or months of life , infants can distinguish betweenscenes and categorizethem on the basis of spatial information such as above below (Antell and Caron 1985 ; Quinn 1994 ; Behl-Chadha and Elmas ), left -right (Quinn and Elmas 1986 1995 ; Quinn and Bomba 1986 ; ), and different orientations of an object ( Bomba 1984 . Colombo et al. 1984 Studies the related ) using technique of time spent looking at possible versus impossible events show that by a few months of age infants also recognizethat objects continue to exist even when they are out of sight (Baillargeon 1986 , 1987 ), that moving objects must follow a continuous trajectory and cannot pass through one another (Spelke et al. 1992 ), and that objects deposited in midair will fall (Needham and Baillargeon 1993 ). The proper interpretation of such findings is still a matter of debate. Some researchers argue that children can representand reasonabout the physical world with

Learning How to Structure Spacefor Language

389

core knowledge that is derived from neither action nor perception, but is inborn " ; Spelke et al. 1994 (e.g., Spelke et al. 1992 ). Others argue instead for highly constrained learning mechanismsthat enable babies to quickly arrive at important " generalizationsabout objects (Needham and Baillargeon 1993 , 145 ) or for powerful abilities to detect perceptual invariances in stimulus information (Gibson 1982 ). In any event, there can be little doubt that even babieswell under a year of age command a formidable set of spatial abilities. 10.1.1.3 TemJM }RajPriority of Nontinguisticover Linguistic Spatial Knowledge Consistent with this, whenever children' s nonlinguistic understanding of particular aspectsof spacehas beendirectly compared with their knowledge of relevant spatial words, an advantageis found for nonlinguistic understanding. For example, Levine and Carey ( 1982 ) found that children can success fully distinguish the fronts and - as demonstrated backs of objects such as dolls, shoes chairs and stoves , , , for example their to orient them to form a , by ability appropriately parade well before they can pick out theseregions in responseto the wordsfront and back (seealso Johnston 1984 , 1985 for a related study) . Similarly , E. V . Clark ( 1973a ) found that young children play with objects in ways that show an understanding of the notions of containment and support before they learn the words in and on (seealso Freeman , Lloyd , and Sinha 1980 ). 10.1.2 Relianceon NonHnguisticSpatial Knowledgein Learning New Spatial Words Not only do children show a grasp of a variety of spatial notions before they can talk about them, but they also seemto draw on this knowledge in learning new spatial words. Young children often show signsof wanting to communicate about the location of objects, and before acquiring spatial morphemes , they may do so simply by two nouns or a verb and a noun with what seems to be a locative intention combining " towel bed" for a towel on a bed and " sit " for , , , example pool for sitting in a ; Bowerman 1973 ; Siobin 1973 wading pool (Bloom 1970 ) . The prepositions most often called for but usually missing in the speechof R. W. Brown' s ( 1973 ) three were in and on. At a later these were the first two , subjects stage prepositions to be . This has to researchers that the motor driving the reliably supplied pattern suggested acquisition of locative morphemes is the desire to communicate locative meanings that are already conceptualized(e.g., Siobin 1973 ). 10.1.2.1 Strategiesfor Interpreting Spatial Words Children ' s nonlinguistic spatial notions also affect how they interpret spatial words in the speechthey hear. For how children comply with instructions to place , in an experiment assessing example A in on or BE under . V. Clark ( 1973a , , object object ) found that her youngest

390

Melissa Bowerman

' ' ' ' , and on if B had a flat , supporting subjects put A in if B was container-shaped surface , regardlessof the preposition mentioned. This meant that they were almost always correct with in, correct with on unless B was a container, and never correct ' with under. Clark proposed that prepositions whose meaningsaccord with learners nonlinguistic spatial strategiesare acquired before prepositions whose meaningsdo not ; hence , in is easierthan on, which in turn is easierthan under.

, range of referent situations that share an abstract spatial similarity . For example " to all use old child extended on the first of that a twelve month day up reporting vertical movement of the child himself or of objects," Nelson ( 1974 , 281) proposed that " there is a core representationof this action concept . . . something like Vertical ' Movement." Similarly, Bloom ( 1973 , 29) concluded that the use of up by Leopold s ) daughter Hildegard in connection with objects and people, including herself, ( 1939 " " is a function of the underlying conceptual notion itself. On the basis of data from ' "' her two subjects , Gruendel ( 1977 ) concurred that upness is an early-cognized or " " conceptualizedrelation and added that in also appearedfrom the outset to take a readily generalizable form , suggestingthat meaning relations had been articulated before production began." In studying relational words in the one-word stagespeech of five children, McCune-Nicholich ( 1981 , along ) found that up, down, back, and open with severalother relational words, came in abruptly , generalizedrapidly , and were lesslikely to be imitated than other words. She concluded from this that the words - specifically, operative knowledge of the encodepreestablishedcognitive categories late sensorimotor period . . and Overexte . io. Further evidencethat children draw 10.1.2.3 Underextensio in on their nonlinguistic spatial conceptions acquiring spatial words is that they sometimesapply the words to a range of referentsthat differs systematicallyfrom the adult range. For example, English-speaking children first use behind and in front of only in connection with things located behind or in front of their own body; the " " " intended meanings seemto be " inaccessibleand/ or hidden versus visible. Later behind is also used when a smaller object is next to and obscured by a larger one (under is also sometimes inappropriately extended to these situations) . Still later, behind and in front of are also produced when an object is adjacent to the back or front of a featured object such as a doll . Finally they are also used projectively to " mean " second ) . According to Johnston, / first in the line of sight (Johnston 1984 " when we seelocative , predictable meaningschangeover many months in a specific " fashion, we are invited to assumethat new spatial knowledge is prompting growth

Learning How to Structure Spacefor Language

391

(p . 421) . Another example of nonadultlike usageis the common overextensionof the verb opento actions like pulling apart paper cups or Frisbees , unlacing shoes , taking a pieceout of a jigsaw puzzle, and pulling a chair out from a table ( Bowerman 1978 ; and Atkinson 1978 . Nonadultlike uses whether E. V. Clark 1993 see also Griffiths , ; ) restricted or overextendedrelative to adult norms, have been interpreted as strong -independentspatial notions. evidencefor children' s reliance on their own language es that infants understand a great deal about The literature just reviewedestablish before words , that they learn spatial words in a consistent space they acquire spatial in order roughly mirroring the order which they come to understand the relationships the words encode , and that they rely on their spatial understanding in learning new words- for example , in making predictions about what thesewords could mean and in extending them to novel situations. There can be little doubt , then, that nonlinguistic ' spatial development plays an important role in children s acquisition of . But does the evidenceestablish that children map spatial words spatial morphemes ? Here there is still room for directly onto spatial concepts that are already in place doubt .

' s Semantic of Space ? 10 .2 DoesLanguage Input Playa Rolein Children Structuring


In a dissenting view, Gopnik ( 1980 ; Gopnik and Meltzoff 1986 ) has argued that in words do not fact early spatial expresssimple spatial concepts that are already " " thoroughly understood, but , rather, ones that are emerging and still problematic for children of about eighteen months. She notes that although by about twelve to fourteen months children show an interest in how objects fall and can be balanced , and in the properties of containers, there is evidencethat even fifteen- to twenty-onemonth-olds do not fully understand gravity and movement into and out of containers ' . For instance , until seventeenmonths Piaget s ( 1954 ) daughter Jacqueline threw objectsto the ground rather than dropping them, and at fifteen months shewas still trying to put a larger cup into a smaller one. Gopnik ( 1980 ) suggeststhat language - for may in fact help children solve spatial puzzlesduring the one-word stage " " " " , hearing adults say up and down in connection with their experiments example with gravity " may help [children] to understandthat all thesepreliminary actions lead " . 291 . to the sameconsequence (p ) -month -olds learn words How can we reconcile Gopnik ' s hypothesis that eighteen for spatial concepts that are still problematic for them with evidence that much ? To younger babieshave a relatively sophisticatedperceptual understanding of space how between what infants seem able to and the perceive they act discrepancy explain ' failure to search for hidden objects despite upon objects (or do not act cf. infants evidencethey remember the existenceand location of these objects; seeBaillargeon

392

Melissa Bowerman

et al. 1990 ), some researchershave suggestedthat core knowledge of the physical properties of objects and their relationships is modular , and at first somewhat inaccessible to other domains of child thought and action (Spelke et al. 1994 ) . Others in to limitations skills. In order to success early point problem solving fully manipulate , children not only must have spatial knowledge but also be able to devise space and executea situation-appropriate plan, and this often appears to be difficult for reasonsindependentof the actor' s spatial understanding (Baillargeon et al. 1990 ). For somespatial notions, however, there is reasonto suspectthat despiteevidence for some early perceptual sensitivity, understanding may still be incomplete until , by as early as ) . For example eighteenmonths of ageor beyond (seealso Gopnik 1988 six months, babies anticipate that an opening in the surface of an object allows a second , smaller object to pass through (Sitskoorn and Smitsman 1995 ; see also Pieraut- Le Bonniec 1987 . But it is not until about seventeen to months that ) twenty in seem to that order to contain a container must have a , they recognize something bottom . Only at this age do they ( I ) look longer at an impossible event in which a bottomless cylinder seemsto contain sand than at a possible event with an intact cylinder, and (2) choosewith more than chance frequency an intact cup over a bottomless cup when encouraged to imitate an action of putting cubes in a cup and ; seealso Bower 1982 , and MacLean rattling them (Caron, Caron, and Antell 1988 and Schuler 1989 ) . Similarly , although by four to six months infants recognizethat an object cannot stay in midair without any support at all (Needhamand Baillargeon 1993 ; Sitskoorn and Smitsman 1995 ; Spelkeet al. 1992 ), eventoddlers as old as thirty months are not surprisedwhen a block construction staysin place after one of its two critical supporting blocks is removed (Keil 1979 ). These findings are consistent with Gopnik ' s proposal that toddlers talk about spatial events whose properties they are still in the processof mastering, and lend someplausibility to her suggestionthat linguistic input - hearing adults usethe same word across a range of situations that are in some way similar - may contribute to the processof mastery. But although Gopnik stress esthat languagecan help children to consolidate their grasp of spatial notions, she seemsto assumethat the fonD the " conceptswill take is ultimately detennined by nonlinguistic cognition : the cognitive -month -olds are similar enough so that they will be likely to acquire concernsof all 18 the samesorts of meaningsby the end of the one-word period" (Gopnik and Meltzoff 1986 , 219, emphasisadded). So linguistic input servesprimarily to reinforce natural tendencies ; it does not in itself introduce novel structuring principles. As long as we restrict our attention to children learning our own native language , we have no reason to doubt that linguistic input can at most only help to reinforce spatial concepts that children will acquire in any event. This is becausethe spatial " " categoriesof our languageseemso natural to us that it is easyto imagine they are

Leamin ~ How to Structure Spacefor Language

393

the inevitable outcome of cognitive development. But a close look at the treatment of space in diverse languages suggeststhat language may playa more powerful . For example , hearing the sameword repeat structuring role than Gopnik suggests ' edly across differing events might draw children s attention to abstract properties shared by these events that might otherwise pass unnoticed. Let us consider this possibility more closely. 10.2.1 CrosslingWsticPerspectives on Spatial Categorization no two , , attributes, or spatial configurations Objectively speaking objects, events are completely identical- consider two dogs, two events of falling , or two acts of kindness . But each discriminably different referent does not get its own label: one of the most basic properties of languageis that it carvesup the world into (often overlapping of things that can all be referred to with the sameexpression ) classes , such as dog, pet, fall , open and kindness . These classes or are , , , categories composed of entities that can be treated as alike with respectto someequivalencemetric. Under the hypothesis that preexisting spatial concepts provide the meanings for children' s spatial words, it is assumedtheseconceptsprovide the grouping principles, or , put differently , the metric along which a word will be extendedto novel situations. But what principles are these ? Here it is critical to realize that there is considerable variation across languages in which similarities and differences " count " in establishing whether two spatial situations belong to the same spatial semantic category- that is, can be referred to with the samespatial morpheme. As a simple illustration , let us consider some configurations involving the of teninvoked notions of contact, support, and containment: (a) " cup on table," (b) " apple in bowl ," and (c) " handle on cupboard door " (cf. figure 10.1) . In many languages , " handle contact with and a vertical surface such as , relationships involving support by on cupboard door ," are treated as similar to relationships involving contact with " " and support by a more-or -lesshorizontal surface , such as cup on table. In English, " " " for example , the spatial relationships in (a) cup on table and (c) handle on cupboard door " are both routinely called on; a different word - in- is neededfor " containment " relations like b " " ( ) apple in bowl. This grouping strategy (shown in figure 10.la ) seemsto make perfect sense : after all , both " cup on table" and " handle on " " " door , but not apple in bowl , involve contact with and support by an external . surface But sensibleas this strategy may seem , not all languagesfollow it . In Finnish, for " " , situations like (c) handle on cupboard door are grouped linguistically example with those like (b) " apple in bowl " (both are encoded with the inessivecaseending " " " " -ssa , usually translated as in ); for (a) cup on table a different case ending " " . The motivation for this , -Ila, usually translated as on ) is needed (the adessive

394

Melissa Bowennan

ON

OW SSA R
b. Finnish

8. English

EN

R AAN
c. Dutch

R
d. Spanish . situations in English, Finnish, Dutch , and Spanish

Learning How to Structure Spacefor Language

395

" " " " -oriented support, on a dimension of intimacy or incorporation (other surface " " " , in , include Band-aid configurations that can be encodedwith the caseending ssa " " " " " " on leg, ring on finger, coat on hook , sticker on cupboard," and " glue on " Bowerman 1996 . scissors ; , ) In still a third pattern, exemplified by Dutch , situations like (c) can be collapsed ' ' ' ' together with neither (a) (op on ! ) nor (b) (in in ), but are characterizedwith a third ' ' spatial morpheme, Dan on2 , that is somewhatspecializedto relations of hanging and other projecting attachment, (e.g., " picture on wall ," " apple on twig," " balloon on " " " " " , 1996 string, coat on hook , hook on door ; Bowerman 1989 ); this pattern is shown in figure IO.lc . And in a fourth pattern, displayed by Spanish , it is quite unnecessaryto differentiate among (a), (b), and (c)- a single preposition, en, can , the situations can be comfortably be applied to all of them! (figure IO.ld ) . (If desired ' on ' for a and dentro de ' inside of ' for use of encima de of distinguished by ( ) top 3 (b . Thesevarious classification patterns, although different, all make good sense class membership is in each caseestablishedon the basis of an abstract constancy in certain properties, while other properties are allowed to vary . " " " In still other languages , the familiar notions of contact and support and containment " undergo much more radical deconstruction than in the examplesshown so far. For example , in Tzeltal, a Mayan language of Mexico , there is no all -purpose containment word comparable to English in (P. Brown 1994 ) . Different forms are neededto indicate that ' ' (2) a. A man is in a house (ta y -util at its-inside ) b. An apple is in a bowl (pachal ' be located' , of somethingin a bowl-shaped containeror of the container itself ) c. Water is in a bottle (wax -al ' be located' , of somethingin a taller- than- wide rectangular or cylindrical object or of the object itselfd . An apple is in a bucket of water (t 'umul ' be located' immersedin liquid) e. A bag of coffee is in a pot (xojol ' be located' , having been insertedsingly into a closelyjitting container ) f. Pencilsare in a cup (xijil ' be located' , of long/ thin object, havingbeeninserted carefully into a boundedobject) . A bull is in a corral (tik 'il ' be located' , having been insertedinto container g with a narrow opening ). Similarly , in Mixtec , an Otomangueanlanguagealso spoken in Mexico, there is no all -purpose contact-and-support word comparable to English on. Instead, spatial " " relationships between two objects are indicated by invoking a body part of the

grouping - shown in figure 10.1b - may be that attachment to an external surface can be seen as similar to prototypical containment , and different from horizontal

396

Mell ~ ~

Bowennan

reference object in a conventionalized but completely productive way (Brugman : 1983 , 1984 ; Lakoff 1987 ) . For example ' .located] siki -fte ? e ' animal.back-house (3) a. A man on a roof ([ be ) ' ' b. A man on a hill ( . . . sini yuku head hill ) c. A cat on mat ( . . . nuu-yuu ' face-mat' ) d. A man on a tree branch ( . . . nda? a-yunu ' arm-tree' ). Someof theseforms can also be usedfor an area adjacent to the named " body part " ' ' of the referenceobject, for example , [be.located] sini-yunu head-tree could be said of a bird either located on the top of a tree, or hovering abovethe tree. Comparable body part systems are also employed by Tzeltal and other Mayan languages (Levinson 1994 ) and many other languagesof Meso-America and Africa , although details of body-part assignmentvary widely (Heine 1989 ; MacLaury 1989 ). Let us take an example from a different domain, manipulations of objects. Consider thesethree actions: (a) " hanging up a coat," (b) " hanging up a mobile," and (c) " " hooking two toy train cars together. English speakerswill typically use hang (up) for both (a) and (b), conceptualizing them as similar on grounds that in both events , an entity is arranged so that it dangles downward with gravity . They will use a - perhapshook together - for (c), which lacks this property . This different expression .2a. Korean speakerswill make a different categorization pattern is shown in figure I O implicit grouping, using the verb keha for both (a) and (c), and a different verb, taha, for (b) . (Korean lacks the semantic category associated with English hang.) This pattern is shown in figure IO.2b. Why is hanging up a coat assignedto the same spatial category as hooking together two train cars? Becauseof the way they are attached: in both events , an entity is fixed to something by mediation of a hooking keha whereas in the " hanging a mobile" event shown in (b), the ), configuration ( entity is attached directly (taha ; this verb could also be usedfor attaching asidewaysprojecting handle to a door ) . Notice that both theseclassification strategiescan achievethe samecommunicative effect- e.g., to call a listener' s attention to an action of hanging up a coat. But they do so in different ways. When English speakersusehang for hanging up a coat, they assertthat the coat is arranged so that it dangleswith gravity , but they say nothing about how it is attached; the listener must infer the most likely kind of attachment on the basis of his knowledge of how dangling coats are usually attached. Conversely , when speakersof Korean use ke/ta for the same action, they assert that the coat is attached by hooking, but they say nothing about dangling with gravity; again, the listener must infer on the basisof his world knowledge that when coats are hooked to . For communicative purposes , , dangling with gravity is likely to ensue something then, the expressionsof the two languageare equivalent: in concrete contexts, they

Learning How to Structure Spacefor Language

397

HANG HO En TO G \ G ~ ~ ~
KELTA \ G ~ ~ ~ ~ ~ ~

b. Korean Figure 10.2 Classification of three actions in and Korean. English

398

Me ]i~

in the listener' s mind. But the spatial conceptsunderlying can invoke the samescenes the words are different, and so, consequently , are the overall setsof eventsthey pick out. " It is clear, then, that the situations that fall together as instances of the same " spatial category vary widely across languages in accordance with differences in the properties of situations that are conventionally used to compute similarity for -cut eachother in complex purposesof selectinga word . The resulting categoriescross in which are distinguishedin Mixtec , all involve ways. For example, the situations (3) , an object resting on a horizontal supporting surfaceand so are relatively prototypical for English on. However, Mixtec does not simply subdivide the English category of on more finely : recall that situations that English obligato rily distinguishes as on versusaboveoften fall together in Mixtec - both instantiate adjacencyto the named body part of the referenceobject. In order to talk about space , then, it is not sufficient for children to understandthat one object can be put above, on, below, inside, or that if not fall supported, objects A perceptual or action-basedunderstanding of and so on. occluding another object, what is going on in given spatial situations is probably a necessarycondition for , but this knowledge alone does not buy children knowledge learning to talk about space - for example of how to classify space in their language , it will not tell them same the be seen as in bowl should an a whether apple spatial relationship instantiating as a bag of coffee in a pot, or whether hanging a coat should be treated as more similar to hanging a mobile or to hooking two train cars together. To be able to make -appropriate way, it is essentialto discover the implicit thesedecisionsin a language words are distributed acrosscontexts.4 in how spatial patterning 10.2.2 What Else Doesthe Child Needto Learn? Determining the right way to categorizespatial relations is an important problem for the language learner, but it is not the only task revealedby an examination of how . A few others can be briefly summarized as different languages deal with space ! follows. ' 10.2.2.1 What Do Languages Conventionally Treat as ' Spatial Relatio~ . . to BeginWith? In the discussionof figure 10.1, I simply assumedthat all the configurations " shown can be construed as " spatial - the problem was just to identify which properties languagesare sensitive to in classifying them as instancesof one spatial category or another. But languagesin fact differ not only in how they classify spatial c~nfigurations, but also in the likelihood that they will treat certain configurations as spatial at all. Some relationships seemto be amenableto spatial characterization perhaps in all - for example, a cup on a table, an apple in a bowl , and a tree adjacent languages

Learning How to Structure Spacefor Language

399

to a house . But other relationships are treated more variably . In some languages , including English, part -whole relations are readily described with the same spatial expressionsused for locating independent objects with respect to each other; e.g., " the handle on the " " the muscles in cupboard door (is broken) my left calf (are " " sore) , and the lid on this pickle jar (has a funny picture on it ) ." But in many , analogous constructions sound odd or impossible ; for example , speakers languages of Polish consistently usegenitive constructions along the lines of " the handle of the " " " " " cupboard door , the musclesof my left calf, and the lid of the pickle jar . " " In a second example , consider entities that do not have good Gestalt, such " as unbounded substanceslike glue, butter , and mud , or bounded negative object " ; Landau and Jackendoff 1993 parts (Herskovits 1986 ) like cracks and holes. English speakers are again relatively liberal in their willingness to treat these entities as " located " " !?" (or " Why do my objects - e.g., Why is there butter on my scissors " " ' scissorshave butter on them? ) and There s a crack in my favorite cup!" But speakers of many languagesresist " locating" such entities with respectto another entity , " " preferring instead constructions comparable to My scissorsare buttery/ havebutter " ,, 6 and My cup is cracked / has a crack. Differencesin the applicability of spatial languageto entities like butter and cracks seem to reflect pervasive cross linguistic differences in conventions about whether constructions that are typically usedfor locating objects- for example , for narrowing the searchspacein responseto a " where" question- can be usedfor describing what objects look like , or how they are configured with respectto each other (cf. Wilkins " and Senft 1994 ) . Notice that when English speakersexclaim, Why is there butter on " ' ? or " There s a crack in my cup!" they are not telling their listeners my scissors " where" the butter or the crack is but rather , making an observation about the condition of the cup or the scissors . Different conventions about the use of spatial languagefor describing what things look like also seemto lie behind the tendency of ' ' Spanish speakersto choose constructions with tener have in many contexts where " ' ; compare There s a ribbon around the English speakerswould use spatial language " " " Christmas candle with The Christmas candle has (tiene ) a ribbon .
10.2.2.2 What Should Be Located with Respect to What ? The difference between directing listeners to where something is versus telling them what something looks like probably also lies at the bottom of another intriguing difference between languages . Assuming a spatial characterization of the relationship between two entities , which one will be treated as the figure ( located object ) and which as the ground ( referent object )? As Talmy ( 1983) has pointed out , it is usual for speakers to treat the smaller , more mobile object as the figure and the larger , more stable object as the ground :

400
(4) a. The book is on the table. b. ?The table is under the book. (5) a. The bicycle is near the church. b. ?The church is near the bicycle.

Melissa Bowerman

This principle is likely to be universal when the purpose of languageis to guide the ' searchfor an listeners entity whose location is unknown to them. But when spatial is used for a more descriptive purpose, languages may follow different language For conventions. example, when one entity completely covers the surfaceof another, " " English consistently assignsthe role of figure to the coverer and the role of ground to that which is covered (cf. sentences6a and 7a) . Dutch , however, reversesthis 6b and 7b): assignment(sentences ' (6) a. There s paint allover my hands. b. Mijn handen zitten helemaalonderde verf. ' ' My hands sit completely under the paint . ' (7) a. There s ivy allover the tree. b. De boom zit helemaalonderde klimop . " ' The tree sits completely under the ivy . This difference between English and Dutch might be ascribable to the lack in Dutch of an equivalent to the English expression allover - but we can also ask whether the absenceof such an expressionmay not be due to a conventional assignment . of figure and ground that rendersit unnecessary 10.2.2.3 How Are Objects Conventionally Conceptualizedfor Purposesof Spatial ? Many cross linguistic differences in spatial organization are due, as Description in section 10 .2.1, to variation in the makeup of spatial semanticcategories discussed that is, in the meaning of spatial words. But even when morphemes have roughly similar meaningsin different languages , variations in encoding may arise becauseof . in the differences way objects are conventionally conceptualized systematic it . In section 10.1.2.3 was in and behind Consider, for examples , , front of pointed out that English-speaking children initially use these words only in the context of " featured" referent objects- objects that have inherent fronts and backs. But which ? Peopleand animals are clearly featured. Trees are often mentioned objects are these as examplesof objects that are not. But it turns out that this is a matter of convention . For speakersof English and familiar European languages , trees indeed do not have inherent fronts and backs. But for speakersof the African languageChamus, , or , if it does not lean, they do!- the front of a tree is the side toward which it leans the side on which it has its longest branches (Heine 1989 ; seealso Hill 1978for some

Learning How to Structure Spacefor Language

401

systematic cross linguistic differences in the assignment of front and back regions to nonfeatured objects) . Cienki ( 1989 ) has suggestedthat many differencesbetween English, Polish, and Russian in the application of prepositions meaning " in " and " on" to concrete situations are due to differencesnot in the meanings of the morphemes themselves , but in whether given referent objectsare conceptualizedas planes or containers. Children must learn, then, not only what the spatial morphemes of their language mean, but also how the objects in their environment should be construed for purposesof their " fit " to thesemeanings . 10.2.2.4 How Much Information Should a Spatial Description Convey ? From among all the details that could be encoded in characterizing a given situation spatially . Within a language , speakersmake a certain selection , the choice betweena less versusmore detailed characterization of a scene(e.g., " The vaseis on the cupboard" versus " the vase is on top of the cupboard" ) is influenced in part by pragmatic considerations like the potential for listener misunderstanding . But holding context constant, there are striking cross differences in conventions for how much linguistic and what kind of information to give in particular situations (seealso Berman and Slobin 1994 ; Slobin 1987 ). " " " " For example , for situations in which objectsare in or on objects in a canonical " " " " way (e.g., cup on table , cigarette in mouth ), speakersof many languages , such as Korean , typically usea very generallocative marker and let listenersinfer the exact nature of the relationship on the basis of their knowledge of the objects. English, in contrast, is relatively picky, often insisting on a distinction betweenin and on regardless of whether there is any potential for confusion. But English speakers are more lax when it comes to relationships that canonically involve encirclement as well as contact and support: although they can say around, this often seemsexcessive(" ring onJ?around finger," " put your seatbelt onJ?around you" ). For most Dutch speakers , in contrast, the encoding of encirclement wherever it obtains (with om ' around' ) is as routine as the distinction between in and on in English. This attentiveness to encirclement may in a sensebe " forced" by the lack in Dutch of an equivalent to the English all -purpose on: both op ' on l ' and aan ' on2' cover a narrower range of topological relationships, and neither one seemsquite appropriate for most casesof " encirclementwith contact and support." Another kind of information that is supplied much more frequently in some languages than in others is the motion that led up to a currently static spatial situation. In English and other Germanic languages , it is common to encode a static scene " ' " " ' without referenceto this event: for example , There s a fly in my cup and There s " a squirrel up in the tree! Although a static description of such scenes is also possible in Korean, speakers typically describe them instead with a verb that explicitly

402

Melissa Bowennan

" by the English sentences A fly has entered specifiesthe precedingevent, as suggested " " and " A the tree. squirrel has ascended my cup in the amount of infonnation typically differences also cross There are linguistic ) . Speakersof provided in descriptions of motion events (Bennan and Slobin 1994 and Gennan, tend to languageswith rich repertoires of spatial particles, like English " characterize motion trajectories in considerable detail (e.g., The boy and dog fell " off the cliff down into the water ), while speakersof languagesthat expressinfonna infonnation less such as in verb the tion about trajectory primarily , give , Spanish " " " " overall about trajectory (e.g., fell from the cliff j fell to the water ), and often simply imply the kind of trajectory that must have been followed by providing static : there is a cliff above, there is descriptions of the locations of landmarks (in this case . fall and water below, and the boy ) dog To summarize , I have argued that different languagesstructure spacein different ting semantic ways. Most basically, they partition spaceinto disparate and often crosscut situations whether two for criteria different spatial establishing categoriesby using " or " different" in kind . In addition " same as the , they should be considered differ in which classesof situations can be characterized readily in spatial tenDS at all , in how the roles of figure and ground are assignedin certain contexts, in how objects are conventionally conceptualizedfor purposesof spatial description, and in how much and what kind of infonnation spatial descriptions routinely convey. These differencesmean that there is a big discrepancybetweenwhat children know about spaceon a nonlinguistic basis and what they need to know in order to talk about it -appropriate way. in a language Accounts of spatial semantic development over the last twenty-five years have . Among students of languageacquisition linguistic differenceslike these neglectedcross " " there has been a strong tendency to equate semantic structure directly with " " conceptual structure - to view the meaningsof words and other morphemesto a large extent as a direct printout of the units of human thought . But although semantic structure is certainly dependenton human conceptualand perceptual abilities, it is by - and often of larger constructions no meansidentical: the meaningsof morphemes )- representa highly structured and conventionalized layer of organization (Goldberg 1995 in different languages(seeBierwisch 1981 ; Lakoff different ; Bowennan 1985 , In to . Pinker 1989 in 1987 fully appreciate ; ) failing ; Levinson, press ; Langacker 1987 " " " the distinction between" conceptual and semantic , developmentalistshave overestimated ' the part played in spatial semanticdevelopmentby children s nonlinguistic , and so underestimatedthe magnitude of what children must learn. In consequence concepts , we as yet have little understanding of how nonlinguistic spatial understanding ' and linguistic input interact in children s construction of the spatial system . of their native language

Learning How to Structure Spacefor Language

403

10 .3 Studying Cross SpatialSemantic Categorization linguistic ally


-specificspatial semanticcategories How early in life do children arrive at language ? If the hypothesis is correct that the structure of spatial semantic concepts is provided - at least initially - by nonlinguistic spatial cognition , we would expect language specificity to be precededby a period of cross linguistic uniformity (or of individual differencesthat are no greater betweenthan within languages ) . Hypothesizing along these lines for spatial and other meanings encoded by grammatical morphemes , " Slobin ( 1985 , 1174 ) proposed that children discover principles of grammatical - categoriesthat are not yet tuned to the marking according to their own categories " distinctions that are grammaticized in the parental language ; only later are they led " by the languagespecific usesof particular markers to conceive of grammaticizable notions in conformity with the speechcommunity ." This scenario predicts extensive errors at first in the use of spatial morphemes , possibly suggestiveof the guiding influence of " child -style" spatial conceptsthat are similar acrosslanguages . Another possibility is that although children may perceive many properties of spatial situations, they do not start out strongly biased in favor of certain grouping principles over others. In this casethey might be receptive from a very early age to semantic categoriesintroduced by the linguistic input and quickly home in on the neededprinciples with relatively few errors. Of course , there are many possiblegradations between the two extreme scenarios sketched here- that is, early reliance on nonlinguistic conceptsversusearly induction of categoriesstrictly on the basisof the linguistic input . And somedomains may be more susceptibleto linguistic structuring than others. For example , Gentner ( 1982 ) has argued that the mapping betweenverbs and other relational words onto events is less transparent- more imposed by language - than the mapping betweenconcreteobject nouns and their referents(seealso note 21 on differential transparencyin another domain) . The hypothesis that language can influence the formation of children' s semantic categoriesfrom the start of lexical development played an important role in earlier views of how children learn the meanings of words. For example , Roger Brown likened the process of learning word meanings to a game (" The Original Word about how to classify referents on Game" ) in which the child player makes guesses the basisof the distribution of forms in adult speech that " a speech , and he suggested invariance [e.g., hearing the sameword repeatedlyin different contexts] is a signal to form some hypothesis about the corresponding invariance of referent" ( 1958 , 228) . But this approach to learning word meaningshas been out of fashion for a number of years. One reason for its unpopularity is that it clasheswith the contemporary stressin " developmentaltheorizing on the needfor constraintson word learning: an observer

404

Melissa

who notices everythingcan learn nothing, for there is no end of categoriesknown and constructable to describe a situation " (Gleitman 1990 , 12 ; see also Keil 1990 and Markman 1989 ) . Another reason is that the appeal to guidance by language in the construction of semantic categoriesis associatedwith the perennially controversial Whorfian hypothesis ( Whorf 1956 )- the proposal that the way human beings view . reality is molded by the semantic and grammatical organization of their language The Whorfian position has seemed implausible to many, especiallyas infant research shows ever more clearly the richness of the mental lives of babies (although see Levinson and Brown 1994 ; Lucy 1992 ; and Gumperz and Levinson 1996 for new on the Whorfian ) . But in the widespread rejection of the perspectives hypothesis Whorfian hypothesis , the baby has been thrown out with the bathwater. Regardless of whether the semanticcategoriesof our languageplaya role in fundamental cognitive activities like perceiving, problem solving, and remembering , we must still learn them in order to speak our native language fluently . But how learners home in on thesecategoriesis a topic that has beenlittle explored.8 In trying to evaluate the relative strength of nonlinguistic cognitive organization and the linguistic input in guiding children' s early semantic structuring of space ,a ' -age useful research strategy is to compare same children learning languageswith . Because we are interestedin how early children strikingly different spatial categories -specific ways of structuring space can arrive at language , it is sensible to focus on meanings that are known in principle to be accessibleto young children (thus, ' ' ' ' in ' and ' on' ' type meanings are preferable to projective in front of / behind -type ) . With this in mind , I have beenexploring, in projects together with various meanings ), how colleagues(Soonia Choi , Dedre Gentner, Lourdes de Leon, and Eric Pederson children, and languages , handle topological notions of contact, separation, inclusion, and encirclement ; functional and causal notions like support, containment, attachment and adhesion , ; and notions to do with vertical motion and orientation (up and down) . 10.3.1 Spatial Encodingin the Spontaneous of Learnersof Korean and Speech English In one study, Soonia Choi and I compared how children talk about spontaneousand causedmotion in English and Korean (Choi and Bowerman 1991 ; Bowerman 1994 ). These two languages differ typo logically in their expression of directed motion . " " . These , 1991 ) calls a satellite-framed language English is what Talmy ( 1985 , languages which include most Indo European languagesand also, for example Chinese and Finnish - characteristically express path notions (movement into , out of , up, down, on, off , etc.) in a constituent that is a " satellite" to the main verb, such as a prefix or (as in the caseof English) a particle/preposition. Korean, in con-

Learning How to Structure Spacefor Language

4OS

- which include, for example trast, is a " verb-framed" language ; these languages , Hebrew, Turkish , and Spanish expresspath in the verb itself (Korean lacks a class of spatial particles or prepositions entirely) . For present purposes , the most important differencebetweenEnglish and Korean is that many of their semanticcategoriesof path are different. In general , the prepositions and particles of English identify paths that are highly abstract and schematic , whereasmost of the path verbs of Korean are more specific . For example , in English, a motion along a particular path is encodedin the sameway regardless of whether the motion is spontaneousor caused(cf. " Go in the closet" versus" Put it in the closet" ; " Get out of the bathtub " versus" Take it out of the bathtub " ) . In Korean, in contrast, spontaneous versus caused motions along a particular path are typically encoded with entirely different verb roots (cf. tule ' enter' versus nehta ' put loosely in (or around)' ; na ' exit' versus kkenayta ' take out (or take from loosely around)' .9 Further , English path categoriesare relatively indifferent to variation in the shape and identity of the figure and ground objects, whereasKorean path categoriesare more sensitive to this, with the result that they subdivide and crosscut the English path categoriesin complex ways; this is illustrated in table 10.1 (seeChoi and Bowerman 1991for more detail) . The overall tendencyfor path categoriesto be larger and more schematicin English than in Korean is no doubt related to the systematicdifference -classmorphemes(prepositions and particles) in how they are expressed : with closed -class morphemes (verbs) in Korean (see also Landau and in English and open Jackendoff 1993and Talmy 1983 ). If the meanings that children initially associate with spatial morphemes come , these differencesin the way directly from their nonlinguistic conceptions of space are structured in versus Korean should have no effect on spatial meanings English ' learners use of words children should extend the words on the basisof early spatial their own spatial concepts . To seewhether , not the categoriesof the input language this is so, Choi and I compared spontaneousspeechsamplescollected longitudinally from children learning English and Korean. 10 We found that both sets of children first produced spatial morphemes at about fourteen to sixteen months (particles like up, down, and in for the English speakers ; verbs like kkita ' fit tightly ' and its opposite ppayta ' unfit ' for the Korean speakers ; cf. table 10.1), and began to use them productively (i.e., for events involving novel configurations of objects ) by sixteen to twenty months. They also talked about similar events for , , manipulations such as putting on and taking off clothing; example opening and closing containers, putting things in and taking them out , and attaching ; position and posture changessuch as climbing up and down things like Lego pieces from furniture and laps; and being picked up and put down. The spatial concernsof children learning quite different languagesare, it seems , quite similar at this age ,

406

Melissa Bowennal1

English:
in on

up Korean: nehta kkita

, cherries in basket) (e.g., put ball in box, earplug in ear, flower in vase / (e.g., put box on table, sticker/magnet on refrigerator, hat/ coat/ shoes bracelet on) (e.g., put a cup up high, pick a child up, sit up, stand up) ' put loosely in (or around) (e.g., ball in box, loose ring on pole) ' ' fit tightly ; put tightly inion /together/ around , tight ring (e.g., earplug in ear, top on pen, two Lego piecestogether on pole) ' ' put elongated object to base , hairpin in hair , book upright on shelf) (e.g., flower in vase ' ' put multiple object in container ) (e.g., cherries in basket ' ' surface on horizontal put (e.g., box on table) ' stick as if , juxtapose surfacesthat are flat , or can be conceptua1ized flat ' (e.g., sticker/magnet on refrigerator, two Lego piecestogether ) ' ' ) put clothing on head (e.g., hat, scarf, mask, glasses ' ' put clothing on trunk (e.g., shirt , coat, pants) ' ' , shoes ) put clothing on feet (e.g., socks ' e. . belt ' or wrist on at waist , bracelet) , diaper, dagger ( g, put clothing / ' e. . lift a ' causeto ascend ( g, cup up) ' ' pick uP/ hold in arms (e.g., pick a child up) ' ' assumea ) sitting posture (e.g., sit up, sit down ' ' assumea standing posture (e.g., stand up) '

kkocta lam Ia nohta pwuchita ssuta ipta sinta charD olliia anta ancta ( ile ) seta

Learning How to Structure Spacefor Language

407

revolving primarily around topological notions and motion up and down (seealso ' section 10.1, and Sinha et ale 1994 ) . But were the children s spatial semanticcategories similar , as inferred from the range of referent events to which they extended their words? They were not. By twenty months of age , the path semanticcategoriesof the two setsof children were quite different from each other and clearly aligned with the . For example: categoriesof the input language I . The English learners used their spatial particles indiscriminately for both spontaneous and causedmotion into and out of containment, up and down, and so on. In contrast, the Korean children used strictly different verbs (intransitive vs. transitive) for spontaneousand causedmotion along a path . For instance , English learnerssaid in both when they climbed into the bathtub and put magnetic letters into a small box; in comparable situations the Korean learners used the verbs rule ' enter' versus nehta ' put loosely in (or around)' . 2. The English learnersusedup and downfor a wide range of eventsinvolving vertical motion , including climbing on and off furniture , posture changes(sitting and standing up, sitting and lying down), raising and lowering things, and wanting to be picked or up put down. Recall that , as reviewedin section 10.1.2.2, the rapid generalization of up and downhas beeninterpreted as evidencethat thesewords are coupled to nonlinguistic . But the Korean children used no words for a comparable spatial concepts as is appropriate in their language of motion or down: , they used different range up or down words for posture changes , climbing up , being picked up and put down, and so forth . 3. The English learners distinguished systematically between putting things into containers of all sorts (in) and putting them onto surfaces(on), but were indifferent to whether the figure fit the container tightly or loosely, or whether it was set loosely on a horizontal surfaceor attached tightly to a surfacein any orientation , or - in the caseof clothing items- what part of the body it went onto . The Korean learners , in ' ' contrast, distinguished betweentight and loose containment (kkita fit tightly versus nehta ' put loosely in (or around)' ), betweenattaching things to a surface(kkita again) ' and setting things on a surface (nohta ' put on horizontal surface), and between ), trunk (ipta), and feet (sinta) . Someexamplesof putting clothing on the head (ssuta thesedifferencesare given in table 10.2. -specific features of Although the children had clearly discovered many language , their command of the adult path categories spatial encoding in their input language was by no meansperfect- there were also errors suggestingdifficulties in identifying the boundaries of the adult categories , such as the use of open for unbending a ' ' doll (cf. last example in ( I ) of introduction ), or the use of kkita fit tightly for flat surface attachments involving stickers and magnets (e.g., entry 6 in table 10.2; this

408

Melissa Bowennan

.2 . Table10 of in the Spontaneous ContactRelations and Surface TheTreatment of Containment Speech


Children Learning English and Korean Age (in months) Utterance
Situation

Relation Tight containment ( Korean kkita ) Loose containment ) (Korean nehta Tight surfacecontact ( Korean kkita )

English 1 . 18 2. 3.
19 17

In ' gain. In. On. Horsie on. Can' t wowwow on.

Trying to shove toy chair through . narrow door of doll house When mother dips her foot into the washtub of water. Looking for rein of rocking horse; it has come off and shewants to attach it back on to the edgeof the horse' s mouth. Frustrated trying to put toy dog on a moving phonograph record.

Loose surface contact ( Korean nohta )

Korean Kkila .

6. 7. 8.

27 20 28

Kkita . Nehta. Nohta.

Puttingpegdoll into perfectly -seaton smallhorse fitting niche . hasbrought that investigator fish to Attachinga magnetic beakof duck. magnetic into a pan. blocks Putting Puttingoneblockon top of
another.

Tight ~ containment

in) (English

Tight surfacecontact (English on) Loose containment (English in) Loose surfacecontact (English on)

The Korean examples show only citation form of the verb , not whole utterances .

' .1) . These errors are should be pwuchita ' stick, juxtapose flat surfaces; cf. table 10 ' categories of the learners that the because language specificity they suggest important cannot be dismissedon grounds that the children perhapswere simply mimicking what they had heard people say in particular situations, and had no real grasp of . (Appropriate usagefor novel situations, asillus the underlying semantic concepts trated by most of the examplesin table 10.2, also arguesagainst this interpretation .) ' We will come back to errors later, because they provide invaluable clues to children s relative sensitivity to different kinds of spatial semanticdistinctions. 10.3.2 Spatial Encodingin Elicited Descriptionsof Actions in Children Learning English, Korean, and Dutch The examination of spontaneousspeechcan give a good overview of the early stages of spatial semanticdevelopment , and this approach has the advantagethat , because

Learning How to Structure Spacefor Language

409

the utterancesare freely offered, they reflect how children are conceptualizing situations for their own purposes . But a disadvantageis that the specificspatial situations that children happen to talk about vary, so comparing the distribution of forms requires matching situations that are not identical (as is done in table 10.2) . To get more control over what subjectstalked about, Choi and I decided to conduct a production study in which we elicited descriptions of a standardized set of spatial actions from all subjects ( Bowerman and Choi 1994 ). This time we focused exclusively on causedmotion involving spatial manipulations of objects. To English and Korean , we added Dutch . Recall that an interesting way in which Dutch differs from English is its breakdown of spatial relations encompassedby English on into ' ' " " ' ' " two subclass es , op on l (e.g., cup op table ) and aan on2 (e.g., handle aan cupboard " door ); these differences are relevant to motion as well as to static spatial configuration . - seventy -nine in all - were selected on grounds that they The actions we used are grouped and distinguished in interestingly different ways in the three languages . " " They were both familiar and novel, and covered a broad range of joining and " " separating situations such as donning and doffing clothing of different kinds (carried out with a doll ), manipulations with containers and surfaces(e.g., putting a toy boat into a baby bathtub and taking it out, laying a doll on a towel after her bath, taking a dirty pillow caseoff a pillow and putting a clean one on), openingand closing -fitting , a cardboard box with flaps), putting tight - and loose things (e.g., a suitcase rings on a pole and taking them off, buttoning and unbuttoning , hanging and " " unhanging (towel on/offhook ), hooking (train cars together / apart), sticking ( Bandaid on hand, suction hook on/off wall ), and otherwise attaching and detaching things , Popbeads , Bristle blocks). For these last(e.g., magnetic train cars, Lego pieces mentioned actions, we varied whether the objects were moved laterally or vertically, and whether the motions were symmetrical (e.g., one Lego piece in each hand, both hands moving together) or asymmetrical (e.g., one hand joins a Lego piece to a stack of two Legos pieces held in the other hand) . (English and Dutch , but not Korean , are sensitive to these properties- compare, for example , put on with put and take with take . , together off apart ) For each languagewe had 40 subjects : 10 adults, and 30 children, 10 each in the , and 3;0 3;5 years. Subjects were tested individually . age ranges 2;0 2;5, 2;6 2; 11 We elicited spatial descriptions by showing the objects involved in each action and indicating what kind of spatial action should be performed with them, but not quite " " ll This performing it , and saying things like What should I do? Tell me what to dO. procedure worked quite well: even in the youngest age group, 87% of the children , although not necessarilythe same one the adults gave a relevant verbal response from the children learning English and Dutch were particles, gave. Typical responses

410

Melissa Bowennan

either alone(e.g., in, on) or with verbs(e.g., put it in); from the children learning form ofkkita ' fit tightly' ). Koreantheywereverbs(e.g., kkie, imperative
10.3.2.1 Action De script io. . as Similarity Data The data collected can be seen as analogous to the data obtained in a sorting study. But instead of giving subjects a set of cards with , say, pictures of stimuli , and asking them to sort these into " " piles of stimuli that go together, we take each word produced by a subject as defining a category (analogous to a pile), and look to seewhich actions the subject applied the word to (i.e., sorted into that pile) . Actions a speaker refers to with the sameexpressionare consideredmore alike for that speakerthan actions referred . 12Seenin this way, the data can be analyzed with any to with different expressions technique suitable for similarity data, such as multidimensional scaling or cluster 13 analysis. In one analysis , the data from all the subjectswere subjectedto a multidimensional scaling analysis that allowed us to plot the actions in two-dimensional spaceon the basis of how similar each action was to each other action (as determined by how often speakersacross all three languagescharacterized both actions with the same " " ) . This was done separately for the set of joining actions and the set expression " " of separating actions, after earlier analyses had showed that , with rare (child ) . The , these were distinguished by subjects of all ages and languages exceptions two resulting plots - somewhat modified by hand to spread out actions that were bunched very tightly together (because they were very often describedwith the same we can display the categorization system serve as on which then ) grids expression of any individual , or the dominant categorization of a group of individuals , by drawing in " circles" (i .e., Venn diagrams) that encompassall the actions that were described in the sameway. To see how this works, consider figures 10.3 and 10.4. Figures 10.3a and 10.3b show the dominant classification of the " joining " actions by the English-speaking adults and youngest group of English-speaking children (2;0- 2;5 years ); Figures . The number 10.4a and 10.4b give the same information for the Korean subjects of subjects(out of 10 ) who produced a given responseis indicated on the grid near the label for the action.14 A quick overview of similarities and differencesin how different groups of subjectsclassifiedthe actions can be obtained by an eyeball comparison of the relevant figures: . . . . Figures 10.3a and 10.4a: adult speakersof English versusKorean; -age child speakersof English versusKorean; Figures 10.3b and 10.4b: same Figures 10.3a and 10.3b: adult versuschild speakersof English; Figures 10.4a and 10.4b: adult versuschild speakersof Korean.

Learning How to Structure Spacefor Language

Su

~- -

y~ ~ ~

\~ ~~ b

Q P
09 ~

" '~ ; I ~
~

. \ ~'

~ \ " b \ ~ ~ \
'

~'

ttP ~ ~ ~ \
. ~ .~ lP

9 ~ ~

\ t " , \~ . " \ ~ ~ :
~ ~

~ bO

\" ~

Learning How to Structure Spacefor Language

. ,: , AI Vrn3N ,, , , , ,, , ,,,
!
q U8d 01

~ ~J y ~-

"
'

xoq SJ8 Ot Suq so Ot I ,, ,


SU09

" - tf

N . ~ < . ' 5 1 , " ; 2 ~ E ) J . , ' i c I : o , " . \ 'E < S e O g ~ H 0 . ~ # : . B 0 , 9 i ) 8 ' 1 ~ I ' I ~ S i 0 g OO 1 It = 1 j . O s J Q , ' , ; . 8 \ 5 D \ " s u Q : ~ = oS I D c1 " , . s < t t < 9 . c 8 \ O 2 ~ 9 ' 18 8 10 \ ~ ! S 1 ~ es 0 \ g O il tj Q " 88 1J 8 " 1 38 ~ . c t J . l S Q y 0 \ g o ~ , . s u8 ! R ' ) . c ll j 8 ~ ! 8 f; '~ ! , ' " . . S f D s Qt 2fJ . ~ ~ ! ot ! ~ to \ ! 1 J e t y = 8 , . ~ . : s S t ~ I t U 10 '2 . s Ct
~ -~
Melissa Bo \Vennan

~ J3

Learning How to Structure Spacefor Language

415

These comparisons reveal both similarities and differencesacross subject groups. For example, in addition to agreeing that joining and separating actions should be described differently, subjects of all ages and languagesagree on categorizing the " " closing actions together (to far left on grid), and also the " putting into loose container" actions (lower right) . But they disagreequite dramatically on the classi fication of actions of " putting into a tight container," actions of encirclement , putting on clothing , and so forth . In general outline , the children' s classification patterns are similar to those of the adult speakersof their language , but they are simpler. The children lack some words the adults use(e.g., togetherin English; pwuchita ' stick or juxtapose surfacesthat are flat , or can be conceptualized as if flat ,' in Korean), and they overextend certain words relative to the adult pattern- for example , many English learners overextend on to " together" situations; and many Korean children overextend kkita ' fit ' tightly to hooking train cars together and hanging a towel on a hook , and nehta ' ' put loosely in (or around) to putting a pillow caseon a pillow . 10.3.2.2 Interpreting Children' s Categorization Patterns Comparing across the three languages , theseelicited production data suggestthat the way children initially classify spacefor languageis the outcome of a complex interaction betweentheir own nonlinguistic recognition of similarities and differencesamong spatial situations, on the one hand, and the way spaceis classifiedby adult speakersof their language , on the other. Overall, the influence of the input language is quite strong: statistical analysis shows that in all three languages , the youngest age group of children classified the spatial actions more similarly to adult speakersof their own language -age children learning other than to same languagesS But obedience to the adult systemwas by no meansperfect. Patterns of conformity with and deviation from the adult target systemappear to be influenced by a mix of linguistic and nonlinguistic factors. Let us consider two examples . I . When children of a certain age are in principle capable of making a particular semanticdistinction (as inferred from the observation that children in somelanguage do so), the speedwith which they begin to make it (if it is neededfor their language ) is strongly influenced by the clarity and consistencywith which adult speakersmark it . For example , even the youngest age group of English speakers , like the adults, made an systematicsplit between " removal from containment" (out) and " removal " from contact with an external surface (off ); this is illustrated in figure IO.5a with a 16 subset of the relevant actions. Like English speakers , adult Dutch speakersalso make a distinction between " removal from containment" (u;t ' out ' ) and " removal " ' ' from contact with an external surface (af off ) . But the youngest group of Dutch

416

Melissa Bowerman

children did not observe it - as shown in figure 10.5b, they vastly overextendeduit ' out ' to actions for which adults use ' off ' like , taking a ring off a pole, a pillow case af off a pillow , and a rubber band off a box. Why do the two setsof children differ in this way? Comparison of the adult systems is revealing. In English, the distribution of out and off correlatesclosely with removal from a concavity versusremoval from a flat or convex surface(including body parts) . ' ' ' ' In Dutch , the distribution of uit out and af off is basedon the sameprinciple , but : whereasEnglish usesoff for the removal of with one important class of exceptions ' " ' shoes like coats , Dutch usesuit out ( take out , , and socks , pants envelopingclothing " / coat ; cf. figure 10.5c) . When adult Dutch speakersare asked why they your shoes " " take out your shoes(coat, etc.), they often seemto discover the anomaly for say - when you take your shoe uit [' out '] , it ' s really your foot the first time: " It ' s strange " ' that comes out of the shoe , isn t it , not the shoe that comesout of your foot ! This that adults store this clothing use of uit separatelyfrom its normal reaction suggests use(i.e., as a separatepolyseme ufficiently salient ) . But this atypical useseemsto be ~ to young children to obscure the distinction otherwise routinely made in Dutch between removal from surfacesand removal from containers. it goessquarely against a common claim about This example is intriguing because at first learn and use words only in very specific : that children early word learning contexts~According to this hypothesis , Dutch children should learn the useof uit for lexical item. If so, they should proceed as an off independent essentially clothing taking on the same scheduleas learners of English to discover the semantic contrast ' ' ' ' between more canonical uses of uit out and af off . But this does not happen: ' ' Dutch children appear to try to discover a coherent meaning for uit out that can . The only meaning consistent encompassboth clothing - and container-oriented uses with both uses , in that it is indifferent to the distinction between removal from a " " surface and removal from containment, is the notion of removal itself. Once children ' ' have linked this notion to uit out it licensesthem to use the word indiscriminately across the ' out ' / ' off ' boundary, which is exactly what they do, as shown in t7 figure 10.5b. ' 2 . Children s errors in using spatial words have often been interpreted as a direct , in interpreting the somewhat pipeline to their nonlinguistic spatial cognition; for instance ' different patterns of extension of the words openand off in my two daughters that the children had arrived at different ways of categorizing , I once suggested speech kinds on the basis of their own dealings with the physical of various ~ parations do often seemto be conditioned by factors . Overextensions world (Bowerman 1980 ) for which it is difficult to think of an intralinguistic explanation: for example , across ' extend words for all three languagesin Choi s and my study, children tended to ove~

Learning How to Structure Spacefor Language

topoffpen off pole ring off case pillow pillow box robberban .dress etc off off underpan undenhirt shoes off socks
417

''pen oft tq > out case cassette of oft case cassette out of ring pole out of case oft ' out of Legos bag pillow pillow bag Legos D out of badtwb robber band oft 'box do doll out of badttub cars out of box cars out of box etc . etc . dress oft ' oft underpants undershirt oft '' ' shoes oft socksoft ' 'out ' U1T

2;0 - 2;5 a. ChildrenlearningEnglish , age

b. ChildrenlearningDutch , age2;0 - 2;5

off top pen off ring pole cassette out of case case off 'pillow pillow of out Legos bag x d911 robber band off tx > out of bad1tub x cars out of tx > etc . . etc dress off ' underpantsoff undershirt off shoes off socks off
C. Dutch adults

Fipre 10.5 Classification of actions as ' off ' versus' out ' in English and Dutch .

418

Melissa Bowerman

separationmore broadly than words for joining ; that is, they differentiated lessamong actions of separation, relative to the adult pattern, than among actions of joining (and this is also true for children learning Tzotzil Mayan ( Bowerman, de Leon, and Choi 1995 ) . But a careful look across languagessuggeststhat linguistic factors also : in particular , the category structure of the an important role in overextensions play and the specificpatterning of the overextended words both which influences get input . extensions If overextensionsof spatial morphemeswere driven purely by ways children categorize spatial events nonlinguistically , we would expect similar overextensionsin . And we do in fact find this to some extent: for example different languages , similar for children have been and its translation of open overextensions reported equivalents ) . In learning English, French, and German (seeClark 1993 for review and sources ' Choi s and my production study, open(also spelledopenin Dutch ) was overextended to actions for which adults never usedit about 9 times by English learnersand about 21 times by Dutch learners(e.g., unbuttoning a button , taking a shoe off , separating two Lego pieces ) . But Korean children hardly , and taking a piece out of a puzzle make this error - it does not occur at all in the spontaneousspeechdata we have examined , and it occurs only once in the production study (one child used yelda ' for ' unhooking two train cars) . open ' ' izing open words? A Why is there this difference in the likelihood of overgeneral plausible explanation is that it is due to differencesin the size and diversity of the ' ' open categoriesof English and Dutch (and French and German) on the one hand, ' ' and Korean on the other. In Korean , yelda open applies to doors, boxes , , suitcases that are actions of other and bags , for example, but it cannot be used for a number also called open in English and Dutch , such as opening the mouth , a clamshell, ' doors that slide apart (ppel/ita ' separate two parts symmetrically ), the eyes (ttuta ' ' ' rise' an ), envelope (ttutta tear away from a base), and a book , a hand, or a fan ' ' ' ' (phyelchita spread out a flat thing ) . The breadth of the open category in English and Dutch - that is, the physical diversity of the objects that can felicitously be " " opened - seems to invite children to construct a very abstract meaning; put differently , the diversity discourageschildren from discriminating among candidate ' ' opening eventson the basisof object properties that are in fact relevant to membership in the " open" category for adults. Conversely , the greater coherence in the ' ' physical properties of the objects to which Korean yelda open can be applied eventsthat are along with the coherenceof eachof the other categoriesencompassing 's " " children Korean facilitate Dutch in and also called open recognition may English of the limits on the semanticrangesof the words. ' ' If Korean children do not overextendyelda open , do they have another word that ? They do. In our production study, they they overextendin the domain of separation

How to Structure Spacefor Ian ~ua~e Learning

419

' ' overwhelmingly usedppayta unfit for virtually all the actions of separation- even ' ' including the actions for which adults usually used yelda open , such as opening a ' ' suitcaseand a box! Like open in English, the category of ppayta unfit is big and : out of the 36 " separation" actions in our study, 24 were diverse in adult speech labeled ppayta by at least one of the 10 Korean adults. (The word was used most , Lego pieces , and Bristle blocks, and heavily for events like separating Popbeads the off a but it was also used occasionally a out of a and top pen, taking piece puzzle case for (e.g.) opening a cassette , taking Legos out a bag, taking off a hat, and taking a towel off a hook.) ' ' Do English, Dutch , and Korean children in fact use open , open open , and ppayta ' unfit ' for the same ? If so, this would suggestthe power of an underlying rangeof events child basic, languageindependent notion . But the situations to which children extend open and ppayta ' unfit ' differ , and the differencesare related to the different . meaningsof the words- and their different rangesof application - in adult speech Korean children' s ppayta ' unfit ' category seemsto have its center- as in adult " ' - in the notion of " separating fitted or ' meshing objects with a bit of force speech (e.g., pulling Popbeadsand Lego piecesapart, taking the top off a pen- 9 out of the 10 children usedppayta for these actions) . It is extended from this center to taking , relative to patterns in the adult data, to things out of containers, and overextended " " opening containers, unsticking adhering and magnetized objects, and taking off ' clothing . In contrast, English-speakingchildren s opencategory is centeredon acts of separation as a means of making somethingaccessible(e.g., opening a box to find something inside; opening a door to go into another room), and it is extendedfrom this center only occasionally to pulling apart Popbeadsand Legos and taking off clothing (both much more often called off in the elicited production study), and to taking things out of containers (much more often called out) . English-speaking without any children also useopenfor actions in which something is made accessible water faucets and electric on TVs radios at all such as , , , , light turning separation ' ' switches(Bowerman 1978 , 1980 ) . Korean children do not overextendppayta unfit to its use.in adult speechis concentratedon acts of eventsof this kind , probably because not on se and , separation as a means of making something physical separation per . accessible In sum, children learning these different languages show a shared tendency , to underdifferentiate referent factors conditioned , by nonlinguistic cognitive probably eventsin the domain of separation- that is, they overextend words in violation " of distinctions that their target language honors. But which words they " select to overextend , and the semanticcategoriesdefined by the range of eventsacrosswhich . them, are closely related to the semanticstructure of the input la~guage extend they

420

Melissa

.4 How Do Children 10 Co. .troct theSpatialSemantic of TheirLanguage ? System


We have seenthat languagelearnersare influenced by the semanticcategorization of spacein their input languagefrom a remarkably young age. This does not mean, of course, that they simply passively register the patterns displayed in the input - they do make errors, and thesesuggestthat learners find some distinctions and grouping principles employed by the input languageeither difficult or unclear (or both ) . There is, then, an intricate interaction between nonlinguistic and linguistic factors in the . In this final section, let us speculateabout processof spatial semantic development how this interaction takes place. 10.4.1 Is the HypothesisThat Children Map Spatial Morphemesonto Prelinguistically CompiledSpatial ConceptsStill Viable? The evidence for early language specificity in semantic categorization presentedin section ] 0.3 might seemto argue strongly against the hypothesis that children start out by mapping spatial words onto prepackaged notions of space . But Mandler that the two can, after all , be reconciled. ( ]992 and chapter 9, this volume) suggests Inspired by the work of cognitively minded linguists such as Langacker ( ] 987), Lakoff ( ] 987), and Talmy ( ] 983, ] 985), Mandler hypothesizesthat an important step in the prelinguistic developmentof infants is the " redescription" of perceptual information "into " image-schemas representationsthat abstract away from perceptual details to present information in a more schematicform . Preverbal image schemas would playa number of roles in infant mental life , but of special relevancefor us is Mandler ' s ( ] 992, 598) suggestionthat they " would seemto be particularly useful in the acquisition of various relational categoriesin language ." In particular, Mandler ' ' ' ' -schemasof containment that words meaning in and on are mapped to the image suggests (and the related notions of going in and going out) and support: ' (8) Containment: 0 Going in: <5 Going out: e . (9) Support: In considering evidence that languages partition spatial situations in different ways, as discussedin Bowerman ( ] 989) and Choi and Bowerman ( ] 99] ), Mandler that " however the cuts are made, they will be interpreted [ by the ( ] 992, 599) suggests learner] within the framework of the underlying meaningsrepresentedby nonverbal -schemas ." This means that children " do not have to consider countless image variations in meaning suggestedby the infinite variety of perceptual displays with which they are confronted; meaningful partitions have already taken place" (p. 599) . Reliance on the preorganization provided by the nonverbal image-schemas for containment and support will make somedistinctions harder to learn than others; for

Learning How to Structure Spacefor Language

421

, Mandler suggeststhat children acquiring Dutch will have to learn how to example break down the support schema into two subtypes of support (op ' onl ' and aan ' ' on2 ; cf. section 10.2.1), and this might well take sometime (which is in fact true; see Bowerman 1993 ) . On the other hand, Mandler predicts no difficulty forSpanish ' ' speaking children in learning en in , on (this seemsalso to be true) becausethis involves only collapsing the distinction betweencontainment and support . But what about the ' tight fit ' category of the Korean verb kkita , which crosscuts the categories of both in and on in English, and, as Choi and Bowerman ( 1991 ) showed(cf. section 10.3.1), is acquired very early? Mandler ( 1992 that , 599 ) suggests the early mapping of kkita onto the ' tight fit ' meaning " is only a puzzleif one assumes that in and on are the only kinds of spatial analysesof containment and support that have been carried out ." But ' tight fit ' may well be an additional meaning that is , and thus is available for mapping to a word . Mandler prelinguistically analyzed acknowledges that we do not yet have independent evidence for this concept in " prelinguistic infants, as we do for containment and support, and adds that until such researchis carried out it will not be possible to determine whether a given language merely tells the child how to categorize[i.e., subdivide or lump] a set of meaningsthe child has already analyzed or whether the languagetells the child it is time to carry " out new perceptual analyses (pp. 599- 600) . ' Mandler s hypothesisis by no meansimplausible, but it comesat a price. Suppose we discover that , from a very young age, toddlers learning a newly researchedlanguage , L , extend a word across a range of referents that breaks down or crosscuts the spatial semantic categories we already know children are sensitive to , like the -schemas of containment, support, and tight categoriesdefined by the putative image ' fit . This means , by the logic of Mandler s argument, that there is yet another universal preverbal image-schemaout there that we were not aware of before, and we must assumethat all children everywherehave it , regardlessof whether they will ever need it for the languagethey are learning. This price may be acceptable as long as the putative preverbal image schemas uncoveredby future researchare not too numerous, and do not overlap eachother in complex and subtle ways. But this seemsdoubtful , even on the basis of the limited data that is currently available. For example , the categories picked out by open ' ' and ppayta unfit in the early speechof children learning English versus Korean . This might suggest that both words are mapped to the same overlap extensively schema , but , as arguedearlier, the overall range of the two categories preverbal image

422

Melissa Bowerman

extension patterns such as those just discussedmay representdevelopmentsbeyond -schemas are doing little this point . This is possible. But in this casethe spatial image of the work that has often motivated the postulation that children map words - namely, to provide a principled basis on to prelinguistically establishedconcepts which children can extend their morphemesbeyond the situations in which they have -schemasserve as the starting frequently heard them. Regardlessof whether image ' we cannot rely on them to account for productivity in children s points, then, it seems . For this, we will have to appeal to a processof learning usesof spatial morphemes build in which children spatial semantic categoriesin responseto the distribution of spatial morphemesacrosscontexts in the languagethey hear. - specificCo_ traints 10.4.2 SemanticPrimitives and Domain If semantic categoriesare constructed, they must be constructed out of something, and an important question is what this something is. Here we come squarely up against one of the oldest and most difficult problems for theorists interested in the structure of mind: identifying the ultimate stuff of which meaning is made. , a time-honored approach to this problem has been Among students of language to invoke a set of semantic primitives - privileged meaning components that are available to speakersof all languages , but that can be combined in different ways to . 19 In searching for the ultimate elements from make up different word meanings -class spatial words such as the set of English prepositions which the meaningsof closed of what have been struck by the relative sparseness are composed , researchers can be important . Among the things that can playa role are notions like verticality , horizontality , place, region, inclusion, contact, support, gravity , attachment, dimen , movement, and path (cf. Bierwisch sionality (point , line, plane, or volume), distance 1993 Jackendoff and Landau 1967 ; Miller and Johnson- Laird ; ; H . H . Clark 1973 1976 ; Wierzbicka 1972 ) . Among things that ; Talmy 1983 ; Olson and Bialystok 1983 , or smell of never seemto playa role are, for example , the color , exact size or shape the figure and ground objects (although seealso Brown 1994 ). -class spatial morphemes - specific Learning? If the meanings of closed 10.4.2.1 Domain - children are so restricted and restricted in similar ways acrosslanguages might take advantageof this in trying to figure out the meaningsof new spatial forms. That is, they might approach the task of learning spatial morphemes with a constrained , entertaining only elementsof meaning that are likely to be hypothesisspace relevant for words in this domain. ) hypothesizedthat although children Reasoningin this way, Landau and Stecker( 1990 in account into to take be should prepared learning new words for objects, shape they should attend to shapeonly minimally in hypothesizingmeaningsfor new spatial

Learning How to Structure Spacefor Language

423

words. To test this hypothesis , they showed three- and five-year-old learners of " a the novel on English top front corner of a box, and told them either This is object " a corp (count noun condition ) or " This is acorp my box" (preposition condition ) . Subjectsin the count noun condition generalizedthe new word to objects of the same ' , ignoring the object s location, whereassubjects in the preposition condition shape , as long as they were in approximately the same generalizedit to objects of any shape location as the original (the top region of the box) .2O While thesefindings are compatible with the claim that children' s hypotheses about the meaning of a new preposition are constrained by their obedience to domain-class spatial word , they are specific restrictions on what can be relevant to a closed not compelling evidence . The subjects had, after all , already learned a number of English prepositions for which the shape of the figure is unimportant , so they -specific bias to disregard shape in may have been influenced by a learned language 21 Whether the claimed blasesexist a for a new . hypothesizing meaning preposition 22 prior to linguistic experienceis, then, still uncertain. In hypothesizing about constraints on the meanings of spatial morphemes , and constraints on children in learning them, researchershave concentrated on closed classspatial words- it is agreedthat spatial verbs as -class items can , open , incorporate a wide range of information about the shape , properties, position, and even identity of figure and ground objects, and about the manner of motion ( Landau and " " Jackendoff 1993 , 235- 236; Talmy 1983 , 273). Following the logic of constraints ' -class spatial morphemes argumentation, children s hypothesis space about closed should therefore be more constrained than their hypothesis space about spatial verbs, since spatial verbs- especially in languagesthat rely heavily on them, like Korean - are sensitive to the same things that spatial prepositions are sensitive to , and a lot more besides .23Because the advantageof built -in constraints is supposedto be that they enable learnersto quickly home in on a word ' s meaning without having to sift endlesslythrough all the things that could conceivably be relevant, it seems -class that children should have an easier time arriving at the meanings of closed spatial morphemes(more constrained) than of spatial verbs (more open) . This is an empirical question, and one that can be examined by comparing, for , whether children acquiring English learn the meaningsof spatial particles example more quickly than children acquiring Korean learn the meanings of roughly comparable ' , children learning Korean were spatial verbs. But in Choi s and my studies as fast at the adult just approximating meanings of common spatial verbs used to encode actions of joining and separation as children learning English were at approximating the adult meanings of English particles used to encode the same actions (cf. figures 10 .3 and 10.4). And this is true even though a number of the Korean children' s early verbs incorporated shapeor object-related information such

424

Meh~ ~ Bowennan

" as " figure is a clothing item," " ground is the head / the trunk / the feet (Choi and Bowerman 1991 , 116 ). It was, then, apparently no harder for children to figure out the meanings of -classspatial putatively lessconstrained spatial verbs than of more constrained closed are constraints . casts doubt on what these domain This outcome specific morphemes in our of and whether are needed for the child , theory acquisition. they really buying 10.4.2.2 Does Learning Spatial Words Involve Bundling Semantic Primitives? Regardless -classspatial morphemesare assistedby of whether children acquiring closed domain-specific constraints, we can still ask whether the task of formulating the meaningsof spatial words is correctly seenas a processof assemblingsemanticprimitives into the right configurations. The appeal to semantic primitives has a long history in the study of languageacquisition- a particularly influential statement of this position was E. V. Clark ' s ( 1973b ) Semantic Features Hypothesis, which held that the development of a word ' s meaning is a processof adding semantic components . Clark ' s one by one until the adult meaning of the word has been reached , even by Clark herself approach was discarded after extensivetesting and analysis various made 1983 and for reason ( ), predictions by the theory were simply good not met (seeRichards 1979and Carey 1982for reviewsand discussions ). In an analysisof what went wrong, Carey ( 1982 , 367 ) makesan important point for : many candidate semantic features are " theory-laden" - they " represent our purposes ' a systematizationof knowledge, the linguistic community s theory building . As such, they depend upon knowledge unavailable to the young child , and they are " therefore not likely candidatesfor developmental primitives (seealso Gopnik 1988 and Murphy and Medin 1985for related arguments ). of with an from the domain , Carey points out that the space example Illustrating 1967 Bierwisch extent ( ) as one of a set of semantic )] proposed by component [tertiary ( needed to distinguish long, and features (along with [ primary] [secondary ]) tall , wide , that young , and thick is highly abstract. It is implausible, she suggests of the use children start out with a notion of [tertiary] that allows them to make sense of the word thick in such diversecontexts as the thicknessof a door , the thicknessof an orange peel, and the thicknessof a slice of bread. More likely , they at first understand what thick picks out in each of these contexts independently , and only later extract what these various uses of thick have in common to arrive at the feature ) and Keil and [tertiary] . A similar analysisis applied to the word tall by Carey ( 1978 of tall in the context first children learn how to use : at Carroll ( 1980 specificreferents ) (e.g., building : ground up; person: head to toe), and only later extract the abstract . According to this critique features (e.g., [spatial extent] [vertical]) that unites theseuses of a are the outcome then semantic features , , lengthy developmental process

Learning How to Structure Spacefor Language

425

" the " lexical organizers (Carey 1978 ) that children extract from words to make sense of their useacrosscontexts not the elementsin terms of which learnersanalyzetheir experienceto begin with . ' Carey s criticism of semantic primitives can be seenas related to the problem of category structure that has preoccupiedus throughout this chapter. Proposedprimitives are usually designated with words of a particular language , often English. Although authors may insist that they do not intend their primitives to be identical with the meanings of words in any actual language , it is not clear what they do in fact intend them to mean. Each language offers a different idea of what some candidate primitive is, and the child must discover this view. Consider, for example , support. Does this candidate primitive include support from all directions, as in English? (cf. " The pillars support the roof ," " The drunkard "" supported himself by leaning against the wall , The actor was supported by invisible " wires as he flew acrossthe stage ) . Or is it restricted to support from below, like the closest equivalent to the English word support in German, stiitzen? Interestingly, these two notions of support are closely aligned with the meaning of ' on' morphemes in the two languages : English on is indifferent to the orientation of the ' ' surface , whereas German auf on is largely restricted to support from supporting below. Figuring out what ' support' is, then, is not entirely a matter of analyzing the circumstancesunder which objects do and do not fall - it also requires discovering how ' support' is conceptualizedin one' s language . semantic to Invoking primitives explain the acquisition of spatial morphemeshas, in the end, a lulling effect- it makes us think we understand the acquisition process better than we do. To the extent that languagesdiffer in what counts as ' support' , as ' containment' or ' inclusion' ' ' ' ' ' ' ( ), as a plane , a point or a volume , and so on, these conceptscannot serveas the ultimate building blocks out of which children construct their meanings . Still left largely unresolved , then, is one of most recalcitrant puzzles of human development : how children go beyond their processing of particular " " " - for example Qntexts morphemesin particular C , (this) cup on (this) table , (this) " picture on (this) wall - to a more abstract understanding of what the morphemes mean. To conclude, I have argued that the existence of cross linguistic variation in the semantic packaging of spatial notions creates a complex learning problem for the child. Even if learners begin by mapping spatial morphemes directly onto precompiled - which is not at all obviousconceptsof space they cannot get far in this way; instead, they must work out the meaningsof the forms by observing how they ' are distributed across contexts in fluent speech . Learners powers of observation to be acute since their semantic , appear very spatial categories show remarkable language specificity by as early as seventeento twenty months of age. Current

426

Melissa

theories about the acquisition of spatial words do not yet dispel the mystery surrounding this feat . In our attempts to get a better grip on the problem , evidence from children learning different languages will continue to play an invaluable role . Acknowledgments I am grateful to Paul Bloom, Mary Peterson , and David Wilkins for their comments on an earlier draft of this chapter, and to Soonia Choi , Lourdes de Leon, Dedre Gentner, Eric Pederson , Dan Slobin, Len Talmy, and David Wilkins for the many stimulating discussionsI have had with them over the years about spatial semantic organization. For judgments about their languagesdiscussedin section 10.2, I am grateful to Magdalena Smoczy Dska ( polish); -Deschner Susana Lopez (Castillian Spanish , ); Riikka Alanen, Olli Nuutinen , Saskia Stossel and Erling Wande ( Finnish); Soonia Choi (Korean); and many colleaguesat the Max Planck Institute for Psycholinguistics(Dutch ) . Notes 1. Theseexamplesare taken from diary records of my daughter E (cf. Bowerman 1978 , 1980 ; Choi and Bowerman 1991 ). 2. Of course , the idea that human beingsapprehendspacewith a priori categoriesof mind has a much older philosophical tradition . 3. David Wilkins ( personalcommunication) suggeststhat Arrernte , an Arandic language of Central Australia , may instantiate the fifth logical possibility - grouping (a) and (b) together (on grounds that both the cup and the apple are easily grasped and moved independently that on c and locative a both covered by general ) treating ( ) differently ( grounds morpheme the handle, being tightly attached, cannot be moved without moving the whole door ) . 4. A similar but more general point is made by Schlesinger( 1977 ), who arguesthat languages depend on many categories that are not needed and will not be constructed purely in the - 189 . In a related point , Olson ( 1970 courseof nonlinguistic cognitive development , 188 ) notes " selected been that had not . . . of a kind information decisions that linguistic previously require " it . to look for no occasion there was because or or attended , , , perceived S. Someof thesecross linguistic differenceswere identified in the courseof typo logical research I conducted together with Eric Pedersonon how languagesexpressstatic topological spatial relations ( Bowerman and Pederson1992 ). " " " 6. Someanalystshave consideredconstructions like the scissorshave butter , the handle of " " " the kitchen door , and the scissorsare buttery to be underlyingly spatial (seeLyons 1967on constructions and Talmy 1972 on attributive adjectives like buttery and muddy ). possessive The question remains, however, why some languagespermit only thesedescriptions of certain relationships between entities, while others also readily describe them with overtly spatial . characterizations 7. Finnish takes the same perspectiveas Dutch on which is figure and which is ground, but " " instead of locating the hands/ tree under the paint/ivy , Finnish locates them in the paint/ivy ) . An English alternative that at first glance might seem comparable to the ( paint/ivy ssa

Learning How to Structure Spacefor Language

427

" " Dutch / Finnish construction is the passive , for example , The tree is covered by/ with / in ivy . " " This sentencedoes allow the covered entity to be the subject of the sentence , but the verb coverstill assignsthe role of figure to the coverer (the ivy) and the role of ground to the covered " " (the tree) (cf. ivy covers the tree ), and the coveredentity can be gotten into subject position only by passivization. 8. To decouplethe patently important question of how speakerscome to control the semantic categories of their language from the loaded Whoman issue , Siobin ( 1987 ) has coined the " ." expression thinking for speaking 9. Here and subsequently , the reader should keep in mind that the English glossesgiven for the Korean verbs serve only as rough guides to their meaning. The actual meanings do not in fact correspond to the meanings of any English words, and can only be inferred on the basis of careful analysis of the situations in which the words are used. 10. The Englishdata camefrom detaileddiary recordsof my two daughtersfrom the start of the one-word stage literature on the early useof Englishpath particles , supplemented by the extensive reviewedin section 10.1.2. Two setsof Korean data were used : ( I ) from 4 children videotaped every 3- 4 weeksby Choi from 14 months old to 24- 28 months old ; and (2) from 4 additional children taped by Choi , Pat Clancy, and Youngjoo Kim every 2 to 4 weeksfrom 19- 20 months old to 25- 34 months old. We are grateful to Clancy and Kim for generouslysharing their data. II . We adopted this procedure rather than , for example , asking children to describeactions we had already performed becauseseveral studies have shown that children first produce -of -state predicates , including spatial morphemes change , either as requests for someone to are about to perform an action- the words seem carry out an action or when they themselves to function to announceplans of intended action (Gopnik 1980 ; Gopnik and Meltzoff 1986 ; Huttenlocher, Smiley, and Charney 1983 ) . If a child failed to respond after several attempts to elicit a request / command for an about-to - be-performed action, we would go ahead and " " perform it and then ask the child , What did I do'! For adults, who caught on immediately to what kind of response we were looking for , we often soon abandonedthe command scenario and simply displayed the actions we wanted labeled. " 12. Degreesof similarity can also be computed- for example , two actions both called take " " " " " out can be regarded as entirely similar , two called take out and pull out are partially similar , and two called " take out " and " put on" are not at all similar. For certain kinds of ' , it is useful to organize each subject s data as a similarity matrix showing whether, for analyses each action paired with each other action, the subject used the same(e.g., put a I in the cell), similar (e.g., .5) or different (0) expressions ; this allows us to disregard the fact that the expressions themselves are different acrosslanguages used , as, of course , is the number of expressions . by different subjects 13. In the quantitative analysesof the data, Choi and I have beenjoined in our collaboration , for example , Boster 1991 for a relevant comparative analysis applied by James Boster (see to the nonlinguistic classification of mammals by children and adults in two cultures). 14. Actions that fall outside of all the circles in a figure were respondedto either very inconsistently " " (i .e., no dominant response could be identified) or (in the caseof the children) received few relevant verbal responses . The useof solid versusdotted lines for the circles has no special - it just makesit easierto visually distinguish . significance overlapping categories

428

Melissa Bowennan

15. This analysis involved cornparing the sirnilarity rnatrices (cf. note 12 ) of speakers in different groups. We first constructed an aggregate rnatrix for the adult speakers of each . We then correlated the sirnilarity rnatrix of each child with the aggregate adult language rnatrix for each language and with the rnatrices of all the other children. ( The cells of the rnatrices , e.g., action 1 paired with action 2, action 1 paired with action 3, etc., constitute the list of variables over which the correlation is carried out.) Finally , we tested whether the children in the youngest age group for each language correlated significantly better -age children speaking with the adult aggregaternatrix for their own language , or with sarne their correlation with adult speakersof . ( Wealso assessed each of the other two languages .) each of the other two languages 16. The only action to which both out and offwere applied (by different children) was taking : the " container" (the piece a piece out of a jigsaw puzzle, and this is readily understandable unclear it is in case so shallow this , probably ) was extrernely shapedhole in the wooden base " " " " to learnerswhether to construe it as a container or a surface (seesection 10.2.2.3 on the . (For the converse problern of learning the conventional conceptualization of particular "objects) " action of putting the piece into the puzzle, eight children said in and only one said " " " on." Another action ) presenting a sirnilar construal problern was put log on train car. The train car in question had short poles sticking up, two on a side, to keep the tiny logs frorn 30 adults acrossthe three languages conceptualizedthis falling off . Despite the poles, 27 of the ' ' situation as one of placing a log on a horizontal supporting surface(English on ( top) , Korean ' ' ' nohta ' put on horizontal supporting surface , Dutch ( boven ) op on (top) ) . But of the 30 children in the youngest age group across the three languages , only 5 used thesewords; their ' ' ' ' was in (English and Dutch ) and nehta put loosely in or kkita fit tightly rnost typical response (Korean). 17. This pattern in Dutch also arguesagainst a hypothesisthat severalpeople have suggested to rne: that English-speaking children rnay learn on and off in connection with clothing as a , so these usesshould not be analyzed as part of a , self-contained pair of rneanings separate -oriented relationships. The clothing rnore generalpattern of associatingon and offwith surface ' ' to interact in the course of developrnentwith other usesof uit in Dutch use of uit out seerns children, so this argurnent is incorrect for Dutch , and by extension probably also for English. - 113 , for other ernpirical arguments against the proposal , 110 (SeeChoi and Bowerman 1991 ' that there is extensivehornonymy or polysernyin children s early acquisition of spatial words.) 18. A sirnilar exarnple is provided by children learning Tzotzil Mayan ( Bowerman, de Leon, " " and Choi 1995 ) . One of the earliest spatial rnorphernesfor joining actions that thesechildren acquire is the verb xoj , and they seern to use it , before age 2, for a range of events that to the English child categoriesin or on nor to the Korean child category corresponds neither ' kkita ' fit tightly . In adult speech , the root xoj picks out a configuration of a long thing encircled by a ring -shaped thing , and can be used , to describe either putting a , for exarnple Tzotzil . When adult over a a a or speakerswere informally tested pole ring pole through ring on the sarneset of spatial actions Choi and I usedin the elicited production describedin section 10.3.2, they used xoj for putting tight - and loose-fitting rings on poles and occasionally for putting on clothing (the ring -and-pole configuration is instantiated by the encirclernentof arms and pantlegs and legs by sleeves , and head by wool cap) . (Adults rnore , feet by socksand shoes " " often describeddonning clothing with a verb that rneans put on clothing . ) Very srnall Tzotzil

Leamin ~ How to Structure Spacefor Language

429

than adults ) for putting childrenalsoused xoj for putting ringson polesand(morefrequently with our these and experimental hat wool manipulations , beyond on shoes , , and , socks -pole the ring-and materials , they usedit for other actionsconfonningto or approximating into car a and , beads putting suchasthreading , puttinga coiledropeovera peg configuration children of on in and categories Englishspeaking a long thin box. This rangeoverlapsthe ' the kkita ' fit tightly and .3b); it alsooverlaps than either(seefigure 10 but is morerestricted ' ' of the Koreanchildren , is different , but, again nehta put looselyin (or around ) categories .4b). from both (cf. figure 10 unitsonly in their semantic . Opinionsvary on whether 19 primitivesareirreducible proposed or also or are in primitives , perceptual conceptual roleasbuildingblocksfor meaning language in this section . The remarks on a nonlinguisticlevel applyeitherway. -specific sensitivities to whethera learnerconstrained by domain 20. In a differentapproach a connec ) equipped of spatialwordsacross , Regler( 1995 themeanings languages canacquire and motivated structuraldevices by neurobiological psychophysical tionist modelwith specific -by-framefilms instantiatingthe with frame . Presented on the humanvisualsystem evidence of several versions in schematized on home to was able model the words of , meaning spatial sucha Whether . Russian and 10 .2.1 in section 3 cf. Mixtec in ), ) , ( ( English spatialcategories in diverse situations objects , including a morerealisticsetof spatial modelcanlearnto classify . be seen to remains functionalrelationships , all their complicated think a novel in what learners that blases shows 1993 Gentner and 21. A study by Imai ) ( . of a particularlanguage with the properties arisethroughexperience canindeed word means -speakingsubjects - and Japanese , both child and showedthat English Theseinvestigators with a complex in connection objectreferred that a word introduced in assuming adult, agreed to the referred substance a of context in the introduced word that a gooey to the object , and about a word introducedin the contextof . But theydifferedin their assumptions substance that the . Englishchildrenand adultsassumed a novelsimpleobject , suchas a cork pyramid counterparts their whereas material of , Japanese to same word referred objectsregardless shaped . Imai of shape material to entitiesmadeof the same , regardless that it referred assumed ' about ) hypotheses and Gentnerhad predictedthis outcomeon the basisof Lucy s ( 1992 . classifiers that do anddo not havenumeral of nounsin languages in themeanings differences . For example of theseblases cause , if childrenare biased . Also uncertainis the possible 22 words class closed , is this because in spatial againstdetailedshapeinformation learning has argued 1985 1983 As , ? class closed , are because or Talmy ( the words are spatial , they acrossa wide rangeof semantic have highly schematic -classmorphemes meanings closed .) domains - 176 a setof meaning particularlyrelevant components 23. Pinker( 1989 , 172 ) hasproposed -class for closed relevant set the than less constrained is far set for learningverbs , but this " : a stateor motion; " the main event direction , and , It includes path . spatialmorphemes( distribution and or actor theme of a (aspect ; manner temporal causation location ; ; properties ; that canbe to capture everything supposed andphase , etc.) Nor arethecomponents ); purpose to relevant be that can of those but verb a of to themeaning , aspects meaning only important 's . behavior a verb syntactic

430
References

Bowerman Melissa

. Infant of spatialrelationships Antell, S. E. G., and Caron , A. J. ( 1985 ). Neonatalperception - 23. Behavior andDevelopment , 8, 15 : Object the existence and the locationof hiddenobjects , R. ( 1986 ). Representing Baillargeon - 41. -old infants . Cognition in 6- and 8-month , 23, 21 permanence -old infants in 3.5- and 4.5-month . Developmental , R. ( 1987 ). Objectpermanence Baillargeon . , 23, 655 664 Psychology , J. C. ( 1990 , M., DeVos , J., and Black , R., Graber ). Why do younginfantsfail to Baillargeon - 284 . ? Cognition search for hiddenobjects , 36, 255 -Chadha . of left-right spatialrelations Dehl , P. D. ( 1995 , G., and Elmas ). Infant categorization BritishJournalof Developmental , 13 , 69- 79. Psychology in narrative : A cross events Berman , D. I. ( 1994 , R. A., and Siobin ). Relating linguistic developmental . . Hillsdale Erlbaum , NJ: Lawrence study . Foundations of Germanadjectivals Bierwisch universals , M. ( 1967 of Language ). Somesemantic , 3, 1- 36. . In W. Deutch(Ed.), . M. ( 1981 in the development of word meaning Bierwisch ). Basicissues . 387 . New York: Academic Press Thechild's construction 341 , of language . Cambridge : Form andfunction in emerging Bloom , L. ( 1970 grammars ). Language development . , MA: MIT Press . The : Theuseof singlewordutterances Bloom , L. ( 1973 beforesyntax ). Onewordat a time : Mouton. Hague between 2 and 4 monthsof of orientationcategories Bomba , P. C. ( 1984 ). The development - 636 . . Journalof Experimental ChildPsychology , 37, 609 age Boster , J. ( 1991 ). The informationeconomymodelappliedto biologicalsimilarity data. In - 225 . Washington .), Sociallyshared L. Resnick , 203 , J. Levine , and S. D. Teasely cognition (Eds . Association , DC: American Psychological : Freeman . in infancy Bower , 2d ed. SanFrancisco , T. G. R. ( 1982 ). Development : A cross Bowerman , M. ( 1973 reference linguistic studywith special ). Early syntactic development . : Cambridge to Finnish . Cambridge UniversityPress into some current : An investigation of word meaning Bowerman , M. ( 1978 ). The acquisition - 287 . .), Thedevelopment . In N. Waterson and C. Snow(Eds conflicts , 263 of communication NewYork: Wiley. -learning in thelanguage andorigin of semantic Bowerman , M. ( 1980 ). Thestructure categories : Newapproach es to the .), Symbolas sense child. In M. L. Fosterand SH . Brandes (Eds - 299 . . New York: Academic Press , 277 analysis of meaning
' ? In D . I . Slobin ( Ed.), The cross Bowennan, M . ( 1985 ). What shapeschildren s grammars issues 1257 . Hinsdale NJ: . Vol . 2 Theoretical 1319 , , , linguistic study of languageacquisition Lawrence Erlbaum.

How to Structure forLanguage Learnin2 Space

431

Bowerman : What role do cognitivepredispositions , M. ( 1989 ). Learninga semantic system - 169 ? In M. L. Rice and R. L. Schiefelbusch .), Theteachability . , 133 (Eds play of language . Baltimore : Brooks Bowerman on language : Do cross , M. ( 1993 ). Typological perspectives acquisition linguistic ? In E. V. Clark (Ed.), The proceedings of the Twenty -fifth Annual patterns predictdevelopment . StanfordCA: Centerfor the Studyof Language Child Language Research Forum and , 7- 15 Information . -specific Bowerman to language in earlygrammatical . , M. ( 1994 ). From universal development Transactions the London B346 37 45. , , , Philosophical of RoyalSociety ' Bowerman . The : Cognitiveversus , M. ( 1996 ) originsof childrens spatialsemantic categories In J. J. Gumperz and S. C. Levinson .), Rethinking linguisticdeterminants (Eds linguisticrelativity - 176 . Cambridge : Cambridge . , 145 UniversityPress Bowerman of spatial , M., and Choi, S. ( 1994 ). Linguistic and nonlinguisticdeterminants semantic . Paperpresented at the BostonUniversityConference on Language development . , January Development Bowerman : , M., de Leon , L., and Choi, S. ( 1995 , particles , and spatial semantics ). Verbs to talk aboutspatialactions in typologicallydifferentlanguages . In E. V. Clark (Ed.), Learning - 110 -seventh AnnualChildLanguage . Stanford Research Forum , 101 , Proceedings of theTwenty CA: Centerfor theStudyof Language and Information . Bowerman on topologicalspatial , M., and Pederson , E. ( 1992 ). Cross linguisticperspectives . at the annual of the American Association relationships Paper presented meeting Anthropological . , SanFrancisco , December Brown : The semantics of static , P. ( 1994 ). The I Ns and ONs of Tzeltal locativeexpressions - 790 of location . Linguistics . , 32 , 743 descriptions Brown . NewYork: FreePress andthings . , R. W. ( 1958 ). Words
Brown , R. W. ( 1973 : The early stages . Cambridge, MA : Harvard University ) . A first language Press . ). The useof body-part terms as locativesin ChalcatongoMixtec , 235- 290. Brugman, C. ( 1983 . Berkeley: U Diversity Report - no . 4 of the Survey of C. alifnmia and Other Indian Languages

of California .

Metaphor in the elaboration of grammatical categories in Mixtec . Unpublished manuscript, Linguistics Department, University of California , Berkeley. Carey, S. ( 1978 ) . The child as word learner. In M . Halle, J. Bresnan , and G. A . Miller (Eds.), 264 293. MA : MIT Press . and Linguistic theory psychologicalreality, Cambridge, : The state of the art . In E. Wanner and L . Gleitman Carey, S. ( 1982 ) . Semanticdevelopment : The state o (Eds.), Languageacquisition / the art , 347- 389. Cambridge: Cambridge University Press . Caron, A . J., Caron. R. F., and Antell , SE . ( 1988 ) . Infant understanding of containment: An affordance perceivedor a relationship conceived ? DevelopmentalPsychology , 24, 620- 627.

432

Melissa Bowennan

: Choi, S., andBowerman to express motionevents in English andKorean , M. ( 1991 ). Learning -specific The influence of language . lexicalization 41 83 121 . , patterns Cognition , . Cienki and the semantics in English , Polish , , A. J. ( 1989 ) Spatialcognition of prepositions andRussian . Munich: Sagner . Clark, E. V. ( 1973a andtheacquisition of word meanings . Cognition , ). Nonlinguistic strategies - 182 2, 161 . ' ' Clark, E. V. ( 1973b in his first ). What s in a word? On the child s acquisitionof semantics . In TE . Moore Ed. and . , 65- 110 ( ), Cognitive language development theacquisition of language NewYork: Academic . Press Clark, E. V. ( 1983 and concepts . In J. H. Flavell and EM . Markman (Eds .), ). Meanings Mussen handbook . Vol. 3, Cognitive and the acquisition of child psychology development of - 840 . New York: Academic Press . , 787 language Clark, E. V. ( 1993 . Cambridge : Cambridge . ). Thelexiconin acquisition UniversityPress Clark, H. H. ( 1973 . , time, semantics , and the child. In TE . Moore (Ed.), Cognitive ) Space andtheacquisition Press . , 27- 63. New York: Academic development of language Colombo , J., Laurie , C., Martelli, T., and Hartig, B. ( 1984 ). Stimuluscontext and infant - 586 orientationdiscrimination . Journalof Experimental ChildPsychology . , 37, 576 DeValois . Oxford: Oxford UniversityPress . , R., and DeValois , K. ( 1990 ). Spatialvision Freeman tasksreveal of , N. H., Lloyd, S., andSinha , C. G. ( 1980 ). Infant search earlyconcepts - 262 containment andcanonical . Cognition of objects . , 8, 243 usage Gentner : Linguisticrelativity versus beforeverbs natural , D. ( 1982 ). Why nounsare learned . In S. A. KuczajII (Ed.), Language . Vol. 2, Language , thought , and partitioning development - 334 . Hillsdale culture . , 301 , NJ: Erlbaum Gibson of affordances in development : Therenascence of functionalism , E. J. ( 1982 ). Theconcept - 81. Minnesota . In W. A. Collins(Ed.), Theconcept on Child , 55 of development Symposia . Hillsdale . , vol. 15 , NJ: Erlbaum Psychology Gleitman . Language of verbmeanings , 1, 3- 55. , L. ( 1990 ). The structuralsources Acquisition . Chicago : Universityof Chicago . Press , A. E. ( 1995 ). Constructions Goldberg -old children -month of non nominalexpressions in 12 24 ). The development Gopnik, A. ( 1980 . PhiD. diss ., Oxford University . and semantic as theorychange . Mind andLanguage ). Conceptual Gopnik, A. ( 1988 development - 216 . , 3, 197
) . Words, plans, things, and locations: Interactions Gopnik , A ., and Meltzoff , A . N . ( 1986 between semantic and cognitive development in the one-word stage . In S. A . Kuczaj II and - 223. Berlin: Springer. MD . Barrett ( Eds.), The development , 199 of word meaning ' ' Griffiths , P., and Atkinson , M . ( 1978 ). A door to verbs. In N . Waterson and C. Snow ( Eds.), The development , 311- 331. New York : Wiley . of communication

for Language LearningHow to Structure Space

433

-word utterance Gruendel : Studyof " up, J. ( 1977 ). Locativeproductionin the single period " " " " " down at the BiennialMeetingof the Societyfor , on off, and in out. Paperpresented Research in Child Development , NewOrleans , March. . Cambridge : , J. J., and Levinson , S. C. ( 1996 ). Rethinkinglinguistic relativity Gumperz . Cambridge UniversityPress - 127 Heine in African languages . Linguistique . , B. ( 1989 , 2, 77 ). Adpositions Africaine Herskovits andspatialcognition : An interdisciplinary , A. ( 1986 ). Language studyof theprepositions in English . Cambridge : Cambridge . UniversityPress Hill , C. A. ( 1978 of spatial and temporalorientation . Berkeley ). Linguistic representation - 538 . , 4, 524 Linguistics Society Huttenlocher of actioncategories in the , J., Smiley , P., and Charney , R. ( 1983 ). Emergence - 93. child: Evidence from verbmeanings . Psychological Review , 90, 72 ImaiM ., and Gentner : Cross , D. ( 1993 ). Linguisticrelativityvs. universal ontology linguistic studies of theobject distinction . In Proceedings /substance , 29. of theChicago Linquistic Society Johnston : Behind and in front of Journalof , J. R. ( 1984 ). Acquisitionof locativemeanings - 422 ChildLanguage . , 11 , 407 Johnston : The evidence from childrenlearningEnglish . , J. R. ( 1985 ). Cognitiveprerequisites - 1004 In D. I . Siobin (Ed.), The cross . Vol. 2, 961 . linguisticstudy of language acquisition Hillsdale . , NJ: Erlbaum Johnston of locativeexpressions in English , J. R., and Siobin , D. I . ( 1979 , ). The development -CroatianandTurkish. Journalof ChildLanguage - 545 Italian, Serbo . , 6, 529 Keil, F. C. ( 1979 of the youngchild' s ability to anticipatethe outcomes ). The development - 462 of simplecausal . ChildDevelopment events . , 50, 455 Keil, F. C. ( 1990 on constraints : Surveying the epigenetic . Cognitive ). Constraints landscape - 168 . Science , 14 , 135 ' " " Keil, F. C., and Carroll, J. J. ( 1980 ). The child s acquisitionof tall : Implicationsfor an alternative viewof semantic . Papers andReports onChildLanguage , development Development 19 , 21- 28. Lakoff, G. ( 1987 : Whatcategories reveal . aboutthemind , fire , anddangerous ). Women things : Universityof Chicago Press . Chicago " " " " Landau and spatial , B., and Jackendoff , R. ( 1993 ). What and where in spatiallanguage . Behavior a/ and Brain Sciences 16 217 238 . , , cognition Landau andplaces : Syntactic , B., andStecker , D. S. ( 1990 ). Objects geometric representations - 312 in earlylexicallearning . Cognitive . , 5, 287 Development . Vol. 1, Theoreticalprerequisites . , R. W. ( 1987 Langacker ). Foundations of cognitive grammar Stanford . , CA: StanfordUniversityPress . Vol. 1. Evanston , W. ( 1939 ). Speech , ILNorthwest Leopold development of a bilingualchild ern UniversityPress .

434

Melissa Bowennan

ofa concept anda word. Journal Levine , S. ( 1982 , S. C., andCarey ). Up front: Theacquisition - 657 . , 9, 645 of ChildLanguage -part terminology : Tzeltalbody Levinson , andlinguisticdescription , shape , S. C. ( 1994 ). Vision - 855 . . Linguistics andobjectdescription , 32, 791 and nonlinguistic : Linguisticcategories Levinson , S. C. (in press ). From outer to inner space . Cambridge . and E. Pederson Eds . In J. and ), Linguistic conceptual representation ( Nuyts thinking . : Cambridge UniversityPress : Anthropology Kant amongthe Tenejapans Levinson , P. ( 1994 ). Immanuel , S. C., and Brown . Ethos asEmpiricalPhilosophy , 22, 3- 41. : A reformulation of the linguisticrelativity ). Language diversityandthought Lucy, J. A. ( 1992 Press . . : University CambridgeCambridge hypothesis . Foundations . A J. 1967 note on , and locativesentences , existential , of ) possessive Lyons ( - 396 . , 3, 390 Language -part locatives . and metaphoric extensions : Prototypes , R. E. ( 1989 ). Zapotec body MacLaury - 154 . Journalof American International , 55, 119 Linguistics : Theunderstanding in infancy MacLean , D. J., andSchuler , M. ( 1989 ). Conceptual development - 1137 . . ChildDevelopment of containment , 60, 1126 . Psychological Review : II Conceptual , Mandler , J. ( 1992 primitives ). How to build a baby . 99 587 604 , . Cambridge : Problems in children andnaming Markman , E. M. ( 1989 of induction ). Categorization . , MA: MIT Press -word -Nicholich of relationalwords in the single McCune , L. ( 1981 ). The cognitivebases . Journalof ChildLanguage , 8, 15 34. period . Cambridge andperception Laird, P. N. ( 1976 Miller, G. A., and Johnson , MA: ). Language . HarvardUniversityPress . coherence , D. L. ( 1985 ). The role of theoriesin conceptual Murphy, G. L., and Medin . 92 289 316 Review , , Psychological -old infants . Needham , R. ( 1993 , A., and Baillargeon ). Intuitionsabout supportin 4.5-month . , 47, 121 148 Cognition in acquisitionand development : Interrelations Nelson , K. ( 1974 , word, and sentence ). Concept - 285 . . Psychological Review , 81, 267 . : Aspectsof a cognitivetheory of semantics and thought Olson , D. R. ( 1970 ). Language 77 . Review 257 273 , , Psychological : Thestructure anddevelopment D. R. and Olson , E. ( 1983 , , of the ). Spatialcognition Bialystok . . Hillsdale mental , NJ: Erlbaum of spatialrelations representation . In G. B. FloresD ' Arcaisand W. J. Parisi , D., and Antinucci , F. ( 1970 ). Lexicalcompetence . : North-Holland . Amsterdam .), Advances in psycholinguistics M. Levelt(Eds , 197 210 . . NewYork: BasicBooks , J. ( 1954 ). Theconstruction of realityin thechild Piaget

Learning How to Structure Spacefor Language

435

' . London : Routledge and ) . The child s conceptionof space Piaget, J., and Inhelder, B. ( 1956

. Paul Kegan

Pieraut- Le Bonniec, G. ( 1987 ). From visual-motor anticipation to conceptualization: Reaction to solid and hollow objects and knowledge of the function of containment. Infant Behaviorand Development , 8. 413- 424.

. Cambridge : Theacquisition structure Pinker andcognition , , S. ( 1989 ). Learnability of argument MA: MIT Press . . of above andbelowspatialrelations , P. C. ( 1994 ). Thecategorization by younginfants Quinn - 69. ChildDevelopment , 65, 58 for a general of obliqueorientations , P. C. ( 1986 , P. C., andBomba ). Evidence Quinn category - 354 -old infants . in four-month . Journalof Experimental ChildPsychology , 42, 345 in early infancy . Merrill-Palmer , P. C., and Elmas , P. D. ( 1986 ). On categorization Quinn . 32 331 363 , , Quarterly . Cognitive , T. ( 1995 ). A modelof the humancapacityfor categorizing spatialrelations Regler - 88. , 6, 63 Linguistics ' ' ' Richards , M. M. ( 1979 ). Sortingout whats in a word from what s not: EvaluatingClark s . Journalof Experimental semantic features ChildPsychology , 27 , 1 47. theory acquisition and linguisticinput in language , I. M. ( 1977 ). The role of cognitivedevelopment Schlesinger - 169 . Journalof ChildLanguage . , 4, 153 development Sinha , M., and Plunkett , K . ( 1994 , C., Thorseng , L. A., Hayashi ). Comparativespatial . Journal : from Danish and Japanese and language Evidence semantics , , English acquisition - 287 . , 11 , 253 of Semantics ' of dynamicrelations Sitskoom , M. M., and Smitsman , A. W. ( 1995 ). Infants perception - 447 . ? : Passing between , 31, 437 Psychology objects throughor support Developmental . In C. A. for the development of grammar globin, D. I. ( 1973 ). Cognitiveprerequisites . York: I. . 175 208 New and D. globin Eds Studies child , ), ( of language development Ferguson . Holt, Rinehart , andWinston -makingcapacity . In D. I. globin for thelanguage globin, D. I. ( 1985 ). Cross linguisticevidence - 1256 . issues . Vol. 2, Theoretical , 1157 (Ed.), Thecross linguisticstudyof language acquisition . Hillsdale , NJ: Erlbaum AnnualMeetingof . Proceedings globin, D. I. ( 1987 of the Thirteenth ). Thinking for speaking 444 . 13 435 theBerkeley , , Linguistics Society . E. S. , K. ( 1992 , , , K., Macomber , J., and Jacobson ). Originsof knowledge Breinlinger Spelke - 632 . Review , 99, 605 Psychological , S. E., Ehrlich , S. M., andBreinlinger , K. ( 1994 , E. S., Katz, G., Purcell ). Early knowledge Spelke - 130 . . Cognition of objectmotion: Continuityandinertia , 51, 107 . PhiD. diss . University of in Englishand Atsugewi structures , L. ( 1972 ). Semantic Talmy . California , Berkeley

436

Melissa Bowerman

. In H . Pick and L . Acredolo ( Eds.), Spatial Talmy , L . ( 1983 ) . How languagestructures space orientation: Theory, research , and application, 225- 282. New York : Plenum. ) . Lexicalization patterns: Semantic structure in lexical form . In T . Shopen Talmy , L . ( 1985 . Vol . 3, Grammatical categoriesand the (Ed.), Language typology and syntactic description lexicon, 57- 149 . Cambridge: Cambridge University Press . Talmy, L . ( 1991 ) . Path to realization: A typology of event conftation. Proceedingsof the Seventeenth Annual Meeting of the Berkeley Linguistics Society, 17, 480- 519. [Supplement in - 187 the Buffalo Papersin Linguistics, 91-01, 182 .] von der Heydt , R., Peterhans , E., and Baumgartner, G. ( 1984 ) . Illusory contours and cortical - 1262 neuron responses . Science . , 224, 1260 Whorf , B. L . ( 1956 , thought, and reality . Edited by J. B. Carroll . Cambridge, MA : ) . Language MIT Press . Wierzbicka, A . ( 1972 ) . Semanticprimitives. Frankfurt : Athenium . Wilkins , D ., and Senft, G. ( 1994 ) . A man, a tree- and forget about the pigs: Spacegames , . Paper spatial referenceand an attempt to identify functional equivalents across languages presented at the Nineteenth International L .A .U .D . Symposium on Language and Space , Duisburg, March.

Chapter11
-

Sp~ce to Think Philip N . Johnson-Laird

D O 11 .1 l Dtroducti
Perception is the transformation of local information at the sensoriuminto a mental model of the world at a distance , thinking is the manipulation of such models, and action is guided by its results. This account of human cognition goes back to the remarkable Scottish psychologist, Kenneth Craik ( 1943 ), and it has provided both a and a central component of the of human the for of research cognition study program of visual the final . Thus of mental perception, according stage representations theory world of the to Marr ( 1982 , which the visual ), delivers a three dimensional model systemhas inferred from the pattern of light intensities falling on the retinas. Mental : to understand discourseis models likewise underlie one account of verbal comprehension the situation that it model of mental a to construct this account on , , . The author and his 1987 1983 Garnham Laird describes(see ; ) , Johnson , for example colleagueshave developed this account into a theory of reasoning both inductive , and deductive- in which thinkers reason by manipulating models of the world (see . 1991 and Laird Johnson for example ) , Byrne The idea of mental models as the basis for deductive thinking has its origins in the following idea: the inference Consider Thebox is on the right of thechair, the box andthechair, Theball is between Therefore , the ball is on the right of thechair. is madeinvolvessettingup an internal The most likely way in which such an inference . This representation the the scene of may be a vivid premises depictedby representation . The crucialpoint is concern is of no substance its delineation abstract a or fleeting image canbe sothat theconclusion of thescene mirror thespatialrelations that its formalproperties . It may be objected , readoff in almostasdirect a fashionas from an actualarray of objects canbe made is unnecessary however , that the inference , that sucha depictionof the premises

438

PhilipN. Johnson- Laird

, which indicatethat itemsrelatedby , or rulesof inference principles by an appealto general - that relationaltermsaretagged mustbecollinear according between , this view , etc. However - founderson more complexinferences . An inference schema to the inference they permit to be handledwithout to be far too complicated of the following sort, for instance , seems : the scene of internal an representation constructing ball is on the right of thecueball, The blackball is directlybeyondthecueball. Thegreen . them and thereis a red ball between meand the blackball, thenthecueball is Therefore , if I moveso that the red ball is between on my left. to bemadewithout that permitsuchinferences schema to frameinference Evenif it is possible is actually of an internalrepresentation theconstruction , it is mostunlikelythat this approach - 13 -Laird 1975 . (Johnson in makingthe inference , 12 ) adopted of the model theory of deduction, but the intuition This passage captures the essence made are that spatial inferences by imagining spatial scenesturned out not to be shared by all investigators. Twenty years have passedsince the argument above was first formulated , and so the aim of this chapter is, in essence , to bring the story up to date. It contrasts the on formal rules of inference based account with an model theory , and it presents evidencethat spatial reasoningis indeed basedon models. It then arguesthat spatial models may underlie other sorts of thinking - even thinking that is not about spatial relations. It presentssome new results showing that individuals often reason about temporal relations by constructing quasi-spatial models. Finally , it demonstratesthat one secretin using diagrams as an aid to thinking is that their spatial representations should make alternative possibilities explicit . ado. and Mental Models 11.2 Propositional Represent of the answer is that its structure What does one mean by a mental model? The essence . A mental model is accordingly it of what structure to the represents corresponds ' , a biochemist s similar in structure to a physical model of the situation , for example ' . The parts of the model model of a molecule, or an architect s model of a house correspond to the relevant parts of the situation , and the structural relations between the parts of the model are analogous to the structural relations in the world . Hence, individual entities in the situation will be representedas individuals in the model, their properties will be representedby their properties in the model, and the relations among them will be representedby relations among them in the model. Mental models are partial in that they representonly certain aspectsof the situation , and they thus correspond to many possible states of affairs, that is, there is a many-to-one , too , have theseproperties, mapping from situations in the world to a model. Images

Spaceto Think

439

, they may representseveral but models need not be visualizable, and unlike images distinct sets of possibilities. These abstract characterizations are hard to follow , but they can be clarified by contrasting mental models with so-called propositional . representations : , consider the assertion To illustrate a propositional representation A triangle is on the right of a circle. Its propositional representation relies on some sort of predicate argument structure, such as the following expressionin the predicatecalculus: (3x) (3y) (Triangle (x ) & Circley ) & Right of (x ,y , " " where 3 denotes the existential quantifier for some and the variables range over individuals in the domain of discourse , i.e. the situation that is under description. The "- a " hybrid languagespoken expressioncan accordingly be paraphrasedin Loglish only by logicians- as follows: For somex and for somey, such that x is a triangle and y is a circle, x is on the right of y . The information in the further assertion The circle is on the right of a line : can be integrated to form the following expressionrepresentingboth assertions (3x) (3y) (3z) (Triangle (x ) & Circley ) & Line (z) & Right of (x , y) & Right of (y, z . A salient feature of this representationis that its structure doesnot correspond to the . The key component of the propositional representation structure of what it represents is Right -of (x , y) & Right -of (y , z), in which there are four tokens representingvariables. In contrast, the situation itself has three entities in a particular spatial relation. Hence, a mental model of the situation must have the samestructure, which is depicted in the following diagram: I 0 ~ where the horizontal dimension corresponds to the left -to -right dimension in the situation. In what follows , such diagrams are supposedto depict mental models, and will often be referred to as though they were mental models. Each token in the present , mental model has a property corresponding to the shapeof the entity it represents between relation the to and the three tokens are in a spatial relation corresponding . In the caseof such a the three entities in the situation described by the assertions

440

-Laird Philip N. Johnson

and spatial model, a critical feature is that elementsin the model can be accessed . to axes in D ten Sofparameters corresponding updated The processof inferencefor propositional representationscalls for a systembased on rules, and psychologistshave proposed suchsystemsfor spatial inferencebasedon fonnal rules of inference (see , Hagert 1984 ; Ohlsson 1984 ) . Hence, in , for example order to infer from the premisesabove the valid conclusion A triangle is on the right of a line, to rely on a statementof the transitivity of " on the right of " : it is necessary (Vx) (Vy ) (Vz ) Right -of (x , y) & Right -of (y, z -+ Right -of (x , z , where V denotesthe universal quantifier " for any" and -+ denotesmaterial implication " " ( if . . . , then . . . ) . With this additional premise(a so-called meaning postulate) and a set of rules of inferencefor the predicatecalculus, the conclusion can be derived . in the following chain of inferential steps The premisesare ( I ) (3x) (3y) (Triangle (x ) & Circley ) & Right -of (x ,y (2) (3y) (3z) (Circley ) & Line (z) & Right -of (y, z ) (Vz ) Right -of (x ,y) & Right -of (y, z -+ Right -of (x , z (3) (Vx) (Vy The proof calls for the appropriate instantiations of the quantified variables, that is, one replacesthe quantified variables by constantsdenoting particular entities: (4) (3y) (Triangle (aCircley [from ( I )] ) & Right -of (a,y [from (4)] (5) (Triangle(a) & Circle (b) & Right -of (a, b [from (2)] (6, 7) (Circle (b) & Line (c) & Right -of (b, c There are constraints on the processof instantiating variables that are existentially quantified, but universal quantifiers range over all entities in the domain, and so the meaning postulate can be freely instantiated as follows: (8- 10 ) Right -of (a, b) & Right -of (b, c -+ Right -of (a, c [from (3)] . A rule for conjunction The next stepsuse fonnal rules of inference for the connectives B where A and B can denote of the fonD A & that a ), ( given premise stipulates derive the conclusion B. one can of of assertions any degree complexity, compound Hence one can detach part of line 5 as follows: ( II ) Right -of (a, b) [from (5)] and part of line 7 as follows:

Spaceto Think

441
[from (7)]

( 12 ) Right -of (b, c)

Another rule allows any two assertionsin separatelines to be conjoined, that is, given premisesof the form A , B, one can derive the conclusion (A & B) . This rule allows a conjunction to be formed from the previous two lines in the derivation: ), ( 12 )] [from ( 11 ( 13 ) (Right -of (a, b) & Right -of (b, c " This assertion matches the antecedent of line 10 , and a rule known as modus " ponens stipulates that given any premisesof the form (A -+ B), A , one can derive the conclusion B. The next step of the derivation proceedsaccordingly: ( 14 ) Right -of (a, c) )] [from ( 10, ( 13 The rules for conjunction allow the detachment of propositions from previous lines and their assemblyin the following conclusion: ( 15- 18 ) Triangle (a) & Line (c & Right -of (a, c Therefore, the triangle is on the right of the line. The processof inferencefor models is different. The theory relies on the following simple idea. A valid deduction, by definition , is one in which the conclusion must be true if the premisesare true. Hence what is neededis a model-basedmethod to test for this condition . Assertionscan be true in indefinitely many different situations, and so it is out of the question to test that a conclusion holds true in all of them. But a mental model can stand for testing can be done in certain domains preciselybecause . it is done for spatial inferences in is how situations. Here , principle , indefinitely many above: the Consider, again, example A triangle is on the right of a circle. The circle is on the right of a line. The assertionssay nothing about the actual distancesbetweenthe objects. Instead of ,a trying to envisageall the different possible situations that satisfy these premises mental model leaves open the details and captures only the structure that all the different situations have in common: I 0 ~ where the Ieft-to-right axis corresponds to the left -right axis in space , but the distances . This model represents betweenthe tokens have no significance only the spatial of the objects, and it is the only possible model of the premises , that is, no sequence other model corresponding to a different Ieft-to-right sequenceof the three objects . Now consider the further assertion: satisfiesthe premises )] [from ( 5), (7), ( 14 Finally , this propositional representationcan be translated back into English:

442 The triangleis on the right of the line.

Philip N . Johnson- Laird

there are no other models of the premises It is true in the model, and, because , it must reasoners be true given that the premisesare true. The deduction is valid , and because can determine that there are no other possible models of the premises , they can not ). only make this deduction but also know that it is valid (seeBarwise 1993 The sameprinciples allow us to determine that an inferenceis invalid . Given, say, the inference A triangle is on the right of a circle, A line is on the right of the circle, Therefore, the triangle is on the right of the line, the first premiseyields the model 0 ~ but now when we try to add the information from the second premise , the relation betweenthe triangle and the line is uncertain. One way to respond to such an indeterminacy is to build separatemodels for each possibility : 0 I ~ 0 ~ I ignoring the possibility that the triangle and the line might be, say, one on top of the other. The first of thesemodels showsthat the putative conclusion is possible , but the be on the . It follows that the secondmodel is a counterexample to it triangle may right of the line, but it does not follow that the triangle must be on the right of the line. ? Not at Does the model theory abandon the idea of propositional representations of an assertion a of the out to be essential to have all. It turns meaning representation independentof its particular realization in a model. The theory accordingly assumes that the first step in recovering the meaning of a premise is the construction of its - a representationof the truth conditions of the premise propositional representation . then used to update the set of models of the premises is . This representation The use of mental models in reasoning has two considerableadvantagesover the useof formal rules. The first advantageis that it yields a decision procedure- at least the predicate calculus for domains such as spatial reasoningthat can have one, because is provably without any possible decision procedure. An inference is valid if its conclusion holds in all the possiblemodels of the premises , and it is invalid if it fails to hold in at least one of the possiblemodels of the problems. Granted that problems remain within the capacity of working memory, then it is a simple matter to decide whether or not an inferenceis valid. One examinesthe models of the premises , and a is in The situation all of them. conclusion is valid if , and only if , it is true very

Spaceto Think

443

different in the caseof fonnal rules. They have no decision procedure. Quine ( 1974 , 75) commented on this point in contrasting a semantic decision procedure for the propositional calculus (akin in some ways to the mental model account of that domain ) and an approach basedon fonnal rules. Of the use of fonnal rules, he wrote: " It is inferior in that it affords no general way of reaching a verdict of invalidity ; " failure to discover a proof for a schemacan mean either invalidity or mere bad luck. The sameproblem, as Barwise ( 1993 ) has pointed out , haunts psychological theories basedon formal rules. The searchspaceof possiblederivations is vast, and thus such theories have to assumethat reasonersexplore it for a certain amount of time and ' then give up . Barwise remarks: " The ' searchtill you' re exhausted strategy gives one " 337 . Models that not follow at best an educated correct does , ( ) guess something allow individuals to know that there is no valid conclusion. The secondadvantageof mental models is that they extend naturally to inductive inferencesand to the infonnal arguments of daily life to which it is so hard, if not , for example , Toulmin 1958 ) . Such impossible, to apply fonnal rules of inference(see in inferencesand arguments nevertheless differ their strength (Osherson , Smith, and - any inference Shafir 1986 ) . The model theory implies that the strength of an inference - dependson the believability of its premisesand on the proportion of models of the premises in which the conclusion is true (Johnson-Laird 1994 ) . Hence the : model theory provides a unified account of inference . If the conclusion holds in all possible models of the premises , it is necessary given the premises that is valid. , , deductively . If it holds in most of the models of the premises , then it is probable. . If it holds in somemodel of the premises . , then it is possible . If it holds in only a few models of the premises . , then it is improbable . If it holds in none of the models of the premises , then it is impossible , that is, . inconsistent with the premises The theory fonns a bridge between models and the heuristic approach to judgments of probability based on scenarios(see , for example , Tversky and Kahneman in indetenninacies increases 1973 . As the number of , there is an exponential ) premises growth in the number of possible models. Hence the procedure is intractable for all . However, once individuals have constructed a but small numbers of indeterminacies model in which a highly believable conclusion holds, they tend not to search for alternative models that refute the conclusion. The theory according provides a mechanism for inferential satisficing (cf. Simon 1959 ) . This mechanism accounts for the common failure to consider alternative lines of argument- a failure shown by studies of inference , both deductive (e.g., Johnson- Laird and Byrne 1991 ) and infonnal (e.g., -life disasters Kuhn 1991 and real Perkins, Allen , and Hafner 1983 , for ; ), by many

444

Philip N . Johnson- Laird

example , the operators at Three Mile Island inferred that a relief valve was leaking and overlooked the possibilitythat it wasstuckopen .

11 .3 Algorithmfor SpatialReasoning Based 00 MeotalModels The machineryrequired for reasoningby model calls , not for formal rules of inference but for models true in , , formulatingconclusions procedures constructing models are true in models . The present author has , and testingwhetherconclusions implemented computerprogramsthat makeinferences usingsuchan algorithmfor , sententialconnectives , doubly quantifiedassertions , and severalother syllogisms domainsincludingspatialreasoning . The algorithm for spatialinferences works in the followingway. The initial interpretation of the first premise The triangleis on the right of thecircle " , which is constructedby a compositional yields a propositional representation " : semantics I 00 ) ~ 0 ) . The parameters in order to relate ( I 00 ) specifywhich axesneedto be incremented the triangle to the circle (incrementthe right-left axis , i.e., keepadding 1 to it , as ; hold the front-back axis constant , i.e., incrementit by 0; and hold the necessary i.e. down axis constant increment it 0 , , up by ). Thereare no existingmodelsof the discourse the assertion is first, and so a procedure is calledthat usesthis , because to build a minimalspatialrepresentation : propositional representation ~. 0 In theprogram , thespatialmodelis represented , theinterpretation by an array. Likewise of the second premise The circleis on the right of a line yieldsthe propositionalrepresentation 100 ) 0 I). This representation contains an item in theinitial model is called , andsoa procedure that uses the propositionalrepresentation to updatethis modelby addingthe line in : the appropriate position ~. 0 I Giventhe further, third assertion The triangleis on the right of the line,

Spaceto Think

445

both items in its propositional representationoccur in an existing model, and thus a . This procedure returns procedure is called to verify the propositional representation the with the that true and the value , algorithm always constructs all possible proviso models of the premises , the conclusion is therefore valid. The algorithm has no need for a postulate capturing the transitivity of relations, such as " on the right of ," which are emergent properties of the meaning of the relation and of how it is used to construct models. This emergenceof logical properties - the vagaries has the advantage of accounting for a puzzling phenomenon . The inferencesmodeled in the program are for the in everyday spatial inferences " " on the " deictic" of right of , that is, the relation as perceivedfrom a interpretation 's speaker point of view. Other entities have an intrinsic right -hand side and left -hand side, for example , section 6.1.3) . , human beings (seeMiller and Johnson- Laird 1976 : Hence the following premises ' Matthew is on Mark s right Mark is on Luke ' s right 'can refer to the position of three individuals in relation to the intrinsic right -hand sides of Mark and Luke. To build a model of the spatial relation , the inferential systemneedsto locate Mark , then to establisha frame of referencearound him based ' " " on his orientation , and then to use the semanticsof on X s right to add Matthew to the model in a position on the right -hand side of the lateral plane passingthrough Mark (seeJohnson- Laird 1983 , 261) . The same semanticsas the program usesfor " can be used but instead of " on the , applying to the axesof the spatial array, it right applies to axescentered on each individual according to their orientation . Hence, if ' the individuals are seatedin a line, as in Leonardo da Vinci s painting of the Last Supper, then the model supports the transitive conclusion ' Matthew is on Luke s right . On the other hand, if they are seatedround a small circular table, each premisecan be true, but the transitive conclusion false. Depending on the sizeof the table and the number of individuals seatedaround it , transitivity can occur over limited regions, " " and the same semantics for on X' s right accounts for all the vagaries in the . inference
11.4 Experiment in Spatial Reasoning

The key feature of spatial models is not that they represent spatial relations propositional representations also do that but rather that they are functionally organized on spatial axes and , in particular , that information in them can be accessed

446

-Laird PhilipN. Johnson

. Does such an organization imply that when you have a spatial by way of theseaxes model of a situation , the relevant information will be laid out in your brain in a . A programming language , such as LISP, spatially isomorphic way? Not necessarily values of coordinate the of allows a program to manipulate spatial arrays by way no corresponding their axes , but the data structure is only functionally an array and 's as it runs in a found to be is memory of data computer necessarily physical array level to well spatial functional same high The apply the program. principle may models in human cognition . The model theory makessystematicallydifferent predictions from those of theories based on formal rules. In an experiment reported by Byrne and Johnson Laird . The first sort were ), the subjects carried out three sorts of spatial inference ( 1989 of the premises model a , problems that could be answeredby constructing just single such as the following : The knife is on the right of the plate. The spoon is on the left of the plate. The fork is in front of the spoon. The cup is in front of the knife. What ' s the relation betweenthe fork and cup? We knew from previous results that individuals tend to imagine symmetric arrangements of objects, and so thesepremisescall for a model of this sort: s f p k c

where s denotesa representationof the spoon, p a representationof the plate, and so on. This model yields the conclusion The fork (/ ) is on the left of the cup (c) . There is no model of the premisesthat refutes this conclusion, and thus it follows . In contrast, if individuals reach this validly from this single model of the premises conclusion on the basis of a formal derivation , they must first derive the relation , to infer from the second , for example betweenthe spoon and the knife. They need premise The spoon is on the left of the plate that the converseproposition follows: The plate is on the right of the spoon. " " They can then usethe transitivity of on the right of to infer from this intermediate conclusion and the first premisethat it follows that

Spaceto Think

447

The knife is on the right of the spoon. At this point , they can use certain postulates about two -dimensional relations to derive the relation betweenthe fork and the cup (seeHagert 1984and Ohlsson 1984 for such formal rule systemsof spatial inference ). Problems of the second sort yield multiple models becauseof a spatial indeterminacy , but they neverthelesssupport a valid answer. They were constructed by : changing one word in the secondpremise The knife is on the right of the plate. The spoon is on the left of the knife. The fork is in front of the spoon. The cup is in front of the knife. What ' s the relation betweenthe fork and cup? The description yields models corresponding to two distinct layouts: s k p f c p s f k c

Both thesemodels, however, support the sameconclusion: The fork is on the left of the cup. The model theory predicts that this problem should be harder than the previous one, because reasonershave to construct more than one model. In contrast, theoriesbased on formal rules and propositional representationspredict that this problem should be easier than the previous one becausethere is no need to infer the relation between the spoon and the knife: it is assertedby the secondpremise . Problemsof the third sort were similar but did not yield any valid relation between the two items in the question, for example: The knife is on the right of the plate. The spoon is on the left of the knife. The fork is in front of the spoon. The cup is in front of the plate. What ' s the relation betweenthe fork and cup? In one of the experiments , eighteensubjectsacted as their own controls and carried out the task with six problems of eachof the three sorts presentedin a random order. They drew reliably more correct conclusions to the one-model problems (70% ) than to the multiple -model problems with valid answers(46% ) . Their correct conclusions

448

Philip N . Johnson-Laird

) than to were also reliably faster to the one-model problems (a mean of 3.1 seconds ) . It might be argued the multiple -model problems with valid answers (3.6 seconds an irrelevant contain because are harder they that the multiple model problems another in an . However, experiment, the premise that plays no part in the inference : , for example one-model problems contained an irrelevant premise The knife is on the right of the plate. The spoon is on the left of the plate. The fork is in front of the spoon. The cup is in front of the plate. What ' s the relation betweenthe fork and cup? This description yields the following sort of model: s f p c k

and, of course , the first premise is irrelevant to the deduction. Such problems were easier (61% correct) than the multiple -model problems with valid conclusions reliably ( 50% correct) . Thus the results of the two experimentscorroborate the model theory but run counter to theories that assumethat reasoning depends on formal . rules of inference

. Relado : Modelsof Temporal for Time II .S Space


It seemsentirely natural that human reasonerswould representspatial relations by , but let us push the argument one step further . imagining a spatial arrangement underlie models reasoning in other domains, that is, inferencesthat Perhapsspatial hinge on nonspatial matters may be made by manipulating models that are functionally organized in the same way as those representing spatial relations (see section . Before we examine this 11.3) . A plausible extrapolation is to temporal reasoning inference of rules formal how extension might cope. , let us see , but there are someobstaclesto Formal rules might be usedfor temporal reasoning , at least in them. An obvious difficulty is the large variety of linguistic expressions Indo -European languages , that convey temporal information . Consider just a handful , for Verbs differ strikingly in their temporal semantics(see . cases of illustrative " He the assertion instance . For 1949 and , ) ; Ryle ; Kenny 1963 example, Dowty 1979 " was looking out of the window meansthat for some interval of time at a reference ' time prior to the utterance the observers gazewas out of the window. In contrast, the " " assertion He was glancing out of the window meansthat for a similar interval the observersgazewas alternately out of the window and not out of the window. Tempo

Spaceto Think

449

" raj adverbials can move the time of an event from the time of the utterance ( He is " " " , , for example running now ) to a time in the future ( He is running tomorrow ; see Bull 1963 ) . General knowledgecan lead to a sequential ; and Partee 1984 ; Lyons 1977 " " construal of sententialconnectives , as in He crashedthe car and climbed out , or to " " a concurrent interpretation , as in He crashed the car and damaged the fender. A , and theory of temporal language has to specify the semanticsof theseexpressions . Formal rule particularly their contribution to the truth conditions of assertions theories of inference , in addition , must specify a set of inferential rules for temporal . expressions In fact, no psychological theory based on formal rules of inference has so far been proposed for temporal reasoning , but logicians have proposed various es the following pair of . Quine ( 1974 , 82) discuss analysesof temporal expressions : assertions I knewhim beforehelost his fortune I knewhim whilehewaswith Sunnyrinse and suggeststreating them as assertions of the form , Some Fare G, where F " " , represents moments in which I knew him and G representsfor the first assertion " moments before he lost his fortune " and for the second assertion " moments in , , ." This treatment does not readily yield transitive which he was with Sunnyrinse inferencesof the form a before b, b before c, Therefore, a before c. Other logicians have framed temporal logics as variants of modal logic (see , for on but these and 1971 1967 Rescher ; ), logics depend simple Urquhart example, Prior . temporal operators that do not correspond to the tensesystemsof natural language of time. Their scopeis thus too narrow for the various forms of everyday expressions Hence a more plausible way to incorporate temporal reasoning within a psychological theory based on formal rules of inference is to specify the logical properties of " " temporal expressionsin meaning postulates in a way that is analogous to the psychological theories of spatial reasoningdescribedin section 11.2. Temporal relations probably cannot be imagined in a single visual image. In any case , the eventsthemselves may not be visualizable, and manipulations of this factor have no detectableeffectson reasoning(see , Newstead , Manktelow , and , for example Evans 1982 ) . When ; and Johnson Laird , Byrne, and Tabossi 1989 ; Richardson 1987 one imaginesa temporal sequence , however, it often seemsto unfold in time like the . This sort of representation , though not necessarilyat the samespeed original events

450
uses another sequence For time itself possibility of example clerk sounded events , the to represent is in to the represent one temporal temporal axis of corresponds the assertion axis ( see relations to Johnson in time a . - Laird static

Philip N . Johnson-Laird
1983 spatial , 10 ) . However model of the ,

which representation

The

the

alarm

after

the

suspect

ran

away

calls

for

a model

of

the

form

a , r denotes of the clerk a representation sounding , definite or the of the

in

which

the away

time , and

axis

runs

from

left

to

right

suspect . Events . Hence

running can the be further

a denotes as momentary

a representation or as having

alarm

described assertion

durations

indefinite

The

manager that : the

was

stabbed

while

the

alarm

was

ringing between the onset and offset of the

means alarm

stabbing

occurred

at

some

time

a s the to the for . Yet two which , the vertical infinitely premises the dimension many . Thus alarm allows different the sounded . model , or

where for

s denotes contemporaneous that no precise

representation events . This

of

the model the of the

stabbing

, and

corresponds truth duration occurred of

situations contains of the

have explicit point occurred

in

common

only

representation at which the

stabbing ran

conclusion

The

stabbing true I have in this

after

the

suspect is no model

away of the that all the to minimize : two carries possible use the the that temporal of it if the there has to falsifies it . in . If to .

is

model

, and a attempts too large of , the

there computer to , it models

premises out

implemented this way grows construction , for example b b c . It

program construct then so attempts as to

inferences premises is one

exactly the guide Consider number its

models question number

construct

following

premises

h a b

happens happens happens

before before before befored befored

e happens fhappens

befored c happens ' relation What s the

between

a and

d ?

Spaceto Think

451

When the program works through the premisesin their stated order, it has to construct 239 models to answer the question- a number that vastly exceeds the capacity of human working memory. If the program' s capacity is set more plausibly, say, to four models, it will give up working forwards and then try a depth-first searchbased on the question: What ' s the relation betweena and d? It discoversthe chain leading from the secondpremise(referring to a) through the third premise(referring to event b, which is also referred to by the secondpremise ) to the final premise(referring to d ), and constructs just the single model that these premisessupport. This model yields the conclusion that a happensbefored. The advantagesof this procedureare twofold . First , it ignores all irrelevant premises . Second , it dealswith the premisesin a coreferential order in which eachpremiseafter the first refers to an event already represented in the set of models. Of course, there are problems that defy the program' s capacity for modelseven if it ignoresirrelevant premises . In everydaylife, however , individuals are unlikely to present information in an amount or in an order that overburdens human working memory; they are likely to be sensitive to the limitations of their audience(seeGrice 1975 ) . Hence it seemed appropriate in our experimental study of temporal reasoningto use similarly straightforward materials. 11.6 Experimental Study of Temporal Reasoning havenot hitherto studieddeductivereasoning basedon temporal relations Psychologists , ' Ydewalle of and so Walter Schaeken d the , Gery ( University of Leuven in Belgium), and the presentauthor have carried out an seriesof experimentsexamining the topic . Consider the premisesof the following sort: a before b b before c d while a e while c What ' s the relation betweend and e? " " " where a, b, and so on stand for everyday events , such as John shaves , he drinks " his coffee , and so on. Theseeventscall for the construction of a single model: a de b c

where the vertical dimension allows for events to be contemporaneous . This model the conclusiond supports before e.

452

Philip N . Johnson-Laird

The model theory predicts that this one-model problem should be easier than a similar inference that contains an indeterminacy. For example , the following premises call for severalmodels:
a before c b before c d while b e while c ' What s the relation betweend and e?

The premisesare satisfiedby the following models: a b de c b a deb c a de In all three models, d happens before e, and so it is a valid conclusion. The model , which theory also predicts that the time subjectsspend reading the secondpremise be than the creates the indeterminacy leading to multiple models, should longer reading time of the secondpremise of the one model problem. This multiple model , but the following one-model problem problem contains an irrelevant first premise : also contains an irrelevant first premise a before b b before c d while b e while c What ' s the relation betweendand e? In one of our experiments , we tested twenty-four university students with eight three sorts of problemsabove, and eight versionsof amultiple of each of the versions model problem that had no valid answer. The thirty -two problems were presented under computer control in a different random order to each subject. The two sorts of one model problem were easy and did not differ reliably (93% correct for the problems with no irrelevant premiseand 89% correct for the problems with an irrelevant ), but they were reliably easier than the multiple -model problems with premise valid conclusions(81% correct responses ), which in turn were reliably easierthan the with no valid conclusions (44% correct responses model ) . One problems multiple would expect the latter problems to be difficult becauseit is vital to construct more than one model in order to appreciatethat they have no valid conclusion, whereasthe valid answerwill emergefrom any of the multiple models of the problems with a valid answer. Figure 11.1 shows the reading times for the four premisesof the problems. c

Spaceto Think
13

453

12

() I) ( I) c ~

11

. . .

1M 2M NVC

A
10

8 premise 1 premise 1 . premise 3 premise 4

Fipre 11.1 The mean latenciesfor reading the premisesin the temporal inference experiment. The means are for one-model problems ( I - M ) collapsing over the two sorts, the multiple -model problems with a valid conclusion (2-M ), and the multiple -model problems with no valid conclusion ( NVC) .

454

Philip N . Johnson- Laird

As the figure shows , subjects took reliably longer to read the secondpremise of the multiple -model problems- the premise that calls for the construction of more than one model- than to read the secondpremiseof the one-model problems. Our results, both for this experiment and others that we carried out , establishthree main phenomena , and they imply that reasoning about temporal relations depends on mental models of the sequences of events . The first phenomenon concerns the number of models. When a description is consistentwith just one model, the reasoning task is simple and subjectstypically draw over 90% correct conclusions . When a is consistent with more than one model, there is a reliable decline in description perfonnance. As in the earlier study of spatial reasoning , we pitted the predictions of the model theory against contrasting predictions based on fonnal rules of inference . The results showed that the one-model problems were reliably easier than the multiple -model problems, eventhough the one-model problems call for longer fonnal derivations than the multiple -model problems. ' erroneous The second phenomenon concerns the subjects conclusions . Fonnal rule theories make no specific predictions about the nature of such conclusions : are said to err because subjects they misapply a rule or fail to find a correct derivation . The model theory, however, predicts that erroneous conclusions arise because reasonersfail to consider all the models of the premises , and thus theseconclusions should tend to be consistentwith the premises(i.e., true in at leastone model of them) rather than inconsistent with premises (i .e., not true in any model of them) . The results corroborated this prediction of the model theory. The third phenomenonconcernsthe time subjectstook to read the premisesand to , they took reliably-longer to read a premise respond to the questions. As we have seen that led to multiple models than to read a corresponding premise in a one-model problem. Fonnal rule theories make no such prediction, and it is hard to reconcile this result with such theories becausethey make no use of models. The result also that subjectsdo not construct models that representindetenninacieswithin suggests a single model. If they had done so, then they should have taken no longer to read these premises than the corresponding premises of one-model problems. And , of course , they should not have been more prone to err with indetenninate problems. The times to respond to the questions also bore out the greater difficulty of the multiple -model problems. One final comment on our temporal experiments . Problems that depend on a transitive chain of events , as in the following one-model problem: a de b c

Spaceto Think

455

make an interesting contrast with one-model problems in which the transitive chain is not relevant to the answer: a b de c

If subjectswere imagining the eventsunfolding in time at a more or lessconstant rate, then presumablythey ought to be able to respondslightly faster in the secondcasethan in the first. That is to say, the actual temporal interval betweend and e must be shorter in the second case than in the first. We examined this difference in the experiment in the first describedabove. The meanlatenciesto respondwereas follows: 7.0 seconds . This differencewas not too far from significance caseand 5.8 secondsin the secondcase , and thus perhapsat leastsomeof our subjectswere imagining eventsas unfolding in time rather than simply constructing spatial models of the temporal relations. : How Diagrams Can Help Reasoning 11.7 Spacefor Space Diagrams are often said to be helpful aids to thinking . They can make it easierto find relevant information - one can scan from one element to another element nearby much more rapidly than one might be able to find the equivalent information in a list . Diagrams can make it easierto identify instancesof of numbers or verbal assertions a concept- an iconic representation can be recognizedfaster than a verbal description . Their symmetriescan cut down on the number of casesthat need to be examined . But can diagrams help the processof thought itself? Larkin and Simon ( 1987 ) to information and to it but that reasoners find , recognize grant diagrams help doubt whether they help the processof inference itself. According to 8arwise and , 82), who have developeda computer program, Hyperproof , that Etchemendy( 1992 " helps usersto learn logic : diagrams and pictures are extremely good at presentinga wealth of specific , conjunctive information . It is much harder to use them to present indefinite information , negative information , or disjunctive information . For these , sentences are often better." Hyperproof accordingly captures conjunctions in diagrams . The model theory, however, es disjunctions in verbal statements , but express makes a different prediction . A major problem in deduction is to keep track of . Hence a diagram that helps to make them explicit the possible models of premises should also help people to reason. The result of perceiving such a diagram is a model- according to Marr ' s ( 1982 ) of vision- and thus one has a more direct route to a model than that provided by a verbal description. The verbal description needs to be parsedand a compositional semanticsneedsto be usedto construct its propositional , which is then usedin turn to construct a model. Henceit should representation be easierto reasonfrom diagrams than from verbal descriptions.

456

Philip N . Johnson-Laird

We tested this prediction in two experiments based on so-called double disjunctions ( Bauer and Johnson- Laird 1993 ) . These are deductive problems, which are exemplified in verbal form by the following problem: Julia is in Atlanta , or Raphael is in Tacoma, but not both. Julia is in Seattle , or Paul is in Philadelphia, but not both. What follows? The model theory predicts that such problems basedon exclusivedisjunctions should be easierthan those basedon inclusive disjunctions: Julia is in Atlanta , or Raphael is in Tacoma, or both. Julia is in Seattle , or Paul is in Philadelphia, or both. What follows? Each exclusivedisjunction calls for only two models, whereaseach inclusive disjunction calls for three models. Likewise, when the premisesare combined, the exclusive problem yields three models: a st t p P

Here a is a representationof Julia inAtlantas is a representationof Julia in Seattle , t is a representation of Raphael in Tacoma, and p is a representation of Paul in Philadelphia. In contrast, the inclusive problem yields a total of five models: a st a st t t p P p p

In our first experiment, premisesof this sort were presentedeither verbally or else in the form of a diagram, such as figure 11.2. To represent , say, Julia in Atlanta , the " " " " diagram has a lozenge labeled Julia lying within the ellipse labeled Atlanta . Inclusive disjunction, as the figure shows , is representedby a box connectedby lines to the two component diagrams making up the premise as a whole. The experiment confirmed that exclusivedisjunctions were easierthan inclusive disjunctions (for both " the percentages of correct responses and their latencies ); it also confirmed that identical " problems, in which the individual common to both premiseswas in the same " " placein both of them, wereeasierthan contrastive problems such as the one above. But the experiment failed completely to detect any effect of diagrams: they yielded

to Think Space

457

Seattle
? follows What
. Figure 11.2 The diagrammatic presentation of double disjunctions in the first diagram experiment.

28% correct conclusions in comparison to the 30% correct for the verbal problems. Double disjunctions remained difficult , and thesediagrams were no help at all. With hindsight, the problem with the diagrams was that they used arbitrary symbols to representdisjunction and thus failed to make the alternative possibilities explicit . In a secondexperiment, we therefore used a new sort of diagram, as shown in figure 11.3, which is analogous to an electrical circuit . The idea, which we , was to complete a path from one side of the diagram to the explained to the subjects other by moving the shapescorresponding to people into the slots corresponding to cities. We tested four separategroups of subjectswith logically equivalent problems: one group receiveddiagrams of people and places(as in the figure); a secondgroup receivedproblems in the form of circuit diagrams of electrical switches; a third group ; and a received problems in the form of verbal premises about people and places electrical about verbal of form in the received premises fourth group problems switches. There was no effect of the content of the problems- whether they were about people or switches- and therefore we have pooled the results. The percentages , there was a of correct responsesare presentedin figure 11.4. As the figure shows to the correct 74% : of diagrammatic mode of responses presentation striking effect to the verbal problems. The problems in comparison to only 46% correct responses

458

-Laird PhilipN. Johnson

Raphael

I I Tacoma I I L -----------J ,r -------' ,, , i Atlanta i I I

,,Philadelphia ,,,, ,,,,,, " ,r -------' ,, , i i Seattle I I

Julia
The event is occurring .
What follows ? .3 Figure11 in the second . of doubledisjunctions Thediagrammatic diagramexperiment presentation results also corroborated the model theory' s predictions that exclusive disjunctions should be easier than inclusive disjunctions, and that identical problems should be ' had easierthan contrastive problems. The latenciesof the subjects correct responses , subjectswere faster to reason with exclusive exactly the same pattern, for example disjunctions than inclusive disjunctions, and they were reliably faster to respond to the diagrammatic problems (a mean of 99 seconds ) than to the verbal problems . a mean of 135 seconds ) ( People evidently reason by trying to construct models of the alternative possibilities , and diagrams that enable these alternatives to be made explicit can be very helpful . With a diagram of the sort we used in our second experiment, individuals ' perceivethe layout and in their mind s eyecan move people into placesand out again. By manipulating the model underlying the visual image, they can construct the alter-

Spaceto Think

459

IO8JJ

G )

. .

68lua

Diagram Verbal

Form of Disjunction
Figure 11.4 in the seconddiagram experiment. There are two sorts of The percentages of correct responses : disjunction : exclusive (exc.) and inclusive (inc.), and two sorts of relation between premises identical (ident.) and contrastive (con.) .

460

Philip N . Johnson- Laird

. It follows that nativepossibilities morereadilythan they can from verbalpremises in propositional to those arenot merely encoded representations equivalent diagrams constructed from verbalpremises , Pylyshyn1973 , and Palmer (but seeBaylor 1971 1975 for opposing views ).
11.8 Conclusions

Mental models are in many ways a primitive form of representation , which may owe their origin to the selective advantage of constructing internal representations of spatial representationsin the external world . The evidencereviewed in this chapter suggeststhat mental models underpin the spatial reasoning of logically untutored individuals and may also playa similar role in temporal reasoning . Indeed, it may be that human inference in general is founded on the ability to construct spatial, or quasi-spatial models, which also appear to playa significant part in syllogistic reasoning and reasoningwith multiple quantifiers (Johnson-Laird and Byrne 1991 ). Historians of scienceand scientiststhemselveshave often drawn attention to the role of diagrams in scientific thinking . Our studiesshow that not just any diagram has a helpful role to play. It is crucial that diagrams make the alternative possibilities explicit . Theories based on formal rules and propositional representationshave to postulate the extraction of logical form from an internal description of visual . In contrast, the model theory allows for inferencesbasedon visual perception percepts which has a mental model as its end product (Marr 1982 , ) . The two theories accordingly diverge on the matter of diagrams. Formal rule theories argue that performance with a diagram should be worse than with the logically equivalent verbal : with a diagram, reasonershave to construct an internal description from premises which they can extract a logical form . The model theory, however, predicts that performance with a diagram that makes the alternative possibilities explicit should be better than with logically equivalent verbal premises : with a diagram, reasoners . The do not need to engagein the processof parsing and compositional semantics evidence indeed suggeststhat human reasonersuse functionally spatial models to think about space , but they also appear to use such models in order to think in . general AckD Owledgments I am gratefulto Ruth Byrnefor hercollaboration in developing the theoryof deduction based . I am alsogratefulto her, to MalcolmBauer for on mentalmodels , and to Walter Schaeken . in The research was ideas and helpin carryingout thepresent supported part by experiments . theJames S. McDonnellFoundation

Spaceto Think Refereoces

461

. Behavioraland Brain Sciences Barwise , 16, , J. ( 1993 ) . Everyday reasoningand logical inference . 337- 338. Commentary on Johnson-Laird and Byrne 1991 Barwise , J. ( 1992 , J., and Etchemendy ) . Hyperproof : Logical reasoning with diagrams. In N . H . Narayanan ( Ed.), AAAI Spring Symposiumon Reasoningwith Diagrammatic Representations , 80- 84. 25- 27 March , Stanford University, Stanford, CA . . Psychological Bauer, M . I ., and Johnson- Laird , P. N . ( 1993 ) . How diagrams can improve reasoning Science , 4, 372- 378.

. FirstInternational task andprotocol ona mental , G. W. ( 1971 analysis imagery ). Programs Baylor . N. P. onArtificial JointConference Intelligence . Press . Berkeley : University of California Bull , andtheverb , tense , W. E. ( 1963 ). Time and . Journal Laird , P. N. ( 1989 , R. M. J., andJohnson ). Spatial reasoning of Memory Byrne - 575 . , 28 , 564 Language . : Cambridge Press . Cambridge nature Craik , K. ( 1943 University ). The of explanation . . Dordrecht : Reidel andMontague , D. R. ( 1979 ). Word grammar meaning Dowty . Chi chester : Ellis andtext models asrepresentations Garnham , A. (1987 ). Mental of discourse . Horwood .), Syntax and . In P. ColeandJ. L. Morgan Grice , H. P. ( 1975 (Eds ). Logicandconversation Press . . Vol. 3: Speech acts . NewYork: Seminar semantics of spatial in cognitive models : Experiments mental , G. ( 1984 modeling ). Modeling Hagert -Holland . : North .), Advances in artificial . In. T. O'Shea , Amsterdam (Ed intelligence reasoning : Representation .), Reasoning . In R. Falmagne of deduction Johnson Laird , P. N. (1975 (Ed ). Models . and . Hillsdale , NJ: Erlbaum process -Laird science models : Toward a cognitive Johnson , , inference , P. N. ( 1983 ). Mental of language : Cambridge University . Cambridge MA: Harvard Press andconsciousness , University ; Cambridge . Press - 209 -Laird . . Cognition models andprobabilistic Johnson , 189 , P. N. (1994 thinking ). Mental -Laird . . Hillsdale Johnson , NJ: Erlbaum , R. M. J. ( 1991 , P. N., andByrne ). Deduction -Laird : Thecase Johnson , P. (1989 , P. N., Byrne , R. M. J., andTabossi ). Reasoning bymodel - 673 . . Psychological Review of multiple , 96 , 658 quantification Press . , andwill. NewYork: Humanities , emotion , A. (1963 ). Action Kenny . Press . Cambridge : Cambridge skillsof argument Kuhn , D. ( 1991 University ). The . Cognitive XK >words is(sometimes Larkin ,< , H. (1987 ) worthIO , J., andSimon ). Whya diagram - 99 . Science , JJ, 65 . Press : Cambridge . Vols . I and2. Cambridge University , J. (1977 ). Semantics Lyons

462

Philip N . Johnson- Laird

andprocessing into thehuman : A computational Marr, D. ( 1982 representation investigation ). Vision . : Freeman . SanFrancisco information of visual -Laird, P. N. ( 1976 . Cambridge andperception , MA: Miller, G. A., and Johnson ). Language . HarvardUniversityPress in the Newstead , J. St. B. T. ( 1982 ). The role of imagery , K. I ., and Evans , S. E., Manktelow 2 21 32. Research . Current linear of , , Psychological orderings representation a, 57, 46- 67. . Acta Psychologic shiftsin spatialreasoning Ohlsson , S. ( 1984 ). Inducedstrategy . Cognition , 24 Osherson , , E. B. ( 1986 , E. E., and Shafir ). Some , D. N., Smith originsof belief - 224 . 197 : Noteson a modelof sensory and world knowledge Palmer , S. E. ( 1975 ). Visualperception LNR Research and the . E. Rumelhart . In D. ANormanD , Group cognitiveinteraction - 307 . : Freeman . SanFrancisco in cognition .), Explorations , 279 (Eds - 286 . . Linguistics andPhilosophy Partee , 7, 243 , B. ( 1984 ). Nominalandtemporalanaphora . In W. Perkins , J. ( 1983 reasoning ). Difficultiesin everyday , D. N., Allen, R., and Hafner . : Franklin InstitutePress . Philadelphia Maxwell(Ed.), Thinking . Press . Oxford: Clarendon Prior, A. N. ( 1967 , Present , andFuture ). Past ' ' . , Z. ( 1973 ). What the mind s eyetellsthe mind s brain: A critiqueof mentalimagery Pylyshyn Bulletin , 80, 1- 24. Psychological . : Routledge and KeganPaul . 3d ed. London , W. V. O. ( 1974 ). Methods of logic Quine . . NewYork: Springer Rescher , A. ( 1971 ). Temporal logic , N., and Urquhart . son Richard , J. T. E. ( 1987 ). The role of mentalimageryin modelsof transitiveinference . 78 189 203 BritishJournalof Psychology , , . : Hutchinson . London . 1949 The G. , ( of mind ) concept Ryle ' -Laird, P. N., andd Ydewalle andtemporal Schaeken , G. ( 1994 ). Mentalmodels , W., Johnson . in . , reasoningCognition press science . American andbehavioral of decision Simon , H. A. ( 1959 makingin economics ). Theories - 283 . Economic Review , 49, 253 . : Cambridge . Cambridge Toulmin, S. E. ( 1958 UniversityPress of argument ). Theuses and , D. ( 1973 , A., and Kahneman ). Availability: A heuristicfor judging frequency Tversky - 232 . . Cognitive , 5, 207 Psychology probability

Chapter12
Spatial Perspective in Descriptions Barbara Tversky

12 .1 CentralIssue in Perspective
When viewing an object or a scene , people necessarilyhave a specific perspectiveon it . Yet when thinking about or describing an object or scene , people can free themselves from their own perception and their own perspective . For example , when , people often describe their memory images as including themselves recollecting events . Or , (Nigro and Neisser 1983 ) rather than from the perspectiveof experience ' when describing a simple sceneto others, speakersoften take their addresses perspective rather than their own (Schober 1993 ) . Given the freedom to selecta perspective , what determinesthe perspectiveselected ? Spatial perspectivehas beena central issueto scholars with many interests , object environmental recognition, cognition, developmental psychology, neuropsychology , and language . Naturally , researchers in each area have their own concerns , and although someof theseare shared , they often work in blissful ignorance of each other. What accounts for the fascination of perspective , what is it that draws researchers with such diverseinterestsand methods to study it? Although people cannot help but experiencethe world from their own necessarilylimited point of view, taking other points of view is essentialfor a range of cognitive functions and social interactions, from recognizing an object from a novel point of view to navigating an environment 's to understanding someoneelse position . Emerging from the restrictions of the self seems at the basisof human thought and society. Not surprisingly, eachdiscipline has approached the problem of perspectivewith its own set of issues , developing its own set of distinctions. Before examining determinants of choice of perspectivein describing spaceand in comprehendingspatial descriptions, I will first survey views on perspectivein several diverse areas of cognitive science , most notably, object recognition, environmental , framing researchin the issuesrelevant to each discipline. cognition, and language The distinctions in perspectivemade by each of the disciplines contain instructive

464

Barbara Tversky

12.2

Some Perspectives on Perspective

12.2.1 Object Recognition Viewing a three-dimensional object revealsonly part of the object, yet recognizingan object can entail knowing what it looks like from other points of view. A critical issue in object recognition is the formation of mental representationsthat allow recognition of novel stimuli , both the sameobjects from different points of view and objects from the sameclass never before encountered . One question is the extent to which objects can be recognizedsolely on the basis of information from visual input , without drawing on information stored in memory, that is, from bottom -up information as opposed to top -down information (e.g., Marr 1982 ; Marr and Nishihara 1978 ). The visual input gives a viewer centeredrepresentationof an object, derived from the . It yields information projected on a viewer' s retina at the viewer' s current perspective some depth information but , without added assumptions , no information as to how an object would look from sidesnot currently in the field of vision. Because it is based on experienceviewing objects from many different points of view (see , for example , Tarr and Pinker 1989 ), and perhapson geometric principles that allow mental transformations (e.g., Shepard 1984 ), memory can provide an object-centeredrepresentation , a more abstract representation that yields information about how an object . In many cases would look from a different perspective , recognition of an object , an upside down or tilted object, seemsto depend currently under view, for example on mental comparison to an object in memory that is canonically oriented (e.g., Jolicoeur 1985 , ) . Whereasa viewer-centeredrepresentationhas a specificperspective an object-centeredrepresentationmight have a specificperspective , such as a canonical view, or it might have multiple representationseach with its own perspective , or -free, as in a structural description (Pinker 1984 it might be perspective , ) . In any case the distinction betweenthe viewer and the object viewed as basesfor perspectivehas beencritical to thinking about mental representationsof objects. 12.2.2 EnvironmentalCognition A similar issuearisesin the study of environmental cognition . In perceiving a scene , the viewer regardsit from a specificperspective more of scenes , yet generalknowledge from many perspectives is required for successful navigation. Environments are experienced from specificpoints of view along specificroutes. Yet people are able to make inferences , such as reversingroutes or constructing novel ones (see , for example spatial Landau 1988 Levine Jankovic and 1982 and Presson and , ; , , Palij ). Hazelrigg 1984

Spatial Perspectivein Descriptions

465

problem for developmentis similar to that of acquisition. How do children come to take perspectivesother than their own? Most accounts of mental representations of environments propose that as people move about an environment, they perceive the changing spatial relations of objects or landmarks to themselves , and use that information perhaps in concert with (implicit ) knowledge of geometry to construct more general mental representationsof the spatial relations among landmarks independent . As for object recognition, the initial perspective of a particular perspective . Later, people come to use what have is viewer-centered , often called egocentric beentermed a//ocentric referenceframes (e.g., Hart and Moore 1973 ; Pick and Lockman to a referencesystem are defined with reference frames 1981 . Allocentric respect ) -west. However south north external to the environment, usually the canonical axes , , east and turn out to external to a viewer , other objects, notably landmarks, are also be important in organizing environmental knowledge (e.g., Couclelis et al. 1987 ; Hirtle and Jonides 1985 ; Lynch 1960 ; Sadalla, Burroughs, and Staplin 1980 ) . In environmental cognition , then, the viewer and other objects in the sceneserve as . basesfor spatial referenceframes in addition to external or extrinsic bases 12.2.3 Neuropsychological Support Neuropsychological evidencefrom different sourcessupports the finding by environmental : the psychologists that there are three bases for spatial reference systems viewer, landmarks, and an external referenceframe. Perrett et al. ( 1990 ) have recorded of in the lobes movements to observed , finding monkeys temporal responses evidence for three basesfor referenceframes, namely, the viewer, the object being viewed, and the goal of the movement. In the terms of environmental cognition , both the latter categories , the object under view and the goal of the movement, can be regardedas landmarks. From recordings taken from the hippocampi of rats as they ' ' ; O Keefe, chapter 7, this volume) explore environments, O Keefe and Nadel ( 1978 and others have concluded that the hippocampus representsknown environments with respectto an external referenceframe. 12.2.4 Spatial Language ' not currently their own is revealedin their useof Peoples ability to take perspectives languagefrom perspectivesother than the perspectiveunder view as well as in their recognition of objects and navigation of environments. Accounts of spatial language : the viewer, have also found it useful to distinguish three basesfor spatial reference 1975 Fillmore other objects, and external sources(e.g., BUhler 1934 ; 1982 ; Levelt ; Miller and 4 this volume 1984 ; , 1989 , and chapter 3, this volume; Levinson, chapter , Johnson Laird 1976 ) . Thesethree basesat first seemto correspond to deictic, intrinsic , though it will turn out not to be that simple. , and extrinsic uses of language

466

Barbara Tversky

Before getting into the complexities, I will review deictic, intrinsic , and extrinsic uses . of language " " " " The term deictic derives from a Greek root meaning to show or to point . Deictic usescannot be accounted for by the languagealone, but require additional " knowledgeof certain details of the interactional situation in which the utterancesare " , 45), , 35) or , put differently by Levelt ( 1989 produced, according to Fillmore ( 1982 " an audio-visual scenewhich is more or less shared between the interlocutors, the placesand orientations of the interlocutors in this sceneat the moment of utterance, ." Severalkinds ofdeixis and the place of the utterance in the temporal flow of events have beendistinguished(see , 1982 ; Levelt 1989 , Fillmore 1975 , for example ), notably , " " " " person, place, and time, prototypically representedin languageby I , here, and " " " " " now." For , in person deixis, understanding the referentsof you and I example in a discoursedependson knowing who is speakingto whom. In place deixis, understanding " the usesof " this" and " that " or " here" and " there require knowing where . Miller and the participants in a discourse are, relative to the objects in a scene Johnson- Laird define place deixis as " the linguistic system for talking about space " 's relative to a speaker egocentric origin and coordinate axes (Miller and JohnsonLaird 1976 , 396) . It is place deixis that is of concern here. Deictic usescan be subtle, and there is not always agreementon them, as suggested by nuancesin the definitions above. quoted Someof the subtlety of deixis comesfrom the fact that many deictic terms can be " used nondeictically, especiallyintrinsically , such asfront and left . If I say, The tent " is in front of the boulder, I am using the term front deictically. The boulder has no front side, so I must mean that the tent is located between my front side and the boulder. In that case , you must know where I am located and how I am oriented with " to understand what I mean. In contrast, if I say, My pack is to the boulder respect in front of the tent," I can be using the termfront either deictically, as for the boulder, ' . Unlike a boulder, or intrinsically , that is, with respect to the object s natural sides but like a person, a tent has a natural front , back, top , and bottom , and a natural left . Thus, for the intrinsic use and right derived from the other sides , I mean that my , knowing where I am pack is located near the front side of the tent. In this case to understand what I mean. standing is unnecessary . Extrinsic uses of language rely on an external The extrinsic case is the clearest -west. If I say, the canonical directions, north -south, east such as referencesystem , " The tent is south of the boulder " I am . , using languageextrinsically If we just stop here, it seemsas though, in deictic cases , the basis for a reference frame is the viewer; in intrinsic cases , an external , an object; and in extrinsic cases referenceframe. Unfortunately , things are not that simple. For one thing , speakers " can refer to their own bodiesintrinsically . As Fillmore puts it , It should be clear that

Spatial Perspectivein Descriptions

467

it is also possible for the speakerof a sentenceto regard his own body as a physical ' '' ' ; expressionslike in front of me, behind me, or object with an orientation in space ' on ' my left side, are deictic by containing a first person pronoun but they are not " Fillmore 1975 6 . Continuing instancesof the deictic useof the orientational expressions , ) ( this line of reasoning 4 Levinson this volume shows that , (chapter , ) egocentric or viewer-basedusescrosscut intrinsic and extrinsic usesrather than contrasting with them. Fillmore ' s examples are simultaneously egocentric and intrinsic , as in " the boulder is in front of me." Speakers can also be simultaneouslyegocentricand extrinsic " the boulder is south of me." as in , Levinson suggests a different classification of spatial referenceframes in language use : relative, intrinsic , and absolute. To illustrate the distinctions, Levinson usesthe same spatial scenario for all three cases : a man is located in front of a house . The is the man whose location is described relative to the referent , target object object, the 's house whose location and orientation In are known. Levinson , , the intrinsic analysis and absolute (extrinsic) referenceframes are binary, that is, they require two terms to specify the location of the target object; the target object and the referent object. " " Speaking intrinsically , I can say, The man is in front of the house , meaning close ' to the house s intrinsic front . Speaking absolutely or extrinsically, I can say, " The man is north of the house ." The relative caseadds the location of a viewer, and uses three terms, that is, it requires a ternary relation. If I am a viewer located away from ' s left side " the house , looking at the man and the house , I can say, The man is to the " left of the house , that is, the man is left of the housewith respectto me, to my left , from my current location and orientation . The relative referenceframe is more complex because it requires knowing my location and orientation as well as the locations of the man and the house . According to Levinson' s analysis , what Levelt ( 1989 ) termed primary deixis is intrinsic , as when I say, " The tent is in front of me" , and what Levelt termed secondary deixis is relative, as when I say, " The tent is to the right of the boulder." 12.2.5 Basesfor Spatial Reference For a variety of reasons , some sharedand some unique, the analysis of spatial reference systemsand perspectivehas beencentral to severaldisciplines within cognitive science . Each of , notably, object recognition, environmental cognition , and language thesedisciplines has regarded the viewer as an important basis for spatial reference , ' . Most primarily because perception and cognition begin with the viewer s perspective have also regardedan object in the scene(or in the caseof language , the self, referred to as an object) and a referenceframe external or extrinsic to the sceneas important basesfor spatial referencesystems . They provide perspectives more generalthan that from a particular viewpoint at a particular time. The considerations leading to the

468

Barbara Tversky cognitive. I turn now categories of spatial

12 .3 SocialCategories
, occur in a social context; there is either a Spatial descriptions, like most discourse real addressee or an implicit one. Schober ( 1993 ) investigated the use of perspective . He developed a task that required participants to with real or implicit addressees . In one task, take a personal perspective , either their own, or that of their addressee pairs of subjectswho could talk to each other but not seeeach other had diagrams with two identical circles embeddedin a larger circle. The viewpoints of each of the ' subjectswere indicated outside the larger circle. On one subject s diagram, one of the ' smaller circles had an X. That subject s task was to describethe location of the X so that the other subjectcould put an X in the analogouscircle on the diagram. The task . allowed only personal perspectives , either that of the speakeror that of the addressee There were no other objects to anchor an intrinsic perspectiveand insufficient knowledge for an extrinsic one. Schober( 1993 ) found that , on the whole, speakerstook the . In a variation of the task, speakersexplained which perspectiveof the addressee circle had an X to an unknown addressee , in a situation that was not interactive. When there was no active participant to the discourse , speakerswere evenmore likely ' s task ' s . Thus what was of interest in Schober to take the addressee , perspective ' ' s, speakerswould adopt under what was whose perspective , speakers or addressee conditions. ' Although Schobers task did not allow it , another possibility is to use a neutral . A neutral perspectiveis one that is perspectiverather than a personal perspective ' ' neither the addressee s nor the speakers. Neutral perspectives include the possibilities raised earlier, namely, using a landmark , referent object, on the extrinsic systemas a . Mine , yours, or neutral are social categories basisfor spatial reference , , and language more than object recognition or navigation, is social.

12 .4 Determinants of Perspective Choice


Now I return to the determinants of perspectivechoice. After a brief review of previous , I will describeaspectsof three ongoing projects relevant to analysisand research the question. As Levinson (chapter 4, this volume) has pointed out , not every language usesall three systems ; thus some determinants are linguistic . BecauseEnglish uses all three systems the , question of determinants of perspective choice can be

Spatial Perspectivein Descriptions

469

. in English. The experts do not agreeon a dominant or default perspective addressed " Still it is a that the dominant : asserts 1989 52 velt For example , finding , ) general ,~ ( ." or default systemfor most speakersis deictic reference , either primary or secondary " In contrast, Miller and Johnson- Laird ( 1976 , 398 ) maintain: But intrinsic interpretations ; if a deictic interpretation is intended when an usually dominate deictic ones ' intrinsic interpretation is possible , the speaker will usually add explicitly from my ' " ' ' , the disagreeingexperts all point of view or as I am looking at it . As it happens seemto be correct, but in different situations. For extended discourse , in contrast to the single utterances that have often been . Many theoretiarise. One of theseis consistencyof perspective other issues , analyzed clans have assumed that speakers will adopt a consistent perspective , for several of the is a . Consistencyof perspective reasons assumption of consequence necessary a default perspective ; anyone arguing for a single default perspectivealso arguesfor is recognized . Even if the possibility of different perspectives , a consistentperspective to a discourse coherence discourse can within a of , provide consistency perspective at costs carries . , cognitive rendering it more comprehensible Switching perspective least for comprehension(e.g., Black, Turner , and Bower 1979 ). A second issueof interest for extendeddiscourseis determining the order of presenting . As Levelt ( 1982a information , independent of perspective , 1989 ) has observed world the linear. To describe is but speech , the world is multidimensional, linearly, it makes senseto choosea natural order. Becausea natural way of experiencing an environment is by moving through it , a natural way of conveying an environment is through a mental tour (Levelt 1982a , 1989 ). In classic study, Linde and . their in abound Mental tours spatial descriptions Labov ( 1975 ) found that New Yorkers usedtours to describetheir apartments. Similarly , respondentstook listenerson mental tours of simple networks (Levelt I 982a, b; Robin and Denis 1991 ), and of ), of the rooms where they lived (Ullmer - Ehrich 1982 no means are common . Tours Koster 1983 Ehrich and rooms dollhouse , , though by ) ( or tour a universal. For example , in describing locations in a complex network , path was only one of severalstylesadopted by subjects(Garrod and Anderson 1987 ) . And " " " on closer inspection, many of the room tours were gazetours rather than walking tours." Gaze tours are also natural ways of perceiving environments, from astationary view point rather than a changing one. The discourse of a gaze tour , however, differs markedly from that ofa walking tour (Ullmer -Ehrich 1982 ) . In a gazetour , the , noun phrasesare usually headedby objects and the verbs expressstates ; for example " In a " the the headed are the noun tour the table. is behind , by phrases walking lamp " addressee and the verbs expressactions; for example , you turn left at the end of the " corridor and seethe table on your right . Finally , the range of environments studied has beenlimited : single rooms, strings of rooms, and networks.

470

Barbara Tversky

12.4.1 Pragmatic Co. . ideratio. . Assertions about default and consistent perspectivesnonwithstanding, given that , it makes sense English and many other languageshave all three referencesystems that all three be used . Rather than there being a default perspective , choice of perspective is likely to be pragmatically determined. One pragmatic consideration is cognitive difficulty . Certain terms, like left and right, are more difficult for people than others, like up and down (see , for example , Clark 1973 ; Farrell 1979 ) . What is easieror harder can also depend on the number or degreeof mental transformation required to produce or comprehend an utterance. Some environments may lend themselvesto one perspectiveor another, so that describing them using a different perspectivemay increasedifficulty . It stands to reason that speakerswould avoid cognitively difficult tasks, all other things being equal. Another pragmatic consideration is the audience . Speakerstailor their speechto their addressees . In many cases , including the prototypic face-to -face conversation, the perspectiveof speakersand addressees differ. Because addressees have the harder of , speakersmay wish to easethe burden of addressees job comprehending by using ' the addressees perspectiverather than their own (Schober 1993 ) . Moreover, speakers presumablydesirethat their communications be understood and therefore attempt to construct their contributions to be as comprehensibleas possible , given the situation 's e. . Clark 1987 . the addressee should make communications ( g, ) Taking perspective ' more likely to be understood. Finally , using the addressee s perspective is polite (Brown and Levinson 1987 ). In other situations, speakersmay wish to avoid taking either their own or their 's addressee perspective and to adopt instead a perspective that is neutral, neither ' s not addressee ' s. Where there is some ' speaker controversy between the speakers ' s view a neutral view and the addressee , perspectivemay diffuse tension. Or more the interlocutors wish to avoid confusion over whose left and right . simply, may Whether the reasonsare social or cognitive, speakersmay use a neutral perspective , . Landmarks have the advantage using landmarks as referents or an extrinsic system of being visible in a scene , and an extrinsic systemhas the advantage of being more and . In the remainder of the chapter I will permanent independent of the scene discussthree examples drawn from current researchprojects, illustrating the effects , of pragmatic considerations on the selectionof perspectivein the comprehensionor production of spatial descriptions. A number of years ago, Nancy Franklin , Holly Taylor , and I began studying the nature of the spatial mental models engenderedby languagealone. We were stimulated ) and Johnson-Laird ( 1983 by the researchof Mani and Johnson- Laird ( 1982 ), demonstrating the useof mental models in solving verbal syllogism, and of Glenberg, , and Bower ( 1987 ; also Morrow , Meyer, and Lindem ( 1987 ) and Morrow , Greenspan

Spatial Perspectivein Descriptions

471

Bower, and Greenspan 1989 ), demonstrating effectsof distance on situation models constructed from text. Like Mani and Johnson- Laird , Franklin and I were interested in mental representationsand inference of spatial relations. Franklin and I , later joined by David Bryant , began with descriptions of the immediate environment surrounding a person (Franklin and Tversky 1990 ; Bryant , Tversky, and Franklin 1992 ). Like Perrig and Kintsch ( 1985 ), Taylor and I were interested in comprehensionand later production of longer discourses ; we therefore focusedon descriptions of larger environments(Taylor and Tversky 1992a , b) . Both projects brought us to the study of . Scott Mainwaring and Diane Schianojoined in a third project, investigating perspective ' perspectivein variations on Schobers paradigm (Mainwaring , Tversky, and Schiano 1995 ) . Let me describe those enterprisesin that order, beginning with the environments on immediately surrounding people. project
12.5

Environment of the Described Comprehension : Nature ~

As we turn in and move about the world , we seemto be able to keep track of the locations of objects around us without noticeable effort , updating their relative locations , even unseenlocations, with every step. Franklin and I wanted to simulate that , using language( Franklin and Tversky 1990 ) . We wrote a seriesof narratives, process " the " in various environments , some exotic like an opera subject, describing you, " In each setting, you" were surrounded by objects house , some mundane, like a barn. , to all six sidesof your body, from your , suchas a bouquet of flowers or a saddle head, feet, front , back, left , and right . After studying an environment, subjectsturned to a computer that repeatedly reoriented them to face one of the objects, and then , feet, right, and left, for the names probed them with direction terms,front , back, head of the objects in those directions. Subjectsperformed this task easily, almost without the objects in the six directions error , so the data of importance are the times to access from the body . A schematicof the situation appearsin figure 12.1. We considered three models for accessingobjects around the body . According to the equiavailability model , no area of spaceis privileged over any other area, much as in scanning a picture; this model predicts equal reaction times to all directions ) . However, a three-dimensional world surrounding ( Levine, Jankovic, and Palij 1982 a subject, even an imaginary one, is different from a picture all in front of a subject. For this case , objects directly in the imaginary field of view might have an advantage relative to objects at increasing angles from the imaginary field of view. The mental , , for example , inspired by the classic work in imagery (see transformation model to this into account. takes this and 1982 1980 According ; Shepard ), Cooper Kosslyn model, subjects imagine themselvesin the setting, facing frontward . When given a direction and askedto identify the associatedobject, they imagine themselves turning

472

Barbara Tversky

Figure 12.1 Schematicof situation where observer is upright and surrounded by objects.

to face that direction in order to access the object. In this case , times to front should be fastest , and times to back slowest , with times to head, feet, left , and right in between . The obtained pattern of data, displayed in table 12.1, contradicted both thesemodels, but supported a third model, the spatialframework model. The reaction times to access objects in the six directions from the body fit the third model, the spatial framework model. This model was inspired by analysesof Clark ( 1973 ), Fillmore ( 1982 ), Levelt ( 1984 ), Miller and Johnson-Laird ( 1976 ), and Shepard and Hurwitz ( 1984 ), but differs from eachof them. According to it , subjectsconstruct a mental spatial framework , consisting of extensions of the three body axes , and associateobjects to the appropriate direction. The mental framework preservesthe relative locations of the objects as the subject mentally turns to face a new object, to dependon the enduring allowing rapid updating . Accessibility of directions seems characteristics of the body and the perceptual world , rather than on the immediate , the head/ feet axis is most accessible imagery of the world . For an upright observer both becauseit is an asymmetric axis of the body and becauseit coincides with the axis of gravity , the only asymmetric axis of the world . The front / back axis is next becauseit is also an asymmetric body axis, and the left /right axis is least accessible , . The (upright) spatial framework pattern of reaction having no salient asymmetries times, head/ feet faster than front / back faster than left /right , was obtained in five experiments ( Franklin and Tversky 1990 ) and in several replications since (e.g., and 1991 Franklin 1992 , ; Bryant Tversky Bryant Tversky, and ). ' When the observer is describedas reclining in the scene . the observer is described as sometimeslying on front , sometimesback, sometimeseach side, so that no axis of the body coincides with gravity . Accessibility of objects, then, dependsprimarily on

: in Descriptions Spatial Perspective

473

12 .1 Table
Representative

Framework Experiments Mean Reaction Time from Spatial (ms) -

Head / feet
Upright internal internalb Reclining external Upright Two perspectives , differentscenesc
Two perspectives , same scenesd

Front 1 .55 1 .54

Back 1 .68 1 .49

Front / back 1 .62 1.82 1.52 3.99


3 .81

1.51 2.14 1 .30 3.50


3 .80

Left / right 1 .92 2.59 1.76 4.48


4 .05

: Sources 4. a. Bryant , experiment , and Franklin 1992 , Tversky 5. b. Franklin andTversky1990 , experiment 4. c. Franklin, Tversky , experiment , andCoon 1992
d. Franklin , Tversky, and Coon 1992 , experiment 3. for Franklin differed , Tversky, and Coon; times are therefore not comparable to Technique . studies previous

. The asymmetriesof the front / back body axis the relative salienceof the body axes are most salient becausethey separatethe world that can be easily sensedand easily manipulated from the world that is difficult to senseor manipulate. The head/ feet axis is next most salient, for its asymmetries , and the left /right axis is least salient. 12.1 the see table data of This pattern reclining spatial framework pattern, with ), ( than left /right , appeared in two experiments front / back faster than head/ feet faster replications (e.g., Bryant and Tversky ) and in subsequent (Franklin and Tversky 1990 . In this study and the previous ones 1992 Franklin , and 1991 ) , ; Bryant , Tversky ' " " narratives addressedthe subject as you, determining the subject s perspectiveas that of the observer , surrounded by a set of objects. 12.5.1 Central Third -PersonCharacter and Objects The spatial framework studiesdiscussedthus far serveas background for the studies . These studies also presentednarratives investigating perspectiveI will now describe , but subjectswere free to choosea perspective describing objects surrounding observers ; Franklin , among several possible ones (Bryant, Tversky, and Franklin 1992 used the narratives described studies . In the 1992 Coon and , previously ) Tversky, " to draw the reader into the sceneand induce the reader to take " secondperson you . Bryant, Franklin , of a central charactersurrounded by a set of objects the perspective and I (Bryant , Tversky, and Franklin 1992 ) wondered whether use of the secondperson for was necessary perspectivetaking , or whether readerswould take pronoun

474

Barbara Tversky

the perspectiveof an observerdescribedin the third person, or even take the perspective of an object. Because , according to literary lore, readers often identify with we readers to take the perspectivesof third -person observers protagonists, expected as long as the spatial probes were from that perspective . We also expectedreadersto take the perspectivesof objects when the spatial probes were from that perspective . Nevertheless , it was also possible that readerswould take the perspectiveof an outside observer . We altered the narratives so that in one experiment , looking onto the scene " " was a , you replaced by proper name, half male and half female, and in another experiment, " you" was replacedby a central object. The central objects were chosento have intrinsic sidesand were turned in the sceneby an outside force to face different objects. One example was a saddle in a barn, surrounded by appropriate , it would be natural for subjectsto take an external perspective objects. For both cases onto the character or object surrounded by objects rather than the , looking internal perspectiveof the central character or object. In order to distinguish which perspectivesubjectsadopted in thesenarratives, we first neededto know the reaction time patterns for external perspective . We knew the for internal that is, the upright spatial framework pattern obtained , pattern perspectives in previous studies. We developed two types of explicitly external narratives, -person observerlooking onto a scenewhere one where narratives describeda second a character was surrounded by objects to all six sidesof the character' s body and one -person observerlooking onto a cubic array of six where narratives describeda second . The spatial framework in objects. Figure 12.2 portrays both situations schematically this case is constructed from extensions of the three body axes in front of the observer , to the scene , but becausethe objects are located with respect to the central ' character and not the observer , the relative salienceof the observer s body axesis not ' relevant to accessibility . The characteristicsof the observers visual field are relevant to accessibility . The pattern predicted is similar to the upright internal spatial framework . Head/ feet should be fastest becauseof gravity , but for slightly different reasons . Front / back should be next fastestbecause of asymmetriesin the front / back visual field. In the caseof external arrays, all of the objects are in front of the observer , but thosedescribedas to the front (this is English, not Hausa; cf. Hill 1982 ) appear larger and closer and may occlude or partially occlude those to the back. The left/right visual field has no asymmetries . There is one , and thus is predicted to be slowest difference expected between internal and external spatial frameworks. Front is expected to be faster than back for the internal case becausethe objects to the back cannot be seen . The predicted patterns appeared , but not faster for the external case for the two external arrays as well as for the internal arrays (seetable 12.1) . Thus one important factor in determining perspectivein narrative comprehension is the perspective of the narrative. Subjectsadopted an external point of view when narratives

Spatial Perspectivein Descriptions

475

(A)

(B)

I I I I I ----~
liII

12 .2 F1a8re surrounded of external situations : (A ) An observer Schematic by lookingat a centralcharacter . . (B) An observer lookingat a cubicarrayof objects objects

~ d

476

Barbara Tversky

questioned them from that point of view, and an internal point of view when narratives questioned them from an internal point of view. The next step was to seewhat perspectivesubjectswould adopt when narratives allowed either option . With these findings in mind , we can return to the situation of a single central character or object surrounded by objects and describedin the third person. If readers take the internal perspectiveof the central character or object, then times to front should be faster than times to back. If they take the external perspectiveof someone , then times to front and back should not differ. In fact, times to observing the scene front were faster than times to back, suggestingthat readersspontaneouslyadopt the perspectiveof a central character or object, evenif the character or object is described in the third person. The patterns of time to charactersand objectsdiffered in one way. For objects, the terms head and feet are not as appropriate as the terms top and bottom, so the latter terms were used . Top, however, can refer both to the intrinsic top of an object and the top currently upward . The converse holds for bottom. For objects with intrinsic sides oriented in an upright manner, these usescoincide. For , the two usesof top (and bottom ) conflict , and, indeed, objects turned on their sides reaction times to judge what object was located away from the central object' s top and bottom were unusually long when objects were turned on their sides . In any case , readers readily take the perspectiveof either a character or an object central in a scene , even when the character or object is describedin the third person. 12.5.2 Two Perspectives in the SameNarrative The secondset of studiesinvestigatedperspectivetaking in narratives describing two different perspectives(Franklin , Tversky, and Coon 1992 ) . The question of interest was how subjects would handle the two perspectives . Would they switch between perspectivesdepending on which perspectivewas probed, or would they take a perspective that included both but was neither? There were several different kinds of narratives, describing two people in a scene , surrounded by the sameor different set of objects, or two people in two different scenes , surrounded by different sets of objects, or the sameperson in the samescene , surrounded by the sameset of objects, but facing different objects at different times. A schematicof some of the situations appearsin figure 12.3. Subjectscould adopt one of two strategiesfor the caseof two viewpoints. They could take eachperspectivein turn as eachwas probed. That would require perspectiveswitching. Alternatively , they could adopt a single perspective , one neutral in the senseof not being the perspectiveof any of the characters , but one that includes both viewpoints. An oblique perspective , for example , overhead or overhead could include both all the relevant characters , and objects nearly viewpoints, 's . If subjectstake each observer in turn , then the spatial framework viewpoint pattern of data should be evident. If they adopt a perspective that includes both

Spatial Perspectivein Descriptions

477

.3 Fig. e 12 Schematic of situationswith two viewpoints : (A ) Two observers surroundedby different . (B) Two observers objectsfacingdifferentdirectionsin samescene surrounded by different scene or differentscenes . , eitherin same objects viewpoints but is not equivalent to either, then some other pattern of reaction times . may emerge The two strategies seem to differ cognitively . To take each perspective in turn , subjectsneedto keep in mind a smaller set of tokens for charactersand objects, only those currently associatedwith that perspective . However, this would requiremen tally changing the viewpoint and mentally changing the set of tokens each time a new viewpoint is probed. To take a neutral perspectiveon the entire scenewould entail keeping more tokens in mind , but would not require mentally changing the set of tokens each time a new viewpoint is probed. The external spatial framework pattern

0 " 6 ' . , d

478

Barbara Tversky

would not be expected in this case because two characters and objects need to be kept in mind. This seemsto require taking an oblique viewpoint in which the bodies of the characters are not aligned with the body of the subject in the mental viewpoint . The two strategiesseemto trade off the size of the mental model with the need to , neither strategy was preferred switch mental models. Despite their cognitive differences on both used , depending the narrative. When narratives overall. Subjects strategies scene same in the , whether surrounded by the same described two observers , rather or different objects, subjects seemedto adopt a neutral oblique perspective . In this case , the data did not correspond than the viewpoints of either observer the to rather but framework equiavailability pattern, or to to the spatial pattern were times Either . equal to all directions or what we termed weak equiavailability times to right/left were a little slower. This pattern appeared even when one of the " " characters in the scenewas describedas you, and the other was described in the ) third person. This corroborates the finding of Bryant , Tversky, and Franklin ( 1992 the central whether not determine scene , perspective that qualities of the described character is described in the second or third person. When narratives described , subjects took the viewpoint of each observer in two observersin difference scenes turn . In this case , the spatial framework pattern of reaction times obtained (see table 12.1) . In both the caseswhere narratives described a central character or object in t ~e , third person and the caseswhere narratives described more than one perspective . When there were two readers appeared to adopt one perspective for each scene , readers adopted a observerseach with their own viewpoint but in the same scene were two observers there When . observers of the that neutral perspectiverather than . Thus in each scene the observer of in different scenes , readerstook the viewpoints . , determine perspective , the describedscene , in this case qualities of the scene to take a single perspective readers that it seems results the summarize prefer To , or character is a If there . ( object), readerswill adopt on a singledescribedscene single 's that character perspectivewhether or not that perspectiveis explicit in the description , readerswill . If there is more than one perspectiveexplicit in the describedscene . Would the same effects adopt a neutral perspective that includes the entire scene ? We would not expect described to as viewed are that , opposed appear for scenes their own. Without than other viewersof a scene to readily take any perspective . To simultaneously , viewerscannot easily get out of their own perspectives closing their eyes hold their own view as well as the view of another or a neutral view imposes , but not without an extra cognitive burden, one that people assume on occasion effort.

Spatial Perspectivein Descriptions 12.6 Production : Nature of the Environment to be Described

479

Perusinga shelfful of travel guidebooks revealstwo popular stylesof describing a city or other tourist attraction . A route description takes " you," the reader, on a mental tour ; it usesa changing view from within the environment, and locates landmarks with respectto you in terms of " your " front , back, left , and right . A surveydescription , in contrast, takes a static view from above the environment and locates landmarks with respect to each other in terms of north , south, east, and west. A route description usesan intrinsic perspective , where locations are describedin terms of the intrinsic sidesof " you." A surveydescription usesan extrinsic perspective . Thus, both are neutral because of the participants. perspectives they are not the perspectives As noted previously, Levelt ( 1989 a tour is a natural way ) has argued that because of experiencingan environment, a mental tour is a natural way of describing one. A survey, too , is a natural way to experience , hencedescribe , an environment. A survey view can be obtained by climbing a tree or a mountain. A survey is analogous to a map in many ways, and maps have beencreatedby cultures for millennia , evenbefore the advent of writing (see , for example , Wilford 1981 ) . Moreover, there is good evidencethat survey knowledge can be inferred from route experience(e.g., Landau 1988 ; Levine, Jankovic, and Palij 1982 ). In order to investigate the perspectivesthat people spontaneously use when describing environments, Taylor and I (Taylor and Tversky 1992a , 1996 ) gave subjects one of three maps to learn. The maps were of a small town , an amusementpark, and a convention center. The town and the convention center maps appear in figure 12.4. Each had about a dozen landll)arks. After learning the maps, subjectswere asked to describethem from memory. Importantly , all subjectstreated the maps as representing environments rather than as marks on paper; they described the environments, not the marks on paper (cf. Garrod and Anderson 1987 ) . In contrast to previous research , subjects used not only route but also survey perspectivesin their descriptions . Only one of the sixty-eight subjectsdid something different; that subject constructed a gaze tour from a stationary viewpoint . This is curious becauseit required X -ray vision. Also in contrast to previous research , subjects frequently mixed perspectives , nearly half of them, usually without signaling. For example , several subjects described the town by first describing the major landmarks, the mountains, river , and highways, in relation to the canonical directions. and then took readerson a tour of the park and the surrounding buildings. Often subjectscombined perspectives " " " , for example , You turn north or X is on your right , north of Y." The descriptions that subjectsproduced were accurateand complete. They allowed other subjectsto produce maps that had very few errors or omissions . By this measure the mixed , . perspectivedescriptions were as accurateas the pure ones

480 (A) Barbara Tversky

TOWN

White Mountei n ,

~ < f .,8
0 To ...nHelJ

----- --- -j~ ~ ~ ~ ; = = = = = ~ = ~ = = = = = I:


w

Spatial Perspectivein Descriptions

(B)

r s P . ; . t r Caf a Com I I 1 . . E ~ I R . s troo L . _ . _ ~ . _ Offic : ;J ~ ; ~


481

CENTER CONVENTION

's CD

. nts 5t . ,.. 0 Compon

's V R C

35mm Cam . ,-as

Movi. Cam . ras

. ' '0 c

.4 Figure11 , b). Used center the town (A ) andtheconvention of (B) from Taylor andTversky( 1992a Maps . with permission We initially categorizedthe descriptions as route, survey, or mixed on the basis of . Then we counted frequenciesof perspective intuitions and agreedbetweenourselves relevant uses of language for each perspective category. Route descriptions used active verbs such as go or turn most frequently, and survey descriptions used stative verbs such as is most frequently, with mixed descriptions in the middle. Survey descriptions also used motion verbs statively (seeTalmy , chapter 6, this volume); for " " to use example, the road runs east and west. Route descriptions were most likely viewer-centeredrelational terms, such asfront and left , and survey descriptions were most likely to useenvironment-centeredrelational terms, such as north and east, with . Route descriptions were most likely to use the viewer mixed descriptions in between

482

Barbara Tversky

as a referent for the location of landmarks, and survey descriptions were most likely to use landmarks as the referent for other landmarks, again with mixed descriptions in between . With respectto the referent for the location landmarks, route descriptions resembled that of Ullmer -Ehrich' s ( 1982 ) walking tour . Landmarks were describedrelative " to your " changing location , as in " if you turn left on Maple St., you will seethe School straight ahead." Similarly, the discourseof surveydescriptions resembledthat of Ullmer -Ehrich ' s gazetour . Landmarks were describedrelative to other landmarks, as in " The Town Hall is east of the Gazebo across Mountain Road," or " The lamp is behind the table." Because it is fixed and external to the scene , the viewpoint of a tour functions like the cardinal in directions a tour . Nevertheless , gaze gaze survey tours may be relative in Levinson' s sense(seechapter 4, this volume); for example , " The bookcaseis to the " right of the lamp is a ternary relation requiring knowledge ' s location and orientation . Gaze tours routes and of the speaker , , , then, are surveys to extended discourses to relative intrinsic and extrinsic , corresponding , , ways organize . , respectively perspectives Although languagewas usedquite differently in route and survey descriptions, the environments were organized similarly for both perspectives(Taylor and Tversky 1992a ) . A simple and widely used index of mental organization is the order of , for example , Tulving 1962 , the mentioning items in free recall (see ); in this case order of mentioning landmarks. The basic idea, an idea underlying association in memory, is that things that are related are rememberedtogether. The landmarks in the maps could be studied and learned in any order; thus the order of mentioning them is imposed by the subject, and presumably reflects the way the subject has organized them in memory. There was a high correlation acrosssubjectsin the order of mentioning landmarks irrespective of description perspective . Organization of and of to be . Organization description perspective description appeared independent was hierarchical, with natural starting points perceptually and/ or functionally determined . Environments were decomposedto regions by proximity , size , or function . Starting points were typically entrancesor large landmarks. Overall, approximately equal numbers of subjectsgave route, survey, and mixed descriptions, but the proportion of each was not the samefor each map. Perspective seemedto depend on the environment. For the town , there were very few pure route descriptions; the majority of descriptionswere evenly split betweenmixed and survey. For the convention center, there were very few pure survey descriptions, and the majority of descriptions were evenly split betweenmixed and route. For the amusement park, no dominant perspectivewas evident. Both the mixing of perspectives and the priority of organization over perspectivechoice are consistent with Levelt' s distinction between macroplanning and microplanning in speech( Levelt 1989 and

Spatial Perspectivein Descriptions

483

chapter 3, this volume) . Overall organization of the environment would be part of macroplanning, and perspectivechoice part of subsequentmicroplanning . The correlation of perspective with environment suggestedthat features of the . The convention center and town environment determine perspective in language differed in several ways. The convention center was relatively small and the town relatively large; the convention center was enclosedand the town open. In the convention center, the landmarks, in this case , the exhibition rooms, were on the same . In the town , the landmarks were on different sizescales sizescale , the mountains and the and river formed one scale buildings a third . , the roads and highways another, Finally , there was a single path for navigating the convention center, but severalways to navigate the town. In a subsequentstudy (Taylor and Tversky 1996 ), we created sixteen maps to was large or small, the environment whether counterbalance these four factors; whether the environment was closedor open, whether the landmarks were on a single size scale or several size scales , and whether there was a single or several paths the environment. Subjects studied four maps and wrote descriptions after through each. The descriptionswere coded as route, survey, or mixed as before. In contrast to the earlier study, where frequency of route, survey, and mixed descriptions were about equal, in this study, 22% of the descriptions were route, 36% were mixed, and 42% were survey. Neither the overall size of the environments nor whether the environments were enclosedor open- that is, neither global feature- had any effect on . Rather, it was the internal structure of the environments that description perspective affected the relative proportions of route and mixed perspectives(the proportion of , survey descriptions remainedconstant) . When landmarks were on a single size scale there were relatively more route and relatively fewer mixed perspectivedescriptions . When there was a single path than when the landmarks were on several size scales through the environment, there were relatively more route and relatively fewer mixed perspectivedescriptions than when there were multiple paths through the environment . Of course, it is simpler to plot a route among all the landmarks where there is ' one and only one. The apartments that Linde and Labov s ( 1975 ) subjectsdescribed typically had landmarks, that is, rooms, on a single size scaleand had a single path through the environment, and yielded primarily route descriptions. In extended discourse , people frequently switched perspectiverather than maintaining . Perhaps becausethe organization of the description a single perspective the choice of perspective , switching perspectivedid seemto reduce comprehensibility superseded of description. Choice of perspective , whether route, survey, or mixed, was affected by features of the environment. Both route and survey descriptions are analogous to natural ways of experiencing environments but seem appropriate to different situations. Route descriptions or mental tours were more likely when there

484

Barbara Tversky

was only a single way to navigate an environment and when an environment had a uniform size scaleof landmarks. Finally , gaze tours have been obtained for descriptions ) as well as for of single rooms (Ehrich and Koster 1983 ; Ullmer -Ehrich 1982 , b) . Gaze tours seemmore likely when the simple networks on a page (Levelt 1982a entire environment can be viewed from a single place.
12.7 Production : Cognitive and Social Determinants

The previous studies have investigated some of the cognitive factors affecting choice of perspective , and the nature of the environment. , the nature of the describedscene 1993 in Schober cited Hermann , social factors As Schober and ) have observed ( I have also affect perspectivechoice. To incorporate both , proposed another way of , first as to whether perspective is personal or neutral. categorizing perspective ' " " " " Personal perspectivecan be decomposedto yours or mine, that is, speakers or ' s. Neutral perspectivecan also be decomposed , to intrinsic or extrinsic. addressee To get greater clarity on determinants of perspectivein simple situations, Main ) have developed waring, Schiano, and I (Mainwaring , Tversky, and Schiano 1994 several variants of the paradigm of Schober ( 1993 ) described earlier. One of these will be described here. We constructed diagrams that were structurally similar to ' Schobers; in each case , there were two objects, identical except for location. The ' s task was to describe the location of the critical object. The situation is subject ' sketched in figure 12.5, though the actual diagrams were different. Schobers task ' forced subjectsto use a personal referencesystem , either the speakers or that of the

-....

D
D

Figure 12.5 are at right anglesand objects are aligned Schematicof situation where speakerand addressee . with speaker

Spatial Perspectivein Descriptions

485

addressee . This was the case for some of our diagrams, but for others, we added either a landmark or extrinsic directions, so that subjects had the option of using either a personal or a neutral referencesystemon many diagrams. The diagramsmanipulated the difficulty of the personal perspectives by varying the spatial relations between speaker and addresseeand between objects and participants . The speakerwas either facing the addressee . or at right anglesto the addressee 's The two objects were either lined up with the speaker so that from the , speaker point of view one was near and the other far , or positioned so that one object was to ' s left and the other to the ' the speaker speaker s right . When the speaker and the addressee were facing each other, then the type of relation , near/ far or left/right , was the same for both , but when the speakerand addressee were at right angles , then a near/ far relation for one was a right/ left relation for the other. In the first case , , but in the second case , where difficulty was the same for speaker and addressee were at right angles , what was easier for speakerwas harder speakerand addressee for addressee , and vice versa. Instead of communicating in pairs, subjectsgave descriptions for an unknown other. With only personal reference systems possible , 's Schober had found that speakerstended to take the addressee . The perspective ' frequency of taking the other s perspectiveincreasedwhen the other was unknown , rather than an active partner . We also added a cover story. You and the other were special agents in a secret . The diagrams representeddangerous missions that the two of you security agency undertook. Each diagram portrayed a scenein which the locations of you and your partner were indicated, as well as the locations of two identical objects, bombs, treasures , or the like. In each case , you knew which object was the critical one, and when your partner gave a signal, you describedthe critical object briefly and directly into your secretdecoder pad for your partner. The data I am reporting are preliminary; data collection is continuing . Someeffects ' s 1993 research we are already apparent. From Schober , ( ) expectedthat when only a '. was the would take the s. However addressee , , personal perspective possible speaker we expected cognitive difficulty to attenuate that tendency . Left /right distinctions are more difficult to produce and comprehend than near/ far distinctions. When the are at right anglesand the objects are lined up with the speaker speakerand addressee 's the needs to use left or right in order to take the addressee , speaker perspective (seefigure 12.5) . If speakersrealize this difficulty , they may choose to use their own perspectiveand the simpler terms closer or farther , sacrificing politeness to reduce , speakersdid exactly that , compared to 2% of difficulty . In fact, in 37% of the cases the caseswhere the objects were lined up with the addressee and the speakercould ' use closer or farther from the addressee s perspective(could reversethe positions of in figure 12.5) . speakerand addressee

486

Barbara Tversky

ofa neutral perspectiveto attenuate the tendencyof We also expectedthe presence ' . Selecting a neutral reference avoids the speakers to take addresseesperspectives entire issue of whose perspectiveto take. When subjects were told which direction was north , that is, when an extrinsic reference frame was available, they took a of a landmark also reduced personal perspectiveonly 56% of the time. The presence the frequency of taking a personal perspective , but to a lesserextent, to 64% of the time. An extrinsic system may be more likely to replace a personal system than a landmark becausean extrinsic system is more global and permanent than a landmark . This is supported by the finding that subjectswere more likely to describethe location redundantly, that is, to use both a personal and a neutral perspective , when the neutral perspectivewas a landmark than when the neutral option was the cardinal directions. Whether a landmark was useddependedon the difficulty of describing it ; ' here, difficulty translates into binary or ternary in Levinson s terms (seechapter 4) . Using a landmark was more frequent when the target object could be described as 's closer or farther to the landmark from the addressee , that is, used perspective left or when the had to be described as than , right of the target object intrinsically 's . that is used landmark from the addressee , , relatively perspective These results illustrate the complex interplay betweensocial and cognitive factors . When only a personal referencesystemwas available, there in selectinga perspective was a strong tendency , even stronger in a hypothetical rather than a real interaction 's . In the present perspective (Schober 1993 ), for the speaker to take the addressee 's data, that tendency was sometimesovercome when the addressee perspectivewas ' more difficult to produce and comprehend than the speakers. When a neutral perspective was available in addition to a personal perspective , there was a weak tendency 's for the speakerto take the addressee , perspective especiallywhen the neutral than a An extrinsic reference is more global was extrinsic rather landmark. , perspective and permanent than a landmark, a characteristic of the environment. Cognitive difficulty . When a landmark also affectedchoice betweena personal and a neutral perspective . was easierto describethan a personal reference , it was more likely to be used Note that thesedifferent choicesof referencesystemsappearedin the same subjects . Perspective was anything but communicating with the samehypothetical addressees consistent. We can infer from this that the cognitive cost of switching perspectivewas . often lessthan the cognitive cost of describing from certain perspectives
12.8 Summary and Conclusion

Many disciplines in cognitive sciencehave been intrigued with the issueof perspective . It is critical to theories of recognizingobjects and navigating environments, and the development of theseabilities; it has been of concern to neuropsychologistsand

Spatial Perspectivein Descriptions

487

, a survey of thesedisciplinesyielded three linguists. Despite many differencesin issues main basesfor spatial referencesystems : relative (viewer-centered , egocentric , personal based . intrinsic centered landmark and extrinsic external , ), (object ), ( ) Perspectivein languageuse is of particular interest becauselanguageallows us to use perspectivesother than those given by perception. Although there have been , researchon what people actually do many claims about perspectiveuse in language is just beginning. Someof that researchwas reviewedhere, along with more detailed descriptions of three current projects related to perspectivechoice. Severalconclusionsemergefrom the review of thesestudieson the comprehension and production of perspective in descriptions. First , there does not seem to be a . Different perspectives default perspective are adopted in different situations. Someof the influenceson perspectivechoice are cognitive and include the viewpoint of the to be described , and the description, the characteristicsof the describedsceneor scene relative difficulty of various perspectives . Second , perspectiveis not necessarilyconsistent . Peoplenot only spontaneouslyselectdifferent perspectives for different situations also switch often without or use more than one , they , perspectives signaling, . Third , perspective might be perspective redundantly, even in the same discourse better classifiedanother way, one with distinctions at two levels . The primary distinction would be betweenperspectivesthat are personal and perspectivesthat are neutral . Each of theseclasses subdividesinto two futher classes . Personalperspectives are in those of the participants the discourse ; they include yours and mine, that is, the ' ' s. Neutral do not belong to the participants speakers and the addressee perspectives in the discourse ; they include intrinsic or landmark-basedperspectivesand extrinsic . This classification draws attention to social influences on or external perspectives . Interestingly, many choice for , , attributions about the addressee perspective example of the relevant attributions about addressees are cognitive in nature, for example , what may be more or lessdifficult for an addressee to comprehend . Of necessity , individuals begin with their own perspectives , yet to function in the world , to recognizeobjects, to find one' s way in the world , to communicate to others, other perspectivesmust be known and used . Figuring out how we come to have perspectivesother than our own has attracted scholars from many disciplines. Yet . another reason researchers are drawn to the study of perspectiveis its social sense Individuals have different perspectives not on but on the events that take , , just space . They also have different perspectiveson beliefs, attitudes, and values. place in space For the endlessdiscussionspeople have on thesetopics, the mine-yours-neutral distinction is essential . Reconciling my memory or beliefs or attitudes or valuesto yours or not ) best be accomplished by moving from personal to neutral might ( might ground. Going beyond personal perspectiveis as critical to social interaction as it is to spatial cognition .

Barbara Tversky Acknowledgments I am indebted to my collaborators, Nancy Franklin , Holly Taylor , David Bryant , Scott Main , to Mary Petersonand Lynn Nadel waring, and Diane Schiano, for yearsof lively interchanges for valuable comments on an earlier draft , and to Eve Clark , Herb Clark , Pim Levelt, Steve Levinson, Eric Pederson , Michael Schober , and Pam Smul for ongoing discussionson deixis and perspective . Researchreviewed here was supported by the Air Force Office of Scientific Research , Air Force SystemsCommand, USAF , under grant or cooperative agreementnumber AFOSR 89-0076 to Stanford University , and by Interval ResearchCorporation . References

Black , J. B., Turner , T. J., andBower , G. H. ( 1979 , ). Point of viewin narrative comprehension - 198 . Journalof Verbal and Verbal . Behavior , andproduction , 18 , 187 memory Learning Brown : Some universals in language . Cambridge : , P., and Levinson , S. ( 1987 ). Politeness usage . Cambridge UniversityPress J., and Tversky . Paper . , B. ( 1991 ). Locatingobjectsfrom memoryor from sight BryantD at second Annual of the San Francisco , , presented Thirty Meeting Psychonomic Society November . , B., and Franklin, N. ( 1992 BryantD . J., Tversky ). Internalandexternalspatialframeworks - 98. for representing described scenes . Journalof Language andMemory , 31, 74 Biihler anddeicticwordsTranslated from theGerman , K. ( 1934 ). Thedeicticfieldof language in R. J. Jarvella andreprinted andW. Klein (Ed.), Speech , place , andaction , 9- 30. NewYork: . Wiley, 1982 Clark, H. H. ( 1973 , time, semantics , and the child. In TE . Moore (Ed.), Cognitive ). Space andtheacquisition Press . , 27- 63. NewYork: Academic development of language of language . In J. Vershueren Clark, H. H. ( 1987 use and M. Bertuccelli ). Four dimensions Eds . 9 : . , 25. AmsterdamBenjamins ), Thepragmatic Papi( perspective -point Couclelis , W. ( 1987 , H., Golledge , R. G., Gale , N. and Tobler ). Exploringthe anchor 7 99 122 . of . Journal Environmental , , of Psychology hypothesis spatialcognition V. . of Ehrich and Koster C. 1983 Discourse and sentence form: Thestructure , ( , , ) organization - 195 in Dutch. Discourse . roomdescriptions Process es , 6, 169 : Human Farrell , W. S. ( 1979 ). Coding left and right. Journalof Experimental Psychology andPerformance , 5, 42- 51. Perception Fillmore . Bloomington , In: Indiana University , C. ( 1975 ). Santa Cruz lectureson Deixis Club. Linguistics . In R. J. Jarvellaand Fillmore frameworkfor spatialdeixis , C. ( 1982 ). Towarda descriptive W. Klein (Eds .), Speech : Wiley. , place , andaction , 31- 59. London . Journalof Experimental Franklin, N., andTversky environments , B. ( 1990 ). Searching imagined : General , 119 , 63- 76. Psychology

Spatial Perspectivein Descriptions

489

Franklin , N ., Tversky, B., and Coon, V . ( 1992 ) . Switching points of view in spatial rnental rnodelsacquired frorn text. Memory and Cognition, 20, 507- 518. Garrod , S., and Anderson, S. ( 1987 ) . Saying what you rneanin dialogue: A study in conceptual - 218. and sernanticcoordination . Cognition, 27, 181 ) . Mental rnodels contribute to foregrounding Glenberg, A . M ., Meyer, M ., and Lindern, K . ( 1987 . Journal of Memory Language , 26, 69- 83. during text cornprehension Hart , R. A . and Moore , G. T . ( 1973 ) . The developrnentof spatial cognition . In R. M . Downs and D . Sten (Eds.), Image and environment , 246- 288. Chicago: Aldine . Hill , C. ( 1982 ) . Up/ down, front / back, left/right : A contrastive study of Hausa and English. In J. Weissenand W . Klein (Eds.), Here and there: Cross linguistic studieson deixis and demonstration . , 13- 42. Arnsterdarn: Benjarnins

of hierarchies in cognitivemaps Hirtle, S. C., and Jonides . Memoryand , J. ( 1985 ). Evidence 217 . 13 208 , , Cognition -Laird, P. N. ( 1983 Johnson . Cambridge . , MA: HarvardUniversityPress ). Mentalmodels Jolicoeur naturalobjects . MemoryandCognition , P. ( 1985 , 13 , ). The time to namedisoriented - 303 . 289 andmind . Cambridge . , S. M. ( 1980 , MA: HarvardUniversityPress ). Image Kosslyn Landau and useof spatialknowledge in blind and sighted children , B. ( 1988 ). The construction -Davis . In J. Stiles .), Spatialcognition : Brainbases , M. Kritchevsky , and U. Bellugi(Eds - 371 anddevelopment . Hillsdale . , 343 , NJ: Erlbaum Levelt in the useof spatialdirectionterms . In R. J. Jarvella , W. J. M. ( 1982a ). Cognitive styles - 268 andW. Klein (Eds .), Speech . Chichester : Wiley. , place , andaction , 251 . In S. Petersand Levelt , W. J. M. ( 1982b ). Linearizationin describingspatial networks - 220 E. Saarinen .), Process . Dordrecht es : Reidel . , beliefs , andquestions , 199 (Eds Levelt limitationson talking about space . In A. J. van , W. J. M. ( 1984 ). Someperceptual - 358 Doom, W. A. van der Grind, and J. J. Koenderink(Eds .), Limits on perception . , 323 Utrecht : VNU Science Press . Levelt : Fromintention to articulation . Cambridge . , W. J. M. ( 1989 , MA: MIT Press ). Speaking Levine of spatialproblemsolving . Journal , M., Jankovic , I . N., andPalij, M. ( 1982 ). Principles - 175 : General . , Ill , 157 of Experimental Psychology Linde as a site for the study of language and , C., and Labov , W. ( 1975 ). Spatialstructures - 939 . Language . , 51, 924 thought . : MIT Press , K. ( 1960 ). Theimage Lynch of thecity. Cambridge choicein spatialdescriptions , S. D., Tversky , B., and Schiano , D. ( 1996 ). Perspective Mainwaring . Technical . PaloAlto , CA: IntervalResearch Corp. report -Laird, P. N. ( 1982 Mani, K., andJohnson of spatialdescriptions . ). Thementalrepresentation and 10 181 187 . , , Memory Cognition

490 Marr , D . ( 1982 ) . Vision. New York : Freeman.

Barbara Tversky

Marr , D ., and Nishihara , H . K . ( 1978 ) . Representationand recognition of the spatial organization . Proceedings of three-dimensional shapes of the Royal Society, London, B200, 269- 291. . Cambridge, MA : Miller , G. A ., and Johnson- Laird , P. N . ( 1976 ). Languageand perception . Harvard University Press MorrowD . G., Bower, G. H ., and Greenspan , S. ( 1989 ) . Updating situation models during . Journal of Memory and Language narrative comprehension , 28, 292- 312. MorrowD . G., Greenspan , S., and Bower, G. H . ( 1987 ) . Accessibility and situation models in - 187 . . Journal of Memory and Language narrative comprehension , 26, 165 . Cognitive Psychology , , U . ( 1983 ) . Point of view in personal memories Nigro , G., and Neisser 15, 467- 482.

. Oxford: OxfordUniversity asa cognitive O' Keefe , L. ( 1978 , J., andNadel ). Thehippo campus map . Press in theclassifica Perrett , D., Harries , M., Mistlin, A. J., and Chitty, A. J. ( 1990 ). Threestages M. . In H. Barlow C. Blakemore and Weston visual neurons tion of body movements , , by . . Cambridge : Cambridge Press Smith(Eds .), Images andunderstanding , 94- 107 University of text. . and situational W. W. Kintsch 1985 and , , , ) Propositional representations ( Perrig - 518 . Journalof MemoryandLanguage , 24, 503 . to spatialrepresentations of reference Pick, H. L., Jr., andLockman , J. J. ( 1981 ). From frames andbehavior .), Spatialrepresentation In L. S. Liben , andN. Newcombe , A. H. Patterson (Eds - 60. NewYork: Academic . Press : Theory andapplication thelifespan across , 39 . Cognition : An introduction Pinker , 18 , 1- 63. , S. ( 1984 ). Visualcognition Presson , MD . ( 1984 , C. C., andHazelrigg ). Buildingspatialrepresentations throughprimary : Learning . and secondary Journal , Memory , andCognition Psychology learning of Experimental - 722 . , 10 , 716 . In or imagined of perceived Robin , M. ( 1991 , F., and Denis spatialnetworks ). Description - 152 : . Amsterdam in human .), Mental images R. H. Logie and M. Denis(Eds , 141 cognition North-Holland. Sadalla , L. J. ( 1980 , W. J., and Staplin ). Reference , E. K., Burroughs pointsin spatialcognition - 528 . : and Human . Journalof Experimental , 5, 516 Learning Memory Psychology . Cognition Schober , 47 , 1 24. , M. F. ( 1993 ). Spatialperspective takingin conversation kinemat : Resonant on internalrepresentations constraints , R. N. ( 1984 ). Ecological Shepard . . Psychological Review icsof perceiving , 91 , 417 447 , thinking , anddreaming , imaging . , L. A. ( 1982 , R. N., and Cooper ). Mental imagesand their transformations Shepard . , MA: MIT Press Cambridge , mentalrotation, and discrimination , R. N., and Hurwitz, S. ( 1984 ). Upwarddirection Shepard . . Cognition 18 161 193 of left andright turnsin maps , ,

Spatial Perspectivein

Descriptions

491

Tarr , M ., and PinkerS . ( 1989 in shaperecognition ) . Mental rotation and orientation dependence . Cognitive Psychology , 21, 233- 282. Taylor , H . A ., and Tversky, B. ( 1992a ) . Descriptions and depictions of environments. Memory and Cognition, 20, 483- 496. Taylor , H . A ., and Tversky, B. ( 1992b ) . Spatial mental models derived from survey and route . descriptions. Journal of Memory and Language , 31, 261- 292

in spatial descriptions. Journal of Memory Taylor, H. A., and Tversky , B. ( 1996 ). Perspective andLanguage , 35. " words in freerecallof " unrelated , E. ( 1962 . Psychological Tulving ). Subjective organization Review 69 344 354 . , , -Ehrich Ullmer . In R. J. Jarve , V. ( 1982 Ua and ). The structureof living spacedescriptions - 249 W. Klein (Eds .), Speech . NewYork: Wiley. , place , andaction , 219 Wilford, J. N. ( 1981 . New York: Knopf. ). Themapmakers

13 ~ . A
Chapter

Computational Analysis

of

the Apprehension

of Spatial
Relations

GordonD . Loganand DanielD . Sadier

13 .1 Introduction
and cognitive Spatial relations are important in many areas of cognitive science and psychology. Each neuroscience , including linguistics, philosophy, anthropology , area has contributed substantially to our understanding of spatial relations over the last couple of decades , as is evident in the other chaptersin this volume. The psychol ' , a concern ogists contribution is a concern for how spatial relations are apprehended 's individual an esunderlying apprehension for the interaction of representationsand process of spatial relations. This chapter presents a computational analysis of the es involved in apprehending spatial relations and interprets representationsand process . The chapter beginswith this analysisas a psychologicaltheory of apprehension of the theory and with some a theory and ends with data that test the assumptions . commentsabout generality.

.. of SpatialRelatio 13 .2 ThreeOasses in termsof the representations for a phenomenon A computational theoryaccounts on the representations esoperate the how it underlie that es process andprocess , specifying of the representations nature the to clues . behavior to producethe observed Important can be relations of es involved in the apprehension spatial and process of semantics the es literaturethat address found in the linguisticand psycholinguistic and ; Jackendoff ; Herskovits1986 ; Gamham 1989 spatialrelations(e.g., Clark 1973 Vanda and 1983 1976 Laird ; Johnson and Miller ; Talmy 1984 Landau1991 ; ; Levelt of spatial relations , threeclasses es between loise 1991 ). That literaturedistinguish the that distinguishthe classes suggest requisiterepresentations and the discriminanda es . and process

494

andD. D. Sadier G. D. Logan

13.2.1 Basic Relatio. . Gamharn ( 1989 ) distinguished basic relations from deictic and intrinsic ones. Basic relations take one argument, expressingthe position of one object with respectto the viewer (e.g., the viewer thinks , " This is here" and " That is there" ) . 1 Basic relations are essentially the same as spatial indices , which are discussedin the literature on human and computer vision (e.g., Pylyshyn 1984 , 1989 ; Ullman 1984 ) . Spatial indices establish correspondence between perceptual objects and symbols, providing the viewer' s cognitive systemwith a way to access perceptual information about an object . Spatial indices- basic relations- individuate objects without necessarilyidentifying , recognizing, or categorizing them. The conceptual part of a basic relation is " a symbol or a token that stands for a perceptual object. It simply says , Something " " " is there, without saying what the something is. The token may be associatedwith an identity or a categorization, pending the results of further processing , but it need not be identified, recognized , or categorizedin order to be associatedwith a perceptual object. The perceptual part of a basic relation is an object that occupiesa specific . point or region in perceptual space Basic relations representspacein that they associatea conceptual token with the . Conceptually, the representationof spaceis object in a location in perceptual space " here" and " not there." Thus two crude an is very object objects that are indexed in can either be the same location or in different locations. If they are in separately different locations, their relative positions are not representedexplicitly in the conceptual . Information about their relative locations may be available representation in , but it is not made explicit in basic relations. Other implicitly perceptual space relations and other computational machinery are necessary to make relative position explicit .

13 .2.2 Deictic Relado .


) was the first to distinguish basic relations, most linguists Although Gamham ( 1989 and psycholinguists distinguish betweendeictic and intrinsic relations (e.g., Herskovits 1986 ; Miller and Johnson-Laird 1976 ; ; Jackendoffand Landau 1991 ; Levelt 1984 . take two or more as 1983 and Vandaloise 1991 Deictic relations ; ) objects Talmy , specifying the position of one object, the located object, in terms of the arguments other (s), the referenceobjects ) . The position is specifiedwith respectto the reference frame of the viewer, which is projected onto the referenceobject. Deictic relations specify the position of the located object with respectto the viewer if the viewer were to move to the position of the referenceobject. Thus " The ball is left of the tree" means that if the viewer were to walk to the tree, the ball would be on his or her left side.

A Computational Analysis

495

Deictic relations are more complex computationally than basic relations because they relate objects to each other and not simply to the viewer. They represent the relative positions of objects explicitly . The arguments of deictic relations must be . Individuation , or categorized individuated but they need not be identified, recognized is necessarybecause the reference object is conceptually different from the " " " located object (i.e., " X is above Y and Y is above X mean different things), but and located objectscan be made by simply establishing the distinction betweenreference tokens that representperceptual objects, leaving identification , recognition, and es. categorization to subsequentprocess 13.2.3 Intri . ic Relado. Like deictic relations, intrinsic relations take two or more arguments and specify the position of a located object with respectto a referenceobject. They differ from deictic relations in that the position is specifiedwith respectto a referenceframe intrinsic to ' the referenceobject rather than the viewer s referenceframe projected onto the reference object. Whereas deictic relations can apply to any reference object, intrinsic relations require referenceobjects that have intrinsic referenceframes, that is, intrinsic . Objects like people, tops and bottoms, fronts and backs, and left and right sides becausethey intrinsic relations for reference serve as can and cars houses objects , . Objects like balls cannot have fronts , backs, tops, bottoms, and left and right sides serveas referenceobjects for intrinsic relations becausethey have no intrinsic tops, bottoms, and so on. Objects like trees have tops and bottoms but no fronts and backs or left and right sides , so they can support intrinsic aboveand below relations but not intrinsic in front of or left of relations; in front of and left of would have to be specifieddeictically. Objectslike bullets and arrows have intrinsic fronts and backs . They can support intrinsic but no intrinsic tops and bottoms or left and right sides in front of and behind relations, but above and left of would have to be specified deictically. Intrinsic relations are more complex computationally than deictic relations because they require the viewer to extract the referenceframe from the referenceobject. An obvious way to extract the referenceframe is to recognize the referenceobject or classify it as a member of somecategory and to impose the referenceframe appropriate to that category. For example , seeingan ambiguous figure as a duck or a rabbit leadsthe viewer to assign ). front to different regions of the object (Petersonet al. 1992 However, it may be possible in some casesto assign an intrinsic reference frame without actually identifying the object. The main axis of the referenceframe may be ' ; Marr and Nishihara aligned with the object s axis of elongation ( Biederman 1987 ' 1989 Palmer 1987 Biederman ; 1978 ). ) or with the object s axis of symmetry (

496

G. D. Logan andD. D. Sadier

13.2.4 Implicatio18 for Computation The distinction betweenthe three classes of spatial relations has at least two implications for a theory of the computation involved in apprehension . First , each class of relations describesthe position of the located object in terms of a referenceframe. The referenceframe may coincide with the viewer' s, as in basic relations, it may be projected onto the referenceobject, as in deictic relations, or it may be extracted from the asymmetriesinherent in the referenceobject, as in intrinsic relations. In eachcase , the reference frame is a central part of the meaning of the spatial relation , and this suggeststhat reference frame computation is a . central part of the process of . apprehension Second that , the distinction betweenreferenceobjects and located objects suggests the arguments of two- or three-place relations must be individuated somehow . " X is above Y " doesnot mean the sameas " Yis above X ." The processof spatial indexing - instantiating basic relations- is well suited for this purpose. Each object can be represented by a different token, and the tokens can be associatedwith the arguments that correspond to the located and referenceobject in the conceptual representation of the relation. The distinction betweenlocated and referenceobjects is also important in referenceframe computation because the referenceframe is projected onto or extracted from the referenceobject, not the located object. Spatial indexing is useful here as well. It is a central part of apprehension .

13 .3 SpatialTemplates asRegio . - of Acceptability


Referenceframes and the distinction between located and referenceobjects suggest , but something is missing important parts of a computational theory of apprehension . They do not specify how one would decide whether a given spatial relation applied to a pair or triplet of objects. This issuehas beendiscussed extensivelyin the have suggested linguistic and psycholinguistic literature. Various researchers computations ; Miller and Johnson-Laird 1976 involving geometric (Clark 1973 ), volumetric ; Talmy 1983 (Herskovits 1986 ; Talmy ), topological (Miller and Johnson- Laird 1976 and functional 1983 Herskovits 1986 ), ; Vandaloise 1991 ( ) relations. We propose that people decide whether a relation applies by fitting a spatial template to the objects that representsregionsof acceptability for the relation in question (seealso Carlson; Hayward and Tarr 1995 ; Kosslyn et al. 1992 ; Logan Radvansky and Irwin 1993 1994 1995 and 1996 . , ; Logan ) Compton A spatial template is a representationthat is centeredon the referenceobject and aligned with the referenceframe imposed on or extracted from the referenceobject. It is a two - or three-dimensional field representingthe degreeto which objectsappearing in each point in spaceare acceptableexamplesof the relation in question. The

A Computational Analysis

497

main idea is that pairs or triplets of objects vary in the degree to which they instantiate spatial relations. Roughly speaking , there are three main regions of acceptability : one reflecting good examples , one reflecting examplesthat are less than . Good one and but nevertheless , reflecting unacceptableexamples acceptable good Instead them. between border with a , not distinct are and acceptableregions sharp for above , any , example they blend into one another gradually . With the relation object that is aligned with the upward projection of the up- down axis of the reference object is a good example. Any object above a horizontal plane aligned with the top of the referenceobject is an acceptableexample , although not a good one (the closer axis down it is to the upward projection of the up, the better) . And any object below a horizontal plane aligned with the bottom of the referenceobject is a bad, unacceptable example. We propose that people use spatial templates to determine whether a spatial relation applies to a pair of objects. If the located object falls in a good or an acceptable region when the template is centered on the referenceobject, then the relation can apply to the pair . If two relations can apply to the samepair of objects, the preferred relation is the one whosespatial template fits best. If both spatial relations fit reason " " ably, the viewer may assert both relations (e.g., above and to the right ). Spatial about templatesprovide information about goodnessof fit . Exactly how information ' ' goodnessof fit is useddependson the viewer s goals and the viewer s task (seebelow) .

.4 Computational 13 Theoryof Apprebe18ion


At this point the representations and processes necessary to apprehend spatial relations have been described in various ways , some in detail , some briefly , and some only implicitly . Now it is time to describe them explicitly and say how they work together .

13.4.1 Representations that the apprehensionof spatial relations dependson four different The theory assumes : a perceptual representation consisting of objects and kinds of representations , a reference surfaces , a conceptual representation consisting of spatial predicates frame, and a spatial template. It may be more accurate to say there are two kinds of " " , one pef. teptual and one conceptual, and two intermediate representations representation that map perception onto cognition and vice versa. 13.4.1.1 PerceptualRepresentation The perceptual representation is a two -, two . It is formed and-a-half-, or three-dimensional analog array of objects and surfaces of opening an es as local consequence obligatory parallel process automatically by ' 1984 and Ullman 1984 1982 Marr for ) . The ; one s eyes (see ; Pylyshyn , , example

498

G. D . Logan and D . D . Sadier

representationcontains infonnation about the identities of the objects and the spatial relations betweenthem, but that infonnation is only implicit . Further computation is to make it explicit . In other words, the representationcontains the perceptual necessary infonnation required to identify the objects or to compute spatial relations between them, but that infonnation does not result in an explicit identification of the object as an instance of a particular category or specific relation without further " " computation . That further computation is what the other representationsand pro cesses are required for . The current version of the theory assumesthat the perceptual representation is . We make that assumption because relatively low -level, and that neednot be the case it is relatively clear how low-level representationscan be constructed from light impinging on the retina (e.g., Biedennan 1987 ; Marr 1982 ), and we want the theory to be tractable computationally . However, the spirit of the theory would not be very different if we assumedthat the perceptual representation was much more abstract; for example , if we assumedthat spatial infonnation was representedamodally, combining visual, auditory , tactual, and imaginal infonnation . The key idea is that the perceptual representation provides an analog array of objects that can be compared to a spatial template. In principle, the objects can be highly interpreted and abstracted from the sensorysystemsthat gave rise to them. 13.4.1.2 ConceptualRepresentation The conceptual representationis a one-, two-, or three-place predicate that express es a spatial relation. The conceptual representation identifies the relation (e.g., it distinguishes abovefrom below ); it individuates the argumentsof the relation , distinguishing betweenthe referenceobject and the located object; it identifies the relevant reference frame (depending on the nature of the referenceobject); and it identifies the relevant spatial template. The conceptual representation doesnot identify objects and relations directly in the perceptual representation ; further processingand other representationsare neededfor that. An important feature of the conceptual representation is that it is addressableby . The mapping of conceptual representationsonto languagemay be direct in language somecases and indirect in others. In English, French, Dutch , and German, for example , many conceptual (spatial) relations are lexicalized as spatial prepositions; single words represent single relations. However, there is polysemy even in the class of , distinguished severaldifferent senses ), for example spatial prepositions. Lakoff ( 1987 of over. Moreover, some languagesmay use a single word to refer to different relations that are distinguished lexically in other languages . For example , English uses one word for three senses of on that are distinguished in Dutch (i.e., om, op, and aan; seeBowerman, chapter 10, this volume) . Despite thesecomplexities, we assumethat

A Computational Analysis

499

conceptual representationsmay be mapped onto languageand vice versa. The mapping may not always be simple, but it is possiblein principle (seealso Jackendoff and Landau 1991 ; Landau and Jackendoff 1993 ). 13.4.1.3 ReferenceFrame The referenceframe is a three-dimensional coordinate . It servesas a map system that defines an origin , orientation , direction , and scale betweenthe conceptual representation and the perceptual representation , establishing The between reference and located between them. distinction correspondence ' ; the viewer s attention objects gives a direction to the conceptual representation should movefrom the referenceobject to the located object (Logan 1995 ) . The reference frame gives direction to perceptual space , defining up, down, right , front , and . back. It orients the viewer in perceptual space . The different parameters We assumethat referenceframesare flexible representations ' can be set at will , depending on the viewer s intentions and the nature of the objects on which the reference frame is imposed. Many investigators distinguish different kinds of referenceframes- viewer-based , , object-based , environment-based and Irwin 1993 1994 deictic, and intrinsic (Carlson Radvansky , ; Leve1t 1984 ; Marr 1982 ; Marr and Nishihara 1978 ) . We assumethat the samerepresentation underlies all of these different referenceframes (i.e., a three-dimensional , four -parameter coordinate . Viewer) . The differencesbetweenthem lie in the parameter settings system " " based and object-based referenceframes (also known as " deictic and " intrinsic referenceframes ) differ in origin (the viewer vs. the object), orientation (major axis of ' ' " " viewer vs. major axis of object), direction (viewer s head up vs. object s head up), ' ' and scale(viewer s vs. object s) . 13.4.1.4 Spatial Template As we just said, the spatial template is a representation of the regions of acceptability associatedwith a given relation. When the spatial template is centered on the referenceobject and aligned with its referenceframe, it specifiesthe goodnesswith which located objects in different positions exemplify the associatedrelation. We assumethat different relations have different spatial templatesassociatedwith . More specifically them and that similar relations have similar templates , we assume that spatial templates are associatedwith conceptual representationsof spatial relations . Consequently , but the addressingis mediated , they are addressable by language . We assumethere are spatial by linguistic accessto the conceptual representation but in casesof polysemy where for lexicalized , conceptual representations templates there is more than one conceptual representation associated with a given word ), there is a different spatial template for each conceptual (e.g., over; Lakoff 1987

500

G. D. Logan and D . D . Sadier

. Moreover, we assumethat spatial templatescan be combined to represent representation " " compound relations (e.g., above right ) and decomposedto representfiner " " distinctions (e.g., directly above ). 13.4.2 Proceaes that the apprehensionof spatial relations dependson four different The theory assumes : spatial indexing, referenceframe adjustment, spatial template kinds of process es alignment, and computing goodnessof fit . The first two establish correspondence between perceptual and conceptual representations ; the last two establish the relevance or the validity of the relation in question. 13.4.2.1 Spatial Indexing Spatial indexing is required to bind the argumentsof the . relation in the conceptual representationto objects in the perceptual representation Spatial indexing amounts to establishing correspondencebetween a symbol and a " " percept. A perceptual object is marked in the perceptual representation (Ullman 1984 ), and a symbol or a token corresponding to it is set up in the conceptual representation , 1989 (Pylyshyn 1984 ) . The correspondencebetween them allows conceptual the perceptual representationof the object so that perceptual esto access process information about other aspectsof the object can be evaluated (e.g., its identity) . , the viewer assertstwo or three basic relations, one for the located object Essentially and one or two for the referenceobjects. 13.4.2.2 ReferenceFrame Adjustment The relevant reference frame must be imposed esinvolved translate the on or extracted from the referenceobject. The process relevant orientation , choose a origin of the referenceframe, rotate its axes to the . Not all of these adjustments are required for every direction , and choose a scale relation. Near requiressetting the origin and the scale , whereasaboverequires setting direction. orientation and , origin , . The origin es may be involved in setting the different parameters Different process ) or by a processanalogous to mental may be set by spatial indexing (Ullman 1984 curve tracing (Jolicoeur, Ullman , and MacKay 1986 , 1991 ) . Orientation may be set and to mental rotation a ; Corballis (Cooper Shepard 1973 by process analogous . reference frames or different 1988 parameter settings may compete with ) Different each other, and the adjustment process must resolve the competition (Carlson). Radvansky and Irwin 1994

on the spatial

A Computational Analysis

501

' template is aligned with the viewer s referenceframe projected onto the reference object. In intrinsic relations, it is aligned with the intrinsic referenceframe extracted from the object. of Fit Once the relevant spatial template is aligned 13.4.2.4 Computing Goodness with the referenceobject, goodnessof fit can be computed. The position occupied by the located object is compared with the template to determine whether it falls in a , or bad region. We assumethat the comparison is done in parallel good, acceptable compuover the whole visual (or imaginal) field. Spatial templatescan be represented tationally as a matrix of weights, and the activation value of each object in the goodness visual-imaginal field can be multiplied by the weights in its region to assess of fit . Weights in the good region can be set to 1.0; weights in the bad region can be set to 0.0, and weights in acceptablebut not good regionscan be set to valuesbetween , the less the activation 0.0 and 1.0. With these assumptions , the better the example changeswhen the spatial template is applied. The activation of good exampleswill not changeat all ; the activation of bad exampleswill vanish (to 0.0); and the activation of acceptableexampleswill be somewhatdiminished. Alternatively , weights for bad regions could be set to 1.0, weights for acceptable regions could be greater than 1.0, and weights for the good region could be well , the greater the changein above 1.0. With theseassumptions , the better the example of bad exampleswill activation . The is when the activation spatial template applied will not but changea little ; and not change good examples ; the weights of acceptable case , the acceptability the weights of good exampleswill changesubstantially. In either esand other and rank ordered. Other process of candidate objects can be assessed . considerationscan chooseamong the candidates 13.4.3 Programsand Routines Spatial relations are apprehendedfor different reasonsin different contexts. Sorne tirnes apprehensionitself is the rnain purpose, as when we want to determine which horse is aheadof which at the finish line. Other tirnes, apprehensionis subordinate to other goals, as when we want to look behind the horse that finished first to seewho . A cornputational analysis of apprehensionshould account for this finished second esdescribedabove flexibility . To this end, we interpret the representationsand process in different orders to executed and in different be can as elernentsthat ways arranged set in a prograrninstruction the and fulfill different purposes , like the data structures es are interpreted . Ordered cornbinations of representations and process ming language as programs or routines (cf. Ullrnan 1984 ) . In this section, we consider three . different serve that routines purposes

502

G. D. Logan andD. D. Sadier

13.4.3.1 Relation Judgments Apprehension is the main purpose of relation judgments . A viewer who is asked , " Where is Gordon ?" or " Where is Gordon with " respect to Jane? is expected to report the relation between Gordon and a reference object . In the first case, the reference object is not given . The viewer must ( I ) find the located object ( Gordon ); ( 2) find a suitable reference object ( i .e., one the questioner knows about or can find easily); ( 3) impose a reference frame on the reference object ; (4) choose a relation whose region of acceptability best represents the position of the located object ; and ( 5) produce an answer (e.g ., " Gordon is in front of the statue " ) . In the second case, the reference object is given ( i .e., Jane) . The viewer must ( I ) find the reference object ; (2) impose a reference frame on it ; ( 3) find the located object ( i .e., Gordon ); (4) choose a relation whose region of acceptability best represents the " " position of the located object ; and ( 5) produce an answer (e.g ., on her left side. ) . We assume that viewers find located objects by spatially indexing objects in the perceptual representation and comparing them to a description of the specified located object (e.g ., " Does that look like Gordon ?" ) . When reference objects are specified in advance , we assume they are found in the same manner . If they are not specified in advance , as in the first case, then the most prominent objects are considered as reasonable candidates for reference objects ( Clark and Chase 1974; Talmy 1983) . The relation itself is chosen by iterating through a set of candidate relations imposing the associated spatial templates on the reference object , aligning them with the reference frame , and computing goodness of fit - until one with the best fit or one with an acceptable fit is found . Relation judgments have been studied often in the psychological literature . Subjects are told in advance what the arguments of the relation will be, but they are not told the relation between them . Their task is to find the arguments , figure out the relation between them , and report it . Thus Logan and Zbrodoff ( 1979) had subjects report whether a word appeared above or below the fixation point ; Logan ( 1980) had subjects decide whether an asterisk appeared above or below a word . A common focus in relation judgments is Stroop - like interference from irrelevant spatial information (e.g ., the identity of the word in the first case; the position occupied by the word asterisk pair in the second) .

13.4.3.2 Cuing Tasks In cuing tasks, apprehensionis usedin the serviceof another " " goal. A viewer who is asked, Who is besideMary ? must ( I ) find the referenceobject (i.e., Mary ); (2) impose referenceframe on it ; (3) align the relevant spatial template with the referenceframe (i.e., the one for beside ); (4) chooseas the located object the perceptual object that is the best example (or the first acceptableexample ) of the relation ; and (5) produce an answer (e.g., " Paul" ) .

A Computational Analysis

503

Cuing tasks have been studied extensivelyin the psychological literature. Experiments on visual spatial attention require subjects to report a target that stands in a prespecified relation to a cue (e.g., Eriksen and St. James 1986 ) . The cue is the reference object and the target is the located object. Usually , the focus is on factors other than the apprehension of spatial relations; nevertheless , apprehension for is a major computational requirement in these tasks (see , Logan , example 1995 ).

theviewerwith a completely DTasks Verificationtaskspresent 13 .4.3.3 Verificatio " and ask whetherit " Is ? next to Stella e. . relation ) applies ( g, Daisysitting specified . The focusmay be on one or the other of the or a givendisplay to a givenscene " ?" ; or it may be on the relation , as in Is that Daisysitting next to Stella arguments ?" If the focusis on thearguments itself, asin " Is Daisysittingnext to Stella , verification could( 1) find thereference couldbedoneasa cuingtask. Theviewer object(e.g., with relevant 3 the on it a reference frame 2 Stella ; ( ) align spatialtemplate ); ( ) impose a a locatedobjectthat occupies the reference frame(the onefor next to); (3) choose in the ; (4) comparethat object with the one specified region good or acceptable " " or " no" if it doesnot. ? ); and (5) report yes if it matches question(i.e., Is it Daisy could bedoneasa judgment , verification , if the focusis on the relation Alternatively task. Theviewercould( 1) find the located ); (2) find the reference object object(Daisy until frameon it ; (4) iteratethroughspatialtemplates a reference ); (3) impose (Stella fit is found; (5) comparethe relation the best fit is found or until an acceptable in the question with that templatewith the one asserted associated ; and (6) report " " " " if it matches and no if it doesnot. yes . A host of experiments literature Verificationtasksare commonin the psychological and pictures between sentences in the 1970s studiedcomparisons , and spatial and Just 1973 . Clark in that work e. relations , Carpenter , ). Subjects ( g, figuredlargely then that and that described weregivensentences depicted pictures spatiallayouts . the picture described whether the sentence them . The taskwasto decide 13 .5 Evidence for theTheory

504

G. D . Logan and D. D . Sadier

13.5.1 ApprehelBionRequiresSpatial Indexing ) found evidencethat apprehensionof spatial relations requires spatial Logan ( 1994 indexing in visual searchtasks. On eachtrial , subjectswere presentedwith a sentence that described the relation between a dash and a plus (e.g., " dash right of plus" ), followed by a display of dash-plus pairs. Half of the time, one of the pairs matched the description in the sentence(e.g., one dash was right of one plus), and half of the time, no pair matched the description. All pairs exceptthe target were arranged in the opposite spatial relation (e.g., all the other dashes were left of the corresponding pluses ) . The experimentsexamined the relations above , below , left of, and right of In one experiment, the number of dash-plus pairs was varied, and reaction time increasedlinearly with the number of pairs. The slope was very steep (85 ms/ item when the target was present that the ; 118ms/ item when it was absent ), which suggests pairs were examined one at a time until a target was found (i.e., the pairs were spatially indexed elementby element until a target was found) . A subsequentexperiment of practice (6, 144trials), suggesting replicated theseresults over twelve sessions that subjectscould not learn to compute spatial relations without spatial indexing. In a third experiment, the number of pairs was fixed and attention was directed to one pair in the display by coloring it differently from the rest. When the differently colored pair was the target, performance was facilitated; subjects were faster and more accurate . When the differently colored pair was not the target, performancewas . This suggests that apprehensionof impaired; subjectswere slower and lessaccurate spatial relations requires the kind of attentional processthat is directed by cueslike discrepant colors (i.e., spatial indexing) . 13.5.2 Apprehe . ion RequiresReference Frame Computation ) found evidencethat apprehensionof spatial relations requires reference Logan ( 1995 frame computation in experiments in which attention was directed from a cue to a target. The relation between the cue and the target was varied within and between . Overall, six relations were investigated : above experiments , below ,front , back, left of, and right of The operation of a reference frame was inferred from differences in reaction time with different relations: above and below were faster than front and back, and front and back were faster than left of and right of Clark ( 1973 ) predicted thesedifferencesfrom an analysisof the environmental support for eachrelation , and confirmed Clark ' s predictions in tasks that required searching Tversky and colleagues imagined environments (Bryant, Tversky, and Franklin 1992 ; Franklin and Tversky ' s 1973 1990 . to Clark above and below are easy becausethey ) According , ( ) analysis are consistent with gravity, consistent over translations and rotations produced by locomotion , and supported by bodily asymmetries(heads are different from feet) . Front and back are harder because they are supported by bodily asymmetriesbut not

A Computational Analysis

505

by gravity and they changewith locomotion through the environment. Left and right are hardest of all becausethey are not supported by gravity or bodily asymmetries . and they changewith locomotion ; they are often defined with referenceto other axes of Our theory would account for thesedifferencesin terms of the difficulty aligning referenceframes and computing direction. In Logan' s ( 1995 , subjectsreported targets that were defined by their ) experiments Some to a cue. relation experimentsstudied deictic relations, using an asterisk spatial as a cue and asking subjectsto project their own referenceframes onto the asterisk. , below, left, or right) and Subjectssaw a display describing a spatial relation (above then a picture containing severalobjects surrounding an asterisk cue. Their task was to report the object that stood in the relation to the asterisk cue that we specifiedin the first display. Subjectswere faster to access objects aboveand below the cue than ' to accessobjects right and left of it , consistent with Clark s ( 1973 ) hypothesis and with our assumption that orienting referenceframesand deciding direction take time. Other experimentsstudied intrinsic relations, using a picture of a human head as a cue and asking subjects to extract the intrinsic axes of the head. Again , the first , below , front , back, left , or right) and the second display contained a relation (above contained a display in which objects surrounded a picture of the head. Subjectswere faster with aboveand below than with front and back, and faster with front and back than with left and right . via different relations. In some experiments , the same object could be accessed Accessto the object was easywhen the relation was aboveor below and hard when it was left or right . The cue was presentedin different positions, and the regions that were easy and hard to accessmoved around the display with the cue. This suggests . that the referenceframe can be translated acrossspace In other experiments , the orientation of the reference frame was varied. With deictic cues , subjects were told to imagine that the left side, the right side, or the bottom of the display was the top, and the advantage of aboveand below over the other relations rotated with the imagined top . With intrinsic cues , the orientation of the head cue was varied, and the advantageof aboveand below over the other relations rotated with the orientation of the head. Thesedata suggestthat the reference frame can be rotated at will .
13.6 Evidence for Spatial Templates

that spatial relations are apprehendedby computing the goodness The theory assumes the of fit between position of the located object and a spatial template representing the relation that is centeredon and aligned with the referenceobject. The idea that spatial templatesare involved in apprehensionis new and there is not much evidence

506

G. D. Loganand D. D. Sadier

for it (but seeHayward and Tarr 1995 ). Sections13.7- 13.10presentfour experiments that test different aspectsof the idea. The first experiment assess es the parts of space that correspond to the regions of greatestacceptability, using a production task. The secondassess es parts of spacecorresponding to good, acceptable , and bad regions, describe pictures. The third using a task in which subjects rate how well sentences assess esthe importance of spatial templatesin thinking about spatial relations, using a task in which subjectsrate the similarities of words that describe(lexicalized ) spatial relations and comparing the multidimensional similarity spaceunderlying those ratings with one constructed from the ratings of pictures in the secondexperiment. The final experiment tests the idea that spatial templates are applied in parallel, using a reaction time task in which subjectsverify spatial relations betweenobjects.
13.7 Experiment I : Production Task

The first experiment attempted to capture the regions of spacecorresponding to the best examplesof twelve spatial relations: above , below , left of, right of, over, under , next to, awayfrom , near to, far from ; on, and in. Subjectswere presentedwith twelve frames, with a box drawn in the center of each one; above each frame was an instruction to draw an X in one of the twelve relations to the box (e.g., " Draw an X above the box" ) . We assumedthey would draw each X in the region corresponding to the best example of each relation , though we did not require them to. There were 68 . The frames , who were volunteers from an introductory psychology class subjects were drawn on three sheetsof paper, four frames per sheet , and three different orders of sheetswere presented .2 Each frame was 5.9 cm square and the central box was 8.5 mm square. The data were collated by making transparencies of each of the twelve frames. For each relation , we superimposedthe transparencyon eachsubject' s drawing and drew a dot on the transparency(with a felt pen) at the point corresponding to the center of the X that the subject drew, accumulating dots across subjects . The data for above , below , over, under , left of, and right of are presentedin figure 13.1, the data for next to, awayfrom , near, far from , in, and on are presentedin figure 13.2. The relations in figure 13.1 differ primarily in the orientation and direction of the reference frame. The patterns in each panel are similar to each other, except for rotation . The main exception is over, where somesubjectsdrew Xs that were superimposed on the box, apparently interpreting over as covering (which is a legitimate interpretation; seeLakoff 1987 ) . Note that distance did not matter much. Some Xs were placed close to the box but others were placed quite far away, near the edgeof the frame. In eachcase , the Xs appearedroughly centeredon the axis of the reference frame extendedoutward from the box.

A Computational Analysis
Below

D f , .
Over . ., . ~ t ~
Uader 0 t . ..-

Left of

Right of

' . ..

if1I Dre 13.1 Data for above , below, over, under, left of, and right offrom the production task in experiment I . Each point representsthe center of an X drawn by a different subject to stand in the relation to the central box that is specifiedabove each frame.

508
to Next

from Away . . .. . ~ ,. . .. . D . . . ,..


Farfrom ~ . , . 0 00 ..

G.D. Logan and D. D. Sadier

Near . . '". . . . a .. . .. . .. " "

Figure 13.2 Data for next to, awayfrom , near,far from , in, and on from the production task in experiment I . Each point representsthe center of an X drawn by a different subject to stand in the relation to the central box that is specifiedabove each frame.

. . ~ : : : ~ . ~ ~ .

On r!J

A Computational

Analysis

509

The relations in the top four panelsof figure 13.2 depend primarily on the scaleof the referenceframe and not on orientation or direction. Xs exemplifying next to and near were placed close to the box, whereasXs exemplifying awayfrom and far from were placed somedistance from it , close to the corners (especiallyfor far form ) . One unexpectedresult was that next to was interpreted as horizontal proximity . No subject drew an X above or below the box for next to, though many did so for near. This unanticipated result appearsagain in the next experiment. The bottom two panels of figure 13.2 representin and on. All subjectsdrew their Xs so that their centers were within the boundaries of the box for in, but not all subjectsdid so for on. Somedrew the X as if it were on top of the box, and one drew the X centeredon each side of the box. All of theseare legitimate interpretations of the relations.

13 .8 Experiment 2: Goodnea RatingTask


The second experiment attempted to capture the regions corresponding to good, , and bad examples of ten of the relations used in experiment I : above acceptable , below, left of, right of, over, under , next to, awayfrom , near to, and far from . Subjects were shown sentences , followed by pictures on computer monitors , and were asked to rate how well the sentence describedthe picture on a scalefrom 1 (bad) to 9 (good) . Each sentence was of the form " The X is [relation] the 0 " and each picture contained an 0 in the center of a 7 x 7 grid and an X in one of the 48 surrounding positions. The grid , which was not visible to the subjects , was 8.8 cm wide and 9.3 cm high on the computer screen . Viewed at a distance of 60 cm, this corresponded to 8.3 degreesx 8.8 degreesof visual angle. Each of the 48 positions was tested for each relation so that we could get ratings from good, acceptable , and bad regions. There were 480 trials altogether (48 positions x 10 relations) . Subjectsreported their rating by pressingone of the numeric keys in the row above the standard QWER TV , volunteers from an introductory psychology keyboard. There were thirty -two subjects . The data were collated by averaging ratings across subjects class . The average ratings are plotted in figures 13.3 and 13.4 and presentedin table 13.1. Subjectswere very consiste ~t ; the mean standard error of the averagesin figures 13.3 and 13.4 is 0.271. , below , over, under , left of, and Figure 13.3 presentsthe averageratings for above right of drawn as three-dimensional graphs. Screenpositions are representedin the up- down axis and the left -right axis. The up-down axis goesfrom upper left to lower right ; the left -right axis goes from lower left to upper right . Ratings are represented in the third dimension, which is essentiallyvertical on the page. The central position , which was occupied by the 0 , is blank.

510

andD. D. Sadier G. D. Logan

Y K AlT J

~ w

~, .BA

OYK /I

{J )VDAW

.DT 0 , . / I / G . /7 0 . 1 "
.3 Fiaure13 ratingtask , left of , under , andright offrom thegoodness , over , below ratingsfor above Average 9 (good to I from scale on a ) the ) (bad 2. Each in experiment goodness point represents average the relation to an 0 presented in the positionof the point exemplifies with which an X presented . in thecentralposition As with the production task the patterns in the different panels appear to be the same except for changes in orientation and direction. The highest ratings near , left of, or right of , over, under , below 9- were given to the three points directly above " " the central position , which correspond to the best regions that we saw in experiment " " 1. Note that distance did not matter much in the best regions; ratings were close to 9 whether the X was near to the 0 or far from it . Intermediate ratings were given to the 18 positions on either side of the three best positions, and the lowest ratings (near I ) were given to the remaining 27 points . There was a sharp boundary between bad and acceptable regions. The boundary between acceptable and good regions was lessmarked. The acceptableregions themselveswere not uniform . With above , for example, ratings in the first position higher than the 0 tended to decrease

. .4 Figure13 , away Average ratingsfor nexI 10 , andfar from from the goodness from, near10 rating taskin 2. Eachpoint represents the average experiment on a scale from I (bad goodness ) to 9 (good ) with which an X presented in the positionof the point exemplifies the relationto an 0 presented in thecentralposition . as the position of the X extendedfarther to the left and the right , whereasratings for the highest positions were not affected much by distance from the center, as if the . The mean ratings for the first region of intermediate fit were slightly V -shaped position higher than the 0 were 5.63, 6.41, 7.09, 8.53, 7.35, 6.74, and 5.53 from left to right . The mean ratings for positions directly above the 0 were 8.53, 8.55, and 8.61 from bottom to top . The sametrends can be seenwith the other relations. The averageratings for next to, awayfrom , near to, and far from are presentedin figure 13.4 using the same three-dimensional format as figure 13.3. For next to and near to, ratings were highest in positions adjacent to the central position (occupied by the 0 ) and they diminished gradually as distance increased . Consistent with experiment 1, there was a tendency to interpret next to horizontally ; positions to the left and right of the central position were rated higher than positions the samedistance away but above and below the central position . The mean ratings for the positions immediately left and right of the 0 were 8.17and 8.39, respectively , whereasthe mean ratings for the positions immediately above and below the 0 were 6.07 and 6.19, . respectively " " Awayfrom andfar from were mirror images of next to and near to. Ratings were lowest in positions immediately adjacent to the central position and rose gradually as

~ ~ "W .. I : 4 / 1n1O ~ W
A Computational Analysis

511

HA : l1' 7rJ

A' AJ '!.IrJ6

SKllIIm

G. D . Logan and D . D .

.1 Table13 2 as a Functionof the Position Mean Goodness Ratingsfor Each Relationin Experiment X the by Occupied

~ -O

Above 7.00 6.69 5.63 1.94 1.94 1.81 1 .44 Below 1 .50 1.71 1.94 2.16 5.66 6.00 7.42 Over 8.84 6.75 5.69 1.91 2.28 1.69 1.52 Under 1.81 1 .83 1.77 2.06 5.71 6.59 7.22 Left 6.56 7.00 7.13 8.35 6.84 6.03 6.16

N ~ ~

~\ -

- - ff

~ ~ ~

~ -~

QO O ~

- O ff

8.16 8.71 8.40

~ \ In

NNN InN \

~ .N

t -'

~ ~~

o ~ V
1.47 1.84 2.19 2.00 5.84 6.10 7.03

= ~ ~
7.66 6.88 5.53 2.00 1.66 1.58 1.44

- M

A Computational Analysis

513

Table i 13.1 (continued) NNN NNtf N

1.66 2.00 2.13 1 .38 2.25 1 .81 1 .94


2.10 3.31 5.90 8.17 6.59 3.66 2.53

5.50 5.78 6.39 8.35 6.03 5.59 5.47

6.45 6.52 6.84 8.52 6.81 6.72 6.13

- ~ .

2.03 3.91 6.07


~ -~

2.29 3.35 6.57 8.39 5.91 4.00 1.81


7.72 5.69 2.78 1.88 3.44 5.41 7.44

1 .94 3.34 4.72 6.69 5.38 3.32 2.00 8.10 6.72 5.13 4.58 5.41 5.75 7.83
1.81 3.56 4.80 6.13 5.59 3.47 2.13 7.88 5.38 4.13 4.22 4.09 6.00 7.78

7.45 5.74 2.94 2.13 3.09 5.34 7.58


2.84 4.66 7.55 8.52 6.94 4.50 2.03 7.56 5.41 2.28 1.87 2.28 4.88 7.58

2.34 4.90 7.29 7.90 7.31 4.41 2.53 7.38 5.19 2 .84 1.66 2 .31 5.16 7.47

5.19 7.13

~ ~~

~~~

514

G. D. Logan andD. D. Sadier

. The corner positions, which were the most distant, got the highest distanceincreased ratings. As with figure 13.3, the ratings in figure 13.4 appear to capture the regions of best fit that were found in experiment I . The parts of spacethat receivedthe highest ratings were the parts of spacein which subjectstended to draw their Xs. The data in figures 13.3 and 13.4 capture our idea of spatial templatesquite graphically . One can imagine centering the shapein each panel on a referenceobject, rotating it into alignment with a referenceframe, and using it to determine whether a located object falls in a good, acceptable , or bad position .

3: SimilarityRatingTask 13 .9 Experiment
The data in figures 13.1- 13.4 suggesta pattern of similarities among the relations. , left of, and right of have similar , over, under , below Templatescorresponding to above shapesbut differ from eachother in orientation and direction. Templatescorresponding to next to, awayfrom , near to, and far from have different shapesfrom above , to near to are that next and to each other but are similar and so on below , , except reflections of awayfrom and far from . The purpose of the third experiment was to capture thesesimilarities in a task that did not involve external, visible relations. Subjectswere presentedwith all possiblepairs of words describing the twelve relations , next to, awayfrom , near to, far from , , below , left of, right of, over, under , above in, and on, and they were asked to rate their similarity on a scaleof I (dissimilar) to 10 (similar) . The words were printed in pairs with a blank beside them, in which -spaced subjectswere to write their rating . The 66 pairs were presentedin two single columns on a single sheet of paper. There were four groups of subjects (26, 28, 19, and 28 in each group) who receivedthe pairs in different orders. The subjects were . 101volunteers from an introductory psychology class The ratings for each word pair were averaged across subjects , and the averages were subjectedto a multidimensional scaling analysis , using K YST (Kruskal , Young, and Seery 1977 ) . We tried one-, two -, and three-dimensional solutions and found that stress (a measure of goodness of fit , analogous to I - r2) was minimized with a three-dimensional fit . The stressvalues were .383, .191 , and .077 for the one-, two -, . The similarity space for the threeand three dimensional solutions, respectively dimensional solution is depicted in figures 13.5, 13.6, and 13.7. .Figure 13.5 shows the plot of dimension I against dimension 2, which appearsto -below be a plot of an above , dimension against a near-far dimension. Above and over appear in the bottom right , and below and under appear in the top left. A wayfrom andfar appear in the bottom left , and next to, near, in, and on appear in the top right . -below x Left and right appear in the middle, reflecting their projection on the above near-far plane.

A Computational Analysis

515

TERMS OF 12 SPATIAL SCALING SIMILARITY

. 0 5 . 0 0 -0 . 5 -1 . 0 -1 . .5 .5 1 0 1 . 010 0 .5DIMENSION . 0 -0 . 5 -1
0 UNDER
RFL ow 0 ON

NEXT NEAR 0TO

RIGHT

0 LEFT

0 ABOVE 0 OVER

AWAY FROM

- 1.5

.5 Figure13 2 plotted from a similarity spaceconstructed from a multidimen Dimension1 x dimension of similarityratings of twelve spatial terms in experiment 3 (the numbers on the sionalscaling -below x near-Jar . of distance axes arearbitrarymeasures ) . The dimensionsappear to be above Figure 13.6 shows the plot of dimension I against dimension 3, which appears to -below dimension against a left -right dimension. Above and over be a plot of an above appear on the left side, and below and under appear on the right . Left appearson the top, and right appears on the bottom . The other relations are scattered over the -below x middle of the plot , reflecting the projection of the near-far axis on the above left -right plane. Figure 13.7 shows the plot of dimension 2 against dimension 3. This appearsto be a plot of near-far against left-right . In , on, next to, and near appear on the top , whereasfar and away from appear on the bottom . Right appears on the left side, while left appears on the right . Above, over, below , and under are scatteredover the axis on the near - below -far x left-right plane. , reflectingthe projection of the above plane

516

G. D . Logan and D . D . Sadier

OF 12 SPATIAL TERMS SCALING SIMILARITY

0 LEFT

0~ R NEXT
AWAY FROM 0 BELOW 0 UNDER

0 OVER 0 ABOVE
0 ON

-0 . 5 -1 . 0 -1 . 5 -1 . . 0 -0 . 5DIMENSION . . . 5 -1 0 010 .5 51 01
0 FAR 0 IN

0 RIGHT

Figure 13.6 Dimension I x dimension 3 plotted from a similarity spaceconstructed from a multidimensional scaling of similarity ratings of twelve spatial terms in experiment 3 (the numbers on the -below x left -right . of distance axesare arbitrary measures ) . The dimensionsappear to be above The similarity structure in these plots resembles that seen in figures 13.1- 13.4. The templates for above and over have similar shapes, opposite to those for below and under. The templates for left and right are opposite to each other and orthogonal to above and below. The templates for far and away from are similar to each other and opposite to near and next to , and all of their shapes are different from those of above,

below, left, right, and so on. In order to fonnalize these intuitions , we calculated similarity scores from the spatial templates in figures 13.3 and 13.4 and subjected them to multidimensional . We treated the forty scaling, using KYST . The procedure involved several steps similarity betweenrelations by eight ratings for each relation as a vector and assessed That is, we multiplied the vectors. the of the dot corresponding product computing

A Computational

Analysis

517

TERMS OF 12 SPATIAL SCALING SIMILARITY

1 . 0
TO 0 NEAR 0NEXT 0 UNDER
0 BELOW 0 ON t 0 RIGHT 0 LEn

. 7 13 Figure mult a from constru a from 3 space x dimension 2 Dimension the on numb t he 3 in terms ( twelve of experim of spatial ratings similarity scaling . x near be to r igh left f ar The dimension . distance of appear measures ) are axes arbitrary
plotted similarity

-0 . 5 -1 . 0 -1 . 5 .5 . 0 1 -1 . 51 . 0 020 . 5DIMENSION . 0 -0 . 5 -1
ABOVE 0 OVER
FROM AWAY

analogous score a similarity them andadded cells upto produce in corresponding ratings normalized we thedot product , . Before coefficient computing to a correlation valuefor each same the to values their of sum the squared the vectors , setting all possible pairsof the ten , reflecting wereforty-fivedot products . There relation as weretreated dot five 2. These products in experiment forty examined relations two one we tried , -, . Asbefore , theKYSTprogram andranthrough ratings similarity dimensional witha three minimized stress -dimensional andfound solutions andthree dimensions three and two one for .009 and , .139 , .315 were values , , . Thestress solution 13 .8, in is dimensional space plotted figures . Thethree similarity , respectively . .10 .9, and13 13

518

andD. D. Sadier G. D. Logan

RATINGS NORMALIZED GOODNESS


RIGHT

00ABOVE OVER

NEARNEXTTO ~0 FROM FAR AWAY

BELOW c9UNDER

-0 . 5 -1 . 0 -1 . 5 -1 . . 5 -1 0 -0 . . 5DIMENSION 0 010 . 51 . 01 . 5
.8 Figure13 DimensionI x dimension 2 plotted from a similarity space constructed from a multidimensional 2 scalingof dot productsfrom goodness ratingsof ten spatialtermsfrom experiment on the axesare arbitrary measures of distance to be (the numbers ). The dimensions appear -belowx left-right. above The dimensional structure that emergedfrom the scaling analysis of the goodness ratings was very similar to the one that emerged from the similarity ratings. The structure had three dimensions and the three dimensions could be interpreted similarly . Figure 13.8 contains the plot of dimension 1 against dimension 2, which is easily -below axis against the left -right axis. Figure 13.9 interpretable as a plot of the above contains the plot of dimension 1 against dimension 3, which appearsto be a plot of -below axis against the near-far axis. Figure 13.10 contains the plot of dimension the above 2 against dimension 3, which appears to be a plot of the left -right axis the similarity of the fits quantitatively by calculating against the near-far axis. We assessed the correlation between the interpoint distances in the two solutions. Each

A Computational Analysis

519

RATINGS GOODNESS NORMALIZED

00 NEAR NEXTTO

. 0 0 -0 . 5 -1 . 0 -1 . 5 -1 . 5 -1 . . . . . . 0 -0 5DIMENSION 0 00 51 01 5
00 OVER ABOVE

~T ~ 'm

BELOW < 5 > UNDER

0 AWAY FROM 0 FAR

Figure 13.9 Dimension I x dimension 3 plotted from a similarity spaceconstructed from a multidimensional scaling of dot products from goodnessratings of ten spatial terms from experiment 2 (the numbers on the axes are arbitrary measuresof distance ) . The dimensions appear to be -be/ow x near-far. above

. solution gives the distance betweeneach pair of relations in multidimensional space If the solutions are similar , then the distances between the same pairs of relations in the two spaces should be similar. The correlation was .858, indicating good . agreement The similarity of the scaling solutions and the high correlation betweendistances suggeststhat the ratings of pictures in experiment 2 and the ratings of words in the present experiment were based on common, underlying knowledge structures. We would like to conclude that subjects used spatial templates to perform both tasks. Thus they rated pictures by aligning spatial templateswith the referenceobject and computing the goodnessof fit for the located object, and they rated words by

520

1 . 5 1 . 0

G. D . Logan and D . D . Sadier

NORMALIZED GOODNESS RATINGS


RIGHT

FAR 0 0 AWAY FRofI

NEAR NEXT TO

-0 . 5 -1 . 0 -1 . 5 -1 . . 5 -1 0 -. DIMENSION . 2. . .
Figure 13.10 Dimension 2 x dimension 3 plotted from a similarity spaceconstructed from a multidimensional scaling of dot products from goodnessratings of ten spatial terms from experiment 2 the numbers on the axes are arbitrary measuresof distance ( ) . The dimensions appear to be left -right x near-far.

, comparing the spatial templatesassociatedwith them. This conclusion is speculative however. Although there is some evidencethat subjectsmay compare images when escould given words (Shepardand Chipman 1970 ), other representationsand process the . same outcomes The data are consistent with our conclusion but , produce they do not rule out competing interpretations.

13 .10 Experiment 4: Relation Task Judgment The resultsof experiments ) - 3 are consistent with the hypothesis that spatialtem wereappliedin parallelto thewholeperceptual , but theydo not plates representation . The sameresultscould have beenproducedby support that hypothesis uniquely

A Computational

Analysis

521

. Serialvisualroutinesare insteadof spatialtemplates applyingserialvisualroutines a number to compute on perceptual representations esthat operate sequentially process couldbe , above ). For example (Ullman 1984 of things , includingspatialrelations " " and reference upward the moving on object a mentalcursor by centering produced found was located the frame until object along the up-down axis of the reference not was directly , 1991 ). If the locatedobject , Ullman, and MacKay 1986 (Jolicoeur other the to side one from covering move could cursor the , abovethe reference object objectuntil the locatedobjectwasfound. the regionabovethe top of the reference I and 2 may in experiments evidenced From this perspective , the spatialtemplates for serialvisualroutinesratherthan explicit representations reflectpreferred trajectories usedto compute spatialrelationsdirectly(i.e., by multiplyingactivationvalues wasto contrastspatial earlier as described ). The purposeof the fourth experiment also of with serialvisualroutinesin the apprehension spatialrelations(see templates 1991 ). ; Sergent Loganand Compton1996 and serialvisualroutinesis between contrast of spatialtemplates The main point are appliedin . Spatial templates the effectof distancein judging spatial relations reference and located objects between parallelto the wholevisualfield, so distance not should depend . The time taken to apply a spatial template doesnot matter the , examining routines visual serial sequentially contrast operate . , on distance By objectsshould visual field bit by bit, so distancebetweenlocatedand reference increase should routine visual a serial . The time taken to apply makea difference . with distance monotonically hasno effecton the goodness I and 2 that distance in evidence the Note, experiments , left of of above , andright of doesnot bearon this , under , over of examples , below couldhavetakenmore . Subjects measured nor timewasneitherstressed because issue same the rating. Theratingcould eventhoughtheygave distances timeto rategreater frame the locatedobjectand the reference on the relation between havedepended . the relation , not on the time takento compute on the reference centered object between distance the which in task verification a 4 had subjects perform Experiment Clark cf. , , Carpenter ( and locatedobjectswasvariedsystematically reference visual of 6 usedin this experiment ( 1 degrees andJust 1973 ). The rangeof distances increases monotonic , in reactiontime in shows that the within range ) waswell angle , , Ullman, and MacKay 1986 other tasks , such as mentalcurve tracing (Jolicoeur in the relations 1991 ); if serial visual routineshad beenusedto computespatial . with distance haveincreased , reactiontime shouldtherefore experiments present a with . Eachtrial began and below above on the relations focused Theexperiments . It wasextinguished for 500ms in the centerof a computerscreen fixation point exposed a dashand a between relation the with a sentence expressing and replaced " " " " " ?" , or " Plus dash ? , Plus above ? ; Dash below plus plus (i.e., Dash aboveplus

522

G. D. Loganand D. D. Sadier

below dash?" ) that was exposedfor 1,000 ms. After the sentencewas extinguished , the fixation point appeared for another 500 ms. Then a picture of a dash above or below a plus was exposedfor 200 ms, too briefly to allow eyemovements . Half of the time, the relation between the dash and plus matched the sentence , and half of the time, the opposite relation held. Subjectswere told to respond " true" to the former caseand " false" to the latter. After the 200-ms exposure of the picture, the screen went blank until the subject responded . After the response , the screenremained blank for a 1,500 ms intertrial interval. There were 384 trials in all. The main manipulation was the distance between the dash and the plus. There were four different distances . In one version of the experiment, the dash and plus were separated by I , 2, 3, or 4 screen lines (corresponding to .74, 1.48, 2.22, and 2.96 degreesof visual angle when viewed from a distance of 60 cm) . In another version, distanceswere doubled. The dash and the plus were separatedby 2, 4, 6, or 8 screenlines ( 1.48, 2.96, 4.44, or 5.92 degreesof visual angle) . Stimuli separated . In by the different distances appeared in several different locations on the screen the version in which distances were 1- 4 screen lines, stimuli with a distance of I appearedin positions I and 2, 2 and 3, 3 and 4, and 4 and 5; stimuli with a distance of 2 appeared in positions I and 3, 2 and 4, and 3 and 5; stimuli with a distance of 3 appeared in positions I and 4, and 3 and 5; and stimuli with a distance of 4 appeared in positions I and 5. The same schemewas used in the version in which distanceswere 2- 8 screenlines, except that positions 1- 5 were two lines apart . Distances , relations (above vs. below ), and true and false trials occurred in random order. A different random order was constructed for each subject. The subjectswere 48 volunteers from an introductory psychology class . Twenty-four served in each version of the experiment. Mean reaction times were computed for " true" and " false" responses as a function of distance. The meansacrosssubjectsare plotted in figures 13.11 and 13.12. Figure 13.11 plots reaction time as a function of absolute distance , expressedin degreesof visual angle. It shows that reaction time was longer for " false" responsesthan for " true" in both versions of the experiment, F ( I , 44) = 78.97, p < .01, mean responses ,274.38. Reaction time was longer in the version with the square error (MSE) = 102 , but the difference was not significant, F ( 1, 44) < 1.0. The most greater distances important result for our present purposesis the effect of distance. Serial visual routines predict a monotonic increase in reaction time as distance increases , whereas spatial templates predict no effect. Analysis of variance showed a significant main effect of distance , F (3, 132 ) = 4.33, P < .01, MSE = 57,930.55, and the linear trend = was significant, F ( I , 132 ) 4.77, P < .01, indicating a tendency for reaction time to decrease as distanceincreased . The observedpattern is clearly inconsistent with serial

A Computational Analysis

1200

523

V )1100 ~ z .a w / / " ~ / v t= ~ 1000 z

0 I0 ~900 ~
"'

G '" " " " " , " G " 0 ~ '~ " ~ " e
FALSE
G, " ' ,. ." " , ' --0 - - - - - - - - - - - - - 9- - - , , TRUE - , .0 " '-

FALSE

TRUE

",,

800 01DISTANCE 4 2 3 OF 5 6 VISUAL ANGLE


IN DEGREES .11 Figure13 Reactiontime as a functionof absolute distance between reference and locatedobjectsfrom " false " 4 in whichsubjects . " True" versus two versions of experiment andbelow judgedabove . and long (dottedlines short(solid lines arethe parameters ) versus ) distances response visual routines. In both versions of the experiment, reaction time was longest for the shortest and longest distancesand fastestfor the intermediate distances . The pattern of reaction times is not exactly what one would expect from the spatial , which predicted no effect of distance. However, the pattern may template hypothesis be consistent with theory of apprehensionin which spatial templates playa part, if the slower reaction times at the longest and shortest distancescan be explained. We suggestthat the pattern reflects a process of reference frame adjustment. Subjects may have set the scaleof their referenceframes to the averagedistancesthey experienced - distancesof 2 and 3 in one version and distancesof 4 and 6 in the other. They may have adjusted them if the distance were longer or shorter than the average in 4 and I in one version and 8 and 2 the other. This would the distances produce observedpattern of results. The effect can be seenmore clearly in figure 13.12, which plots reaction time as a function of ordinal distance rather than relative distance. The patterns from the two versions of the experiment align nicely in figure 13.12. Of course , this explanation is post hoc, and must be taken with a grain or two of salt (however, no distance effects were found by Logan and Compton 1996and by ). Sergent 1991

~, ~ \,.- - - -

524

G. D . Logan and D . D . Sadier

1200

V '> ~ 1 1 00

G ," , " ~

w ~ i= 1000 z 0 i= u ~ ( k :
900 800

.o TRUE ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ TRUE " " "' - = ~ = ~ = ~- - -------" ~ ""

G ," " "

'" -0 .~ ~ FALSE ~ ~ ~ ' -" ~ ~ ~ ~ ~ ~ > > < ----- - - ; ; <: : :: FALSE ~

" -'

,' '

1 ORDINAL 2DISTANCE 3 4

.12 Figure13 from two reference andlocated between Reaction timeasa functionof ordinaldistance objects " . " True" versus" false and below 4 in which subjects versions of experiment judged above . the short(solid lines are parameters and long (dottedlines ) distances ) versus response
13.11 Conelusiol B

The data from experiments 1- 4 support the idea that spatial templates underlie the apprehensionof spatial relations. Experiments 1 and 2 showedthat the spacearound a referenceobject is divided into regions that represent good, acceptable , and bad . and Tarr 1995 see also a relation of ) Experiment 3 showed ( Hayward given examples for in terms of be accounted can of terms that similarities in the meanings spatial similarities in the spatial templates that correspond to them. And Experiment 4 showed that distance between referenceand located objects has little effect on the time required to apprehend relations, as if spatial templates were applied in to the whole visual field in simultaneously (see also Logan and Compton 1996 ; Sergent 1991 , 1995 ), the experimentssupport the ) . Together with the other data (Logan 1994 computational analysisof apprehensionpresentedearlier in the chapter and argue for its viability as a psychological theory of apprehensionin humans. Severalparts of the theory were taken from existing analysesof spatial relations. Referenceframes and spatial indices play important roles in linguistic and psycholinguistic ; Gam , 1994 ; Clark 1973 analyses(seeCarlson- Radvansky and Irwin 1993 ham 1989 ; Landau and Jackendoff ; Jackendoff and Landau 1991 ; Herskovits 1986

A Computational Analysis

525

-Laird 1976 1993 ; and Talmy 1983 ; Miller and Johnson ). ; Levelt 1984 ; Logan 1995 of fit is computedwith spatial The novel contribution is the idea that goodness it is computationally this idea because . We suggested simpleand easyto templates " " . It would be interesting to contrastspatialtemplates in software or wetware implement of fit in future research with otherwaysto computegoodness , (e.g., geometric . or functional relations volumetric , topological , ) of spatialprepositions to accountfor the apprehension The theorywasdeveloped in English . As is readily apparentin the other chaptersin this volume , different , so it is important to consider spatialrelationsin differentways languages express . What is general across to otherlanguages how thetheorymightgeneralize languages to most ? We suspect that the theorycouldbeadapted and what is specific to English . Most languages relationsbetween objectsin termsof reference express languages . We suspect that reference frame computation framesappliedto reference objects reference and spatialindexing(which is requiredto distinguish objectsfrom located . common to all The be ) may appliedto the reference objects languages spatialtemplates are . We suspect that spatial templates objectsmay vary between languages to capturethe distinctionsthat are important by the linguisticenvironment shaped must be commonto all languages in particular languages . The perceptual representation . The conceptual because it is precognitive and thus prelinguistic representations . We suggest that theconceptual representations clearlyvary between languages with which from eachother in termsof the spatialtemplates may be distinguished . theyareassociated in this chapterare crudeapproximations to the measured The spatialtemplates . measurements use if use them at all The that ) templates peoplemight actually ( they and located 2 useda 7 x 7 grid) and the reference werecoarse(e.g., experiment that the resultswould generalize weresimple(boxes , Osand Xs). We suspect objects . Indeed and more sophisticated to finer measurements , Haywardand Tarr objects Irwin 1993 found similar resultswith several and 1995 and Carlson ) ( ( ) Radvansky could be adaptedto . Certainly differentreference and locatedobjects , the methods of objects more precise measurements , and evendifferentspatial , differentclasses as the final answer relations . Thus we do not view the experiments , but rather , as a of . to an area exciting inquiry promisingbeginning in the presentexperiments The measurements may not havecapturedall of the . ExperimentI , for example the relationswe contrasted differences between , found 2 of over(above and covering of two differentsenses evidence ), whereas experiment 2 not in could . The found evidence of only oneof them(above ) displays experiment the locatedand reference because havepickedup the second objectswere meaning . However to pick up thecontrastwith displays , it shouldbepossible always separated . Subjects should rate overlapping in which locatedand reference overlap objects

526

andD. D. Sadier G. D. Logan

. Thus the limitations displays as good examplesof over but bad examplesof above we used rather than in in the of the present experiments lie specific procedures the general methodology. With appropriately designed displays, rating procedures should be able to capture subtle differencesbetweenrelations. Spatial templates may not capture the meanings of all spatial relations. On, for , implies contact and support ( Bowerman, chapter 10, this volume), neither example . The of which can be describedsufficiently in terms of occupancyof regions of space referenceobject and the located object must occupy the same region of space , but contact and support imply more than that. Contact may be assessed by examining junctions betweenthe contours of the objects using something like templates( Bieder . In as another man 1987 , implies example ), but support cannot be perceivedso easily , containment (Herskovits 1986 ) and that is a functional relationship that cannot be . Flowers in a vase occupy a different described easily in terms of regions of space . in a vase than water of region space Despite these limitations , spatial templates are clearly useful in describing the meanings of many spatial relations. Moreover, they are tractable computationally , and the computational analysis is readily interpretable as a psychological theory of how people actually apprehendspatial relations. The data in the presentexperiments and others (Carlson-Radvanskyand Irwin 1993 , ; Logan 1994 ; Hayward and Tarr 1995 1995 ; Logan and Compton 1996 ) are consistent with the psychological theory, suggesting of geometric, it has somevalidity . Competitive theories, basedon assessment . of this have not reached relations and functional , development stage yet topological ,
Acknowledgments

to Foundationgrant BNS 91-09856 in part by National Science wassupported This research . We would like to Gordon Log ;an. We are g ,ratefulto JaneZbrodoff for valuablediscussion
thank Paul Bloom and Mary Petersonfor helpful comments on the manuscript. Notes " 1. " This is here" and " That is there are often interpreted as deictic relations in linguistic 1984 . However e. . Levelt , the expressionsare interpreted as , in those analyses ) analyses ( g , sentences that one person utters to another. The listener must interpret what the speakersays in terms of two -argument relation between two external objects- the speaker as a reference " " " " must interpret what the object. Moreover, the listener object and this or that as a located " " " ' s frame of reference with " here in terms of the , meaningnear and there speaker speakersays one meaningfar . Basic relations are intrapersonal rather than interpersonal. There is only ' " " " " argument ( this or that ) and there is no external frame of reference(i.e., the viewer s own frame of referencesuffices ) . The viewer is telling himself or herself that an object exists in a to communicate the idea to the the result of that processas a sentence location. We expressed ' reader, but the viewer need not do so. The viewer s representation is conceptual rather than linguistic .

A Computational Analysis

527

2. One sheetcontained under , near, in, and awayfrom in the top left , top right , bottom left , and bottom right positions, respectively . Another contained above , on, right of, and next to. The third contained left of, over, below , and far from . Roughly equal numbers of subjectsreceived the three different orders of sheets(25, 20, and 23, respectively ). References

-by- components Biedennan : A theoryof humanimageunderstanding , I. ( 1987 . ). Recognition Review 94 115 147 . , , Psychological , B., and Franklin, N. ( 1992 BryantD . J., Tversky ). Internaland externalspatialframeworks - 98. for representing described scenes . Journalof MemoryandLanguage , 31, 74 CarlsonRadvansky in vision and language , L. A., and Irwin, DE . ( 1993 ). Framesof reference - 244 : Whereis above ? Cognition . , 46, 223 -Radvansky Carlson frameactivationduringspatial , L. A., and Irwin, DE . ( 1994 ). Reference - 671 tenDassignment . Journalof MemoryandLanguage . , 33, 646 Clark, H. H. ( 1973 . , time .), Cognitive , semantics ) Space , and the child. In TE . Moore ( Ed andtheacquisition Press . development , 27- 63. NewYork: Academic of language Clark, H. H., Carpenter , P. A., and Just , M. A. ( 1973 ). On the meetingof semantics - 381 and perception . In W. G. Chase(Ed.), Visualinformation . New York: , 311 processing Academic . Press Clark, H. H., and Chase in the fonnation and , W. G. ( 1974 ). Perceptual coding strategies - 111 verification of descriptions . MemoryandCognition . , 2, 101 , L. A., and Shepard for a rotatedstimulus . , R. ( 1973 Cooper ). The time requiredto prepare

Memory and Cognition, 1, 246- 250.

Corballis , M . C. ( 1988 . Psychological Review ) . Recognition of disoriented shapes , 95, 115 . 123 Eriksen, C. W., and St. James , J. D . ( 1986 ) . Visual attention within and around the field of focal attention: A zoom lens model. Perceptionand Psychophysics , 40, 225- 240. Franklin , N ., and Tversky, B. ( 1990 ) . Searchingimagined environments. Journal of Experimental : General , 119 , 63- 76. Psychology

Gamham , A. ( 1989 . ). A unified theory of the meaningof somespatial relational terms , 31, 45- 60. Cognition , W. G., and Tarr, M. J. ( 1995 Hayward . ). Spatial languageand spatial representation - 84. , 55, 39 Cognition Herskovits andspatialcognition : An interdisciplinary , A. ( 1986 ). Language studyof theprepositions in English . Cambridge : Cambridge . UniversityPress Jackendoff andspatialcognition , R., andLandau . In D. J. Napoli , B. ( 1991 ). Spatiallanguage and J. A. Kegl (Eds .), Bridges between andlinguists : A Swarth more psychology festschrift for - 169 Lila Gleitman . Hillsdale . , 145 , NJ: Erlbaum

528

andD. D. Sadlel G. D. Logan

in basicoperation : A possible Jolicoeur , L. ( 1986 ). Curvetracing , S., andMacKay , P., Ullman - 140 . . MemoryandCognition of spatialrelations the perception , 14 , 129 . Journalof Jolicoeur , L. ( 1991 ). Visualcurvetracingproperties , P., Ullman, S., and MacKay - 1022 . andPerformance : HumanPerception , 17 , 997 Psychology Experimental versus , O. ( 1992 , C. J., and Koenig ). Categorical , C. F., Marsolek , S. M., Chabris Kosslyn . Journalof and computersimulations : Computationalanalyses coordinatespatialrelations - 577 . andPerformance : HumanPerception , 18 , 562 Psychology Experimental Kruskal , J. B. ( 1977 ). How to useKYST-2: A very flexible , F. W., and Seery , J. B., Young . Bell . Unpublishedmanuscript and do multidimensional to unfolding scaling program Laboratories , Murray Hill , NJ. . reveal aboutthemind : Whatcategories Lakoff, G. ( 1987 , fire , anddangerous things ). Women . Press of : ChicagoUniversity Chicago " " " " Landau , R. ( 1993 ). What and where in spatial cognition and spatial , B., Jackendoff . Sciences . BrainandBehavioral , 16 , 217 238 language . In A. J. van in talking about space Levelt , W. J. M. ( 1984 ). Someperceptuallirnitations - 358 : . Utrecht .), Limits onperception Doom, W. A. de Grind, andJ. J. Koenderink(Eds , 323 . Press VNU Science : Theoryand , G. D. ( 1980 ). Attention and automaticityin Stroopand priming tasks Logan . 12 523 553 . Cognitive data , , Psychology . Journalof of spatialrelations D. 1994 . G. , ) Spatialattentionand the apprehension ( Logan . 1036 20 1015 and : Human , , Performance Perception Psychology Experimental . Cognitive control of visualspatialattention G. D. , ). Linguisticand conceptual ( 1995 Logan - 174 . , 28, 103 Psychology in the apprehension effects anddistraction , B. J. ( 1996 ). Distance , G. D., andCompton Logan andPerformance : Human . , Journal of spatialrelations Perception Psychology of Experimental - 172 . 22, 159 of : Facilitativeeffects , N. J. ( 1979 ). Whenit helpsto be misled , G. D., and Zbrodoff Logan of conflictingtrials in a Stroop , 7, the frequency - like task. Memoryand Cognition increasing - 174 . 166 . . NewYork: Freeman Marr, D. ( 1982 ). Vision of thespatialorganization andrecognition Marr, D., andNishihara , H. K . ( 1978 ). Representation the Transactions . , London , 200 , -dimensional of three RoyalSociety of Philosophical shapes - 294 . 269 -Laird, P. N. ( 1976 . Cambridge andperception , MA: Miller, G. A., and Johnson ). Language . HarvardUniversityPress . In BE . of shape and orientation framesin the perception Palmer , S. E. ( 1989 ). Reference . Hillsdale 121 163 and : Structure Eds . , , and S. Ballesteros process ( ), Object perception Shepp . NJ: Erlbaum

A Computational Analysis

529

Peterson , M . A ., Kihlstrom , J. F ., Rose , P. M ., and Glisky , M . L . ( 1992) . Mental images can be ambiguous : Reconstrua1s and reference frame reversals . Memory and Cognition , 20 , 107123. Pylyshyn , Z . ( 1984) . Computation and cognition . Cambridge , MA : Harvard University Press. Pylyshyn , Z . ( 1989) . The role of location indices in spatial perception : A sketch of the FINST spatial index model . Cognition , 32 , 65 - 97. Sergent , J. ( 1991) . Judgments of relative position and distance on representions of spatial relations . Journal of Experimental Psychology : Human Perception and Performance , 17, 762780. Shepard , R . N ., and Chipman , S. ( 1970) . Second- order isomorphism of internal representations : Shapes of states. Cognitive Psychology , 1, 1- 17. Talmy , L . ( 1983) . How language structures space. In H . L . Pick and LP . Acredolo ( Eds .), Spatial orientation : Theory , research , and application , 225 - 282. New York : Plenum Press. Ullman , S. ( 1984) . Visual routines . Cognition , 18, 97 - 159. Vandaloise , C . ( 1991) . Spatia / preposition : A case study from French . Chicago : University Chicago Press. of

Chapter 14 The Language - to- Object Perception Interface : Evidence from Neuropsychology Tim Shallice

Cognitive neuropsychology has as its principal aim the elucidation of the organization of the cognitive system through the analysis of the difficulties experiencedby neurological patients with selectivecognitive difficulties. As far as the relation between vision and language is concerned , the area that has been most extensively investigatedconcernsthe semanticrepresentationof objects. By contrast, the relation between how representationsof space are accessed from vision and how they are accessedfrom language has been little touched; spatial operations have not been subject to much cognitive neuropsychologyinvestigation. If we consider objects, then the Gibsonian tradition teachesus that the richnessof information available in the visual field is such that many of their properties may be inferred fairly directly from the visual array . Yet there are many other aspectsof the visual world that cannot be inferred from the information in the visual field alonethe structural aspectsof an object that are hidden from the present viewpoint, the potential behavior of an object and of the other objects likely to be found in its vicinity or that go with it in some other way. There are also wider properties of an such as the perceptual features it has when experienced object that may be accessed through other modalities, how it is used and by whom, what its function is, what types of thought processit triggers, and what intentions it may help to create. How are the process es involved in accessingthese properties of an object when it is presented when it is presentedverbally? visually related to the way they are accessed This issue has been the subject of considerable controversy in cognitive neuropsychology in recent years for two reasons . A number of striking syndromesseemto relate very directly to it . In addition , the theory that most directly reflectsthe surface manifestations of the disorders differs from the standard theory in other fields where the issuehas beenaddressed . A model widely referred to in this book and in current cognitive scienceis that of Jackendoff( 1987 ). Languageis viewedas involving three main types of representation - phonological structures, syntactic structures, and semantic /conceptual structures.

532

TimShallice

As far as the semantic , meaningshave internal /conceptual structures are concerned and of from a set built principles of combination , one of primitives up organization " However in addition to its " . the the primitives being , phonological, entity thing contain specifically syntactic and conceptual structures the representationof a word may visual structures. The visual structures involved are, however, explicitly identified with the 3-D structural description level of Marr ( 1982 ). 's was concerned Jackendoff specifically with words and their theorizing Although its in esand particular position on the organization of , the issuesit address meanings the cognitive systemsmediating semantic processingare closely related to issuesrecently . A topic on which there has much debated by cognitive neuropsychologists beenmuch cognitive neuropsychologyresearchin recent years is whether thesemanwhen a word is being comprehendedare the sameas those used tic systemsaccessed in the identification of an object, given that its structural description has already been determined. Somecognitive neuropsychologistshave argued that they are the same , but others have claimed that they differ at least in part . ' Approachesclosely related to Jackendoff s have beenadopted by certain cognitive ; Riddoch and , Berndt, and Brownell 1982 neuropsychologists (e.g., Caramazza of a theory account current . The best 1987 neuropsychological developed ) Humphreys of this type is the organized unitary content hypothesis (OUCH ) of Caramazza . More et al. ( 1990 ), which utilizes a feature basedtheory of semanticrepresentations " an to a semanticrepresentationthrough object will specifically, it holds that access necessarilyprivilege just those perceptual predicatesthat are perceptually salient in an object" . Thus while many elementsof the semantic representation are as easily from visual as from verbal input , someaspectsof the semanticrepresentation accessible from its structural description than from its phonologiare more easily accessed ' . Accessproperties can be asymmetrical. The authors rationale for cal representation assumingan asymmetric relation derives from consideration of certain conditions to be discussed shortly . There is an older tradition in neuropsychology , however, which can be traced back at leastas far as Charcot ( 1883 ) . Certain syndromessuggestthat ) and Wernicke ( 1886 be based partly separablefrom verbally based knowledge. knowledge may visually This perspective has been explicitly adopted more recently by a group of neuropsychologists ; and McCarthy ; Shallice 1987 ; Beauvois 1982 (e.g., Warrington 1975 and Warrington 1988 , ) using the terminology visual semanticsand verbal semantics not been has two of basis of the the clearly representation types conceptual although articulated (see Caramazza et al. 1990 ; and ; Rapp, Hillis , and Caramazza 1993 Shallice 1993 ). An intermediate position has beenadvocatedby Bub et al. ( 1988 ) and by Chertkow and Bub ( 1990 ), they argue that a spe ) . Following Miller and Johnson-Laird ( 1976

-to-Object TheLanguage Interface Perception

533

cific stage intervenes between attaining the structural description and accessingthe amodal " core concept" of an object. Accurate identification of object is held to ' require more than just a characterization of an object s structure, but must involve criteria which are more functional than structural. They therefore argue for the existenceof a subsystemthat contains only the application of the functional and perceptual criteria necessaryfor object identification , receiving the output from the structural description systemand sendingoutput to the core amodal semanticsystem . " is reduced Thus " visual semantics . very considerably in its scope We thus have one position in cognitive neuropsychology (Caramazzaet al. 1990 ) that is entirely compatible with Jackendoff' s perspectivein holding that there is a . In addition it , namely the Caramazzaet al. perspective /conceptual system single semantic certain aspectsof the semantic representation can be , holds that accessing easier from the structural description than from phonology. Two other positions, ' ; Chertkow and Bub 1990 ( Warrington 1975 ) hold that Jackendoff s view is too gross a characterization of the subdivisions of the cognitive system involved in semantic , and that more than one semantic processing /conceptual system exists. A fourth which has to be articulated , holds that semantic representations position , yet formally are processedthrough a connectionist network of which different regions are more specializedfor different types of semantic subprocess , but neither subprocess nor region can be characterizedin an all -or -none fashion (see , for example , Allport 1985 . ; Shallice 1988a ) Two main types of syndrome have beenusedto argue that the semantic - conceptual - those system is not in fact unitary but contains a number of types of subsystem , in involving some form of category specificity, and the modality specific aphasias . I will in review the evidence from each turn and then relate , particular optic aphasia them to the alternative theories. A third syndrome- selectiveprogressiveaphasia will also be addressed .

14 .1 Category Specificity The first groupof syndromes for the plausibilityof the positionthat the responsible semantic is not unitary but composed of a numberof subsystems are those system so -calledcategoryspecificity . The performance of the patient for some manifesting of knowledge is far betterthan for others . Of particularrelevance is the categories in four with described ( War syndrome originally patients herpes simplex encephalitis ). Thesepatientshad a selective rington and Shallice1984 problem in identifying -madeartefactsmuch animals , plants , and foods , while beingable to identify man . For example better , one of thesepatients , JiB.R., could nameonly 6% of living -madeobjects and 20% of foods but could name 54% of man . Moreover , if the things

534

TimShallice

whether a description of a line drawing of the object " grasped the judges assessed core concept," the contrast was even greater (living things, 6% ; foods, 20% ; but man-made objects, 80% ) . A similar effect was found when the patient was asked to ' as to whether the give the meaning of the object s name and this, too , was assessed man made core concept was grasped(living things, 8% ; foods, 30% ; objects, 78% ) . Similar effectshave now been obtained with other patients with the sameetiology ; Silveri and Gainotti 1988 ; Laurent et al. ; Sartori and Job 1988 (Pietrini et al. 1988 and 1993 Sartori et al. 1993 1990 Swales and Johnson 1992 Sheridan ; ; ; ; Humphreys a rash . in the last few there have been De Renzi and Lucchelli 1994 However , ) years of claims that these dissociations are essentially a result of characteristics of the stimulus set rather than evidencefor a particular type of underlying organization of . the semanticsystem Funnell and Sheridan ( 1992 ) initially claimed that the dissociations might arise because words matched for word frequency as used , say, by Warrington and Shallice ) may not be matched for visual familiarity . Indeed, McCarthy and Shallice(see ( 1984 ) had shown that living things were less familiar to Warrington and Shallice 1984 than artefacts when matched for word frequency. Warrington and Shallice subjects ( 1984 ) had dealt with this problem by showing that the dissociationswere still present when differencesin familiarity were taken out as a covariate. Moreover this explanation does not account for the way that the impairment of the patients involved foods as well as living things, as McCarthy and Shallice found foods to be more familiar than artefacts when word frequency is control led. A stronger argument was presentedby Stewart; Parkin , and Hunkin ( 1992 ), who found that the category-specific dissociation of a herpessimplex patient, H .O., disappeared when word frequency, familiarity , and visual complexity were all control led simultaneously. However, the basic dissociation, while statistically significant, was much weaker in H .O. than in some of the patients describedearlier. Moreover, the , volcano, and waterfall instead nonliving category included objects like swamp, geyser of being composedsolely of artefacts. Most critically , Sartori , Miozzo , and Job , ( 1993 ) usedstimuli matched on thesethree variableswith their patient Michaelangelo who showed a clear and significant category-specific effect of artefacts over living things on two different stimulus sets(living things, 30% and 40% ; artefacts 70% and 76% ) . Yet another possible artifact has been suggested ), by Gaffan and Heywood ( 1993 who argued that a critical variable was the density of exemplarswithin a category, which they held to be greater for living things than for artefacts. Becauseliving things are more similar to each other and so less discriminable than artefacts, any discriminability problem would have a greater effect in the category of living things.

-to-Object TheLanguage Interface Perception

535

Riddoch and Humphreys ( 1987 ) had made a similar point previously and shown that there was more overlap betweenline drawings of animals than betweenline drawings of artefacts. Gaffan and Heywood buttress their position on the difficulty in discriminating betweenliving things, as opposed to artefacts, by considering the identification perfonnance of three groups of subjects using the Snodgrassand Vanderwart ( 1980 ) stimuli . The first group were two patients of Farah, McMullen , and Meyer ( 1991 ), who showedstandard category-specificeffects ; the secondwere nonnal subjects , who, however, were given only a 20 ms exposure and the third used six ; monkeys, who were tested on how well they could decide which of two presenteditems was in a previously trained set. All three groups of subjectsin their very different tasks showed an advantageof man-made objects over living things. " Gaffan and Heywood ( 1993 ) argue These results from monkeys are contrary to Warrington and Shallices conjecture . . . that a specific system for identification of man-made objects has evolved in the human brain ; if Warrington and Shallices conjecture were correct, monkeys would show relatively greater difficulty in discriminating among inanimate objects than among living things, compared to human observers ." It is not apparent, however, how such a comparison can be made because the tasks carried out were so different. Moreover, for the monkeys, most of the stimuli would presumably be meaninglessobjects; therefore what should be critical would indeed be raw discriminability . If , however, discriminability were a key factor underlying the perfonnance of both the monkeys and the patients, then one would expect a positive correlation within each of the living and nonliving sets of stimuli between the results of the two group of subjects . In fact, there was no correlation betweenthe items the monkeys found difficult and those the patients found difficult in either the living or the nonliving sets . ' Gaffan and Heywood s work , like that in the other critical studies , used the Snod) stimuli , for which nonns are available on a number of grassand Vanderwart ( 1980 relevant variables. In this set of stimuli the animals, in particular , tend to be rather similar to other membersof their category. Warrington and Shallice( 1984 ), however, also used the so-called Ladybird stimuli , large clear colored pictures designed for preschool children, with three of their patients. Shallice and Cinan have obtained ratings of structural complexity, familiarity , and discriminability from nonnal subjects for the Ladybird stimulus set and used theseto reanalyzethe findings of War rington and Shallice. With these ratings, no difference was found betweenall three categories of stimuli (animals, artefacts, foods) for either familiarity or discriminability , but the animals remained structurally more complex than the other two categories . Becausethe task the patients carried out with this stimulus set had involved

536

TimShallice

word -picture matching using a four -alternative forced-choice task, the relevant degree of discriminability on the Gaffan-Heywood hypothesis was that within each set of five; this is what the subjects of Shallice and Cinan rated. However, with these stimuli two of the three original Warrington and Shallice patients on whom the test had been used performed significantly more poorly on foods than on artefacts with the third showing a strong trend in the same direction. Moreover, on a regression analysis using the ratings obtained by Shallice and Cinan, all three patients showed a significant effect cf category and no effect of the other three variables. Thus it would appear that thesecategory specificity findings cannot just be reducedto somecombination of differencesin word frequency, visual familiarity , structural complexity, and within -category discriminability . In this respect , the work of Shallice and Cinan corroborated an earlier finding of Farah, McMullen , and Meyer ( 1991 ), who used the Snodgrassand Vanderwart ( 1980 ) stimuli with two patients exhibiting the standard category-specific dissociations . In a regressionanalysison picture recognition performance, Farah, McMullen , and Meyer showed that neither name frequency, name specificity, similarity to other objects, structural complexity, nor object familiarity had any significant effect. The . The absenceof a significant only factor to have such an effect was category membership effect of other factors in the presenceof a significant effect of category makes implausible even one final convoluted artifactual explanation put forward by Gaffan and Heywood ( 1993 ) . These authors suggestedthat the category difference arises through performance on items differing in a way dependentupon someother dimension ; following Snedecor and Cochran ( 1967 ), they pointed out that measurement errors on the other dimension can lead to an apparent difference in performance acrosscategorieseven when the differenceson the other variables are allowed for as a covariate. However, what would then be expectedis that there would be a basic effect of some other dimensions ; this was not in fact found in either study. Thus it would appear that the basiccategory-specificeffectscannot be reducedjust to an artifact of somecombination of differencesin word frequency, visual familiarity .A , structural complexity, and within - category discriminability acrosscategories secondtype of finding that supports the conclusion that all neuropsychologicaldissociations in this domain cannot simply be attributed to some artifact of differencesin , namely a presemantic factors is the existenceof the complementary phenomenon in in and two studies foods) some of subjects living things ( superior performance over artefacts ( Warrington and McCarthy 1983 , 1987 ; Hillis and Caramazza 1991 ; Sacchettand Humphreys 1992 ). The first two studies involved global aphasicswho could only be tested by word-picture matching using, for instance , the Ladybird stimuli discussedabove. However, the subjectsin the last two studies were not glob-

-to -Object Perception Interface The Language

537

, C.W . ; thus naming to visual confrontation could be used (for instance ally aphasic in Sacchettand Humphreys 1992scored 19 / 20 on naming animals; but only 7/ 20 on ' naming artefacts). Interestingly, the location of C.W . s lesion (left frontoparietal ) differed from that characteristic of the herpes simplex encephalitis cases(for all of whom the left temporal lobe was involved) . Much the most plausible conclusion is that the category-specificeffectsdo not arise at a presemanticlevel due to some difference in difficulty betweenthe categoriesbut . reflect some qualitative difference in the semantic representationsof the categories in it was terms When the herpesencephalitis syndrome was first described , explained of a contrast betweenstimuli primarily differentiable in terms of their sensoryqualities and those more saliently differentiable in terms of their function . -madeobjects . Theevolutionary functions haveclearlydefined Unlike mostplantsandanimals , man finer functional differentiations of tool has led to finer and of using development . Individualinanimate havespecific functions artefacts for an increasing objects rangeof purposes . Consider for activitiesappropriateto their function and are designed , for instance , and but and are all used for chalk , crayon , ; they drawing writing, they havesubtly pencil . . . . Similarly differentfunctions , , jar, jug, and vaseare identifiedin termsof their function features of each canvaryconsider , but thesensory , to hold a particulartypeof object namely , functional attributescontributeminimally to the identificationof living ably. By contrast attributesprovidethe definitivecharacteristics e. . lion , tiger, andleopard ), whereas sensory things( g , ice 1984 and Shall , 849 , or spotted ) ). ( Warrington (e.g., plain, striped A closely related position was taken to explain the complementary syndrome to be .) discussedlater (seeWarrington and McCarthy 1983 Dector , Dub, and Chertkow (in press ) take a somewhat related position basedon their study of a patient, E.L .M ., who suffered from bilateral temporal lobe strokes. On tests of perceptual knowledge of objects he performed normally , but he was grossly impaired at many tests involving the perceptual characteristics of animals. Dector , Dub, and Chertkow argue that the difference between the superiority of artefacts over animals arises becausedifferent tokens of the sameman-made object may show a considerablevariation in the shapeof its parts but a consistent function that allows for a unique interpretation , thus echoing the Warrington -Shallice position . However, they then argue that artefacts " can be uniquely identified at the basic level through a functional interpretation of their parts" and this is why they are relatively preserved(seeDe Renzi and Lucchelli 1994for a related position) . Many artefacts with a unique function do indeed have a unique organization of distinctly functioning parts; take a lamp, for example. However, others, such as a table tennis ball , do not. As yet it remains unclear to what extent the relative sparing of artefacts depends upon their unique organization of distinctly functioning parts or on the unique functions of the whole.

538

Tim Shallice

14 .2 Sensory of Dift'erentCategories Quality andFunctional Aspects


The position just developed attributes differences in performance across different categories to the way that identification in some categories depends critically on sensory quality information but for others functional information is more critical . One can, however, consider how well different semanticaspectsof the samecategory are understood by patients who show this category-specific pattern . When this is done, knowledge of functional aspectsof biological categories tends to be much better preserved than knowledge of sensory quality aspects (Silveri and Gainotti ' 1988 ) . In a related fashion, Dector, Bub, and Chertkow s (in press ) patient E.L .M . " " was much better at answering encyclopedic questionsabout animals such as " Does a camel live in the jungle or the desert ?" (85% ) than visual ones such as " Does a ' " camel have horns or no horns ? where he was at chance(55% ) . However, the effects are not completely clear-cut. The performance of E.L .M ., say, on functional aspects of animals was still well below that of normal controls, who scored 99% . This was not due just to a general problem with carrying out semanticoperations on concrete objects; when asked to identify artefacts he performed at ceiling. A more dramatic example is given by De Renzi and Lucchelli' s ( 1994 ) herpes encephalitic patient, Felicia. In explaining the perceptual difference betweenpairs of animals, for example , goat and sheep , or paired fruits or vegetables , for example , and she far worse than the worst controls cherry strawberry, performed ( 15% vs. 90% ; 49% vs. 85% ) . However, in explaining the visual difference between paired , lightning rod and TV antenna, she was somewhat better than objects, for example the normal mean (90% vs. 85 % ) . Analogous results have been reported in a number of other studies (e.g., Silveri and Gainotti 1988 ; Sartori and Job 1988 ; Farah et al. 1989 ) although at least one patient, Giuletta (Sartori et al. 1993 ), answerednonvisual questions about animals almost perfectly (seealso Hart and Gordon 1992 ), while at the other extremeS.B. (Sheridan and Humphreys 1993 almost as poorly ) performed on visual as on nonvisual questionsabout animals (70% and 65% , respectively ). Why should the category specific impairment generally recur if in a milder form when the patient is responding to questionsabout animals or foods which appear not to be basedon accessing sensoryqualities? Does it not undermine the explanation of effects outlined earlier, namely, that they arise from damageaffecting category specific ? If one articulates the theory developedthus far sensoryquality representations in a connectionist form , then the problem can be resolved . Farah and McClelland .1) in which somesemanticunits represented ( 1991 ) investigateda model (seefigure 14 the functional roles taken by an item, while others representedits visual qualities. Each of the semantic units was connected (bidirectionally ) to the others, to units representing structural descriptions, and to units representing phonological word -

- to-Object Perception Interface The Language


VISUAL FUNCTIONAL

539

.1 Fig8re14 ' s 1991modelfor -specific of artefact Farahand McClelland ( ) category preservation explaining from Farah and McClelland1991 and naming(reproduced ). by permission comprehension
fonDs . The number of units in the two subsets of semantic representations was deter -

. Subjectsrated the description of mined through an experiment on normal subjects in definitions of both and each item nonliving things in the American Heritage living Dictionary as to whether it described the visual appearanceof the item, what the item did , or what it was for . On averagethere were 2.13 visual descriptions and 0.73 functional ones , but the ratio betweenthe two types was 7.7: 1 for living things and 1.4 1 : for the nonliving things. Thesevalueswere then realized in the representations only of living things and artefacts used for training . The network was trained using an error correction procedure based on the delta rule (Rumelhart, Hinton and McClelland 1986 ) applied after the network had beenallowed to settle for ten cycles following presentation of each input pattern. In each of four additional variants of the basic network , one particular parameter was altered so as to establish the robustnessof any effect obtained. The most basic finding was that lesioning the " visual" semanticunits led to greater impairment for living things than for artefacts with the opposite pattern shown for the lesioning of the functional semantic units. Thus the standard double dissociation was obtained due to " identification " of living things relying more on the visual

VISUAL

PERIPHERAL INPUT SYSTEMS


VERBAL

I I I I I I I I I SYSTEMS SEMANTIC

540

Tim Shallice

semantic units and " identification " of artefacts depending more on the functional semanticunits. More interestingly, if one examineshow closea match occursover the functional semanticunits when a lesion is made to the visualsemanticunits, then there is a difference between the two types of item. The functional representationsof the living things were less adequately retained than those of artefacts. In the original " , the attainment of the representation in one of the semantic subsystems learning process " helps to support the learning of the complementary representation in the other; the richer the representationis in one of the systems , the more use is made of it in learning the complementary representation . Thus the most typical relation between functional and visual impairments with living things is explained. Whether the full range of relations observedcan be explained remains to be investigated . There are two uncomfortable findings that the model would appear well designed to explain. First , the living /nonliving distinction is not absolute. Thus Y .0 .T . was one of the global aphasic patients who performed very much better on word -picture matching with living things and foods than with artefacts( Warringtonand McCarthy 1987 , the impairment did not extend to large man-made objects ) . In Y .O.Tis case such as bridges or windmills . Patient J.J. of Hillis and Caramazza( 1991 ), who had a selective sparing of animal naming like Y .0 .T ., also had the naming of means of transportation spared. Complementarily, the problems of herpes encephalitic patients extendedto gemstonesand fabrics. The semantic representationof all these , especially subcategories may well consist more of visual units than of functional ones if function has to be linked with a specificaction. Second , the living/nonliving distinction is graded. Thus patients have been described in whom the deficit is limited , say, to animals alone (e.g., Hart , Berndt, and Caramazza 1985 ; Hart and Gordon 1992 ) . The sensory quality/ function contrast would seemlikely to be more extreme for animals than foods, say, so that for more minor damage to sensory quality units only the least functional of the semantic categorieswould be affected. Overall, this group of category-specificdisorders fits with the idea that knowledge of the characteristicsof objects is basedon representationsin more than one type of . Realizing the different systemsas setsof units in a connectionist model allows system certain anomaliesin the basic subsystemapproach to be explained. The nature of the mediatedby eachof the systemsremainsunclear, however . The deficit representations appearsnot to correspond simply to damageto visual units. Thus one of the patients studied by Warrington and Shallice ( 1984 ) was unable to identify foods by taste as well as by sight. Moreover, in three of the patients where it has been assessed ; E.L .M . in Dector , Bub, and Chertkow , in (Michaelangelo in Sartori and Job 1988 in ; and SiB. Sheridan and Humphreys 1993 press ), relative sizejudgments could be

-to -Object Perception Interface The Language

541

made fairly accurately , suggesting that even the visual deficit does not extend to all

visualcharacteristics remaIns open . The issue . 14 .3 Optic Aphasia


A secondsyndrome that suggests the needto refine the conceptual / structural description contrast of Jackendoff ( 1987 ) is optic aphasia. First describedby Freund ( 1889 ), optic aphasia refers to an impairment where the patient is unable to name objects presentedvisually but at the sametime givesevidenceof knowing what theseobjects are, for instance , by producing an appropriate mime. Moreover, the problem is not one of just naming; the patient is able to produce the name to a description or on or tactile presentation. A considerable number of patients have been described auditory who roughly fit the pattern (seeBeauvois 1982 ; Iorio et al. 1992 ; Davidoff and De Bleser 1993for reviews ). If one limits consideration to patients who do not appear to have any impairment in accessing the structural description because stimulus quality doesnot affect naming ability , Davidoff and De Bleser( 1993 ) list fourteen patients who have been formally described . Certain of thesepatients performed perfectly in gesturing the use of visually presentedstimuli they could not name (Lhennitte and Beauvois 1973 ; Capian and Hedley-White 1974 ; Gil et al. 1985 ). This apparent preservation of knowledge of the visually presentedobject when it cannot be named has beenexplained most simply by assumingthat the optic aphasic " and " verbal " suffers from a disconnection between " visual semantics semantics , with the name only being accessiblefrom verbal semantics( Beauvois 1982 ; Shallice 1987 at the semanticlevel appearsto differ from ) . The distinction betweensubsystems the one drawn in the previous section betweensystemsrepresentatingfunctional and visual or sensoryquality types of information . I will addressthis issuein more detail later. In any case , a number of authors have contested the claim (seeRiddoch and 1987 ; Garrett 1992 Humphreys ; Rapp, Hillis , and Caramazza 1993 ), holding that the miming could simply be basedon an affordance, that is, an action characteristically -modal associationof sensoryand motor induced by the shapeof the object, or a cross schemas , either of which might in turn be basedonly on an intact structural description . Alternatively , miming might require accessingonly restricted parts of the semantic , in particular those parts most strongly realized from the structural system description because they are also representedin it explicitly , for example , the tines of forks ; this is the privileged accesstheory account of Caramazza et al. ( 1990 ) and Rapp, Hillis , and Caramazza( 1993 ) . A similar explanation might also be given for the preserveddrawing from memory shown in patients such as J.F. ( Lhennitte and Beauvois 1973 ).

542

TimShallice

However, accessto other types of infonnation can be present in these patients ) gave their patient when they cannot name. For instance , Coslett and Saffran ( 1992 ) in which the EM2 a task based on one devised by Warrington and Taylor ( 1978 for similar are items of three which , example, zipper, functionally patient has to judge scoredat 97% . EM2 1989 ) button , coin (seealso patient C.B. in Coslett and Saffran on this task, with the control mean being 94% . Becausethe affordancesof a zipper and a button are not similar , it is difficult to seehow the use of affordancesmight be the basis for this good perfonnance; indeed, there are no subcomponentsof the two ), in confronting structural descriptions that are related. Rapp, Hillis , and Caramazza( 1993 a the argument that such a pattern of perfonnance presents difficulty for " their privileged access ), merely respond by saying, difficulty position (Shallice 1993 naming visually presenteditems in the face of demonstratedintact comprehensionof someaspectof the visual structures, however, indicates that the full semanticdescription required to support naming has not beenactivated from a 3 D representationof " that nonnal perfonnance on the function the stimulus. This argument presupposes matching test can be obtained when activation of the relevant semantic representation . . This claim is merely assertedby Rapp, Hillis , and Caramazza is reduced basic with rather test , However, becausethe task is a three alternative forced choice semantic infonnation being required about each item- concerning its function the assertionhas someplausibility . ) on Similar results have, however, beenobtained by Manning and Campbell ( 1992 Two . more much to be demanding patient A .G. on semantic tasks which appear and Palm Trees was the first . The with these were used test of Pyramids patients types test of Howard and Patterson ( 1992 ) . In a typical item of this test, the patient has to with a pyramid . The stimuli can be presented best fir tree decide which (palm, ) goes either visually, verbally, or in mixed visual-verbal fonnat . In the second test, the patient has to answer sets of questions about each item, (e.g., What is it made of ?) both when the item is presentedvisually and when it is presentedauditorily . A .G. % in perfonned at only 40% 50% in naming objects from drawings, but at 100 thus stimuli , showing naming to description and at 91% in naming tactilely presented a specific naming defect with visual stimuli . However, A .Gis perfonnance on the Pyramids and Palm Trees test, while not at ceiling, was virtually identical acrossthe visual and verbal modalities of presentation (82% vs. 84% ) and in both caseswas . A similar within one standard deviation of the mean of nonnal control subjects . vs. 91% ) Druks and pattern was observed for the question answering test (88% Shallices ( 1995 ) patient LiE .W . behavedin the sameway for both types of test. That no differenceand were not at ceiling on tests of auditory and verbal showed patients ' comprehension seemsimpossible to account for in Rapp, Hillis , and Caramazzas

- to -Object Perception Interface The Language

543

( 1993 ) version of the privileged access . By theory, which involves a unitary semantics contrast, theseresults fit well with the multiple semanticsystemposition . Coslett and Saffran ( 1992 ), on the other hand, presentan interesting variant of the store . multiple position They agreethat two semanticstores do exist and that one is disconnected from the language production mechanismsin optic aphasic patients, but they argue that the stores are primarily distinguished by hemisphere , with the right -hemispheresemantic systembeing disconnectedfrom the languageproduction . However, the patients described by Manning and systemsin the left hemisphere 1992 a Campbell ( ) present difficulty for this position . In the acute condition immediately after a suddenonset lesion (e.g., vascular), the right hemisphereis supposedby right -hemisphere theorists such as Coslett and Saffran not to have accessto any phonological lexicon, although they hold that over time a phonological lexicon becomes available to a semantic system in the right hemisphere(Coslett and Saffran 1989 . This semantic systemor the variety of output phonological word -forms that ) can be accessed from it is then seento have an effectivecontent corresponding to that of the words readable in deep dyslexia (Coltheart 1980a ; Saffran et al. 1980 ; Coslett and Saffran 1989 . In however ) deepdyslexia, , concrete nouns can be read reason ably well but verbs present severeproblems (Coltheart I 980b) . Yet while patients A .G. and LiE .W . were severely impaired in naming objects, which they could identify nonverbally, they could name actions very well. Thus A .G. was 95% correct at naming actions- the samelevel as controls- but worse than 50% at naming objects. This contrast in easeof accessingoutput phonological word -forms from an intact semantic representation is the opposite of what would be expectedaccording to the right -hemispheretheory, where one would assumethat objects should be more easily nameable than actions. The basic multiple semantic store position can perhaps explain the obtained effect by assumingthe existenceof another semanticsubsystem one controlling actions (Druks and Shallice 1995 ); being an essentially high-level output system but accessiblefrom perceptual input , it would have connections to verbal semanticsdistinct from those used by the visual semantic representationsof objects. This, however, remains a highly speculativeaccount. There remains one other counterintuitive aspect of optic aphasia. Many of the patients characterizedas optic aphasicthrough their pattern of success and failure on naming and comprehensiontestsexhibit a strangeset of errors when they fail to name correctly . Of the optic aphasic patients reviewedby Iorio et al. ( 1992 ), who generally ' correspond with Davidoff and De Blesers ( 1993 ) group 2 optic aphasics , nearly all made both semanticand perseverativeerrors, with lessthan half also making visual errors. Moreover, in the most detailed analysisof sucherrors- that of L hermit teand Beauvois ( 1973 ) of their patient J.F.- the authors consider the interaction between

544 Table 14.1 Errors Made by J. F . in Two Experiments Type of error Example 100 pictures

Tim Shallice

30 objects

Horizontal errors Semantic Visual Mixed visual-and-semantic Vertical errors Item and coordinate perseveration

" shoe=- " hat " " coffeebeans=- hazel nuts " " orange=- lemon " T26 . . . =- " wristwatch =- " wristwatch" T27 scissors " " T44 . . . ~ newspaper " " T45 case=- two books " T43 . . . =- " chair " " T47 basket=- cane chair " T53 string =- strand of " weavedcane

9 2 6

3 1 I

Mixed horizontal/ vertical errors

. : Lhennitte and Beauvois 1973 Source

" es(temporally what they call " horizontal errors, understood strictly in terms of the process the and the stimulus of , and between responses presentation ) intervening " " occur. or stimuli of effects where responses what they call vertical errors, preceding It is clear from this analysis that the perseverativeand the semantic errors combine in a complex way (seetable 14.1) . ? Why might such a strangecombination of errors be characteristic of optic aphasia the to dimension a connectionist be given by adding Again a possible answer can pathway models. Plaut and Shallice ( 1993a ) considereda network which had a direct " " a had It also . ones into semantic cleanup pathway visual representations mapping " " that involved recurrent connections from the semantic units to the cleanup units and back (seefigure 14.2) . The network used an iterative version of the backpropagation learning algorithm known as backpropagation through time (Rurnelhart, Hinton , and Williams 1986 ) . Training with an algorithm of this type in such a recurrent network leads to its developing a so-called attractor structure; the effect of the operation of the cleanup pathway is to move a noisy first -passrepresentation at the semanticlevel toward one of the representationsit has beentrained to produce as an output , given that the initial representationis in the vicinity of the trained one. The network contained one other major difference from other networks well ' ) . In known in cognitive psychology, such as Seidenbergand McClelland s ( 1989

-to -Object Perception Interface The Language


C ~

S4S

V~ I

. .2 Figure14 allices ( 1993 Plaut and Sh ) model for explainingthe typical error pattern found in optic 1993a ). from Plautand Shallice by permission (reproduced aphasia

occur at in synapticefficiencyat a single synapse the nervoussystem , changes connection of additional . The 1979 incorporation ) (Kupferman manydifferenttime scales those than in standardly weightsthat changemuch more rapidly training valuable ; it allowsfor temporal is alsocomputationally usedin connectionist modeling ) and into a whole(e.g., von der Marlsburg1988 elements bindingof neighboring and McClelland in described communication Hinton facilitatesrecursion , personal ( a standard combined , therefore network in the Kawamoto 1986 ) . Eachconnection based term weight , short , long term weightwith a rapidly altering slowly changing the activitiesof its input and output units. on thecorrelationbetween A network havingboth typesof weightstendsto reflectin its behaviorboth its the analogue activity; it contains historyand its mostrecent long-term reinforcement to respond trained was of both long term learningand of priming. The network of forty different levelto the structuralrepresentations at the semantic appropriately , it produceda few visual errors but . Whereverthe network was lesioned objects and errors consider typicallymorewith both visualand semantic ably moresemantic to aspect . More critically, therewasa strongperservative similarity to the stimulus well could associates or one of its semantic . The previousresponse the responses to the error pattern well This . stimulus to the error corresponds an occuras present . occurringin optic aphasia allowsthe error pattern to the modeltherefore dimension Adding a connectionist model we usedas a . The informationprocessing to be explained of the syndrome to thoseof Riddochand Humsimulations corresponds basisfor the connectionist et al. ( 1990 ), which wereheld to be unsatisfactory ) and Caramazza phreys( 1987 - and of the simulationis that if short . However , the essence earlierin this chapter influences , the errorswill reflectboth perseverative long-term weightsarecombined .1Thus the obtained at of level the and representation whichstrongattractorsoccur were dimension connectionist if an analogous error patternwould alsobe expected

C~ S

S~ I

units

semantic

86 units up

clean 40

546

Tim Shallice

addedto the multiple semantic system models , provided that one or more of the
semantic systems had analogous attractor properties . 14.4 Conclusion

In the sections 14.1 and 14.2 certain syndromes were discussedinvolving category, specific impairments, particularly those associatedwith herpes simplex encephalitis made arte of man identification where large differencesin performanceexist between facts on the one hand and of living things and foods on the other. Explanations in terms of differencesbetweenthe categorieson a number of potentially confounding that partially . The favored explanation assumes dimensionswere consideredand rejected and of the functional of semantic representations separablesystemsunderlie the the sensory quality properties of stimuli . In section 14.3 another syndrome optic - was considered ; here it was argued that the most plausible explanation aphasia " " " " " . involved disconnecting" visual and verbal or lexical semanticrepresentations that a the view for difficulties The evidencepresentedin all three sections poses , together with a structural description systemthat can also single conceptual system from above, is a sufficient material basefor representingsemanticoperations be addressed . The sensory quality component of the semantic systemcannot be conftated with the structural description system becausevariables relevant to disorders of the latter system , presentation of items from unusual views ( Warrington , for example and Taylor 1978 ) , do not predict the stimuli that are difficult for patients with ; Dector , Bub, and impairments to the former system( Warrington and Shallice 1984 of from the clearer is even . The issue Chertkow in press perspective the secondset of ) disorders. In certain optic aphasicpatients much more semanticinformation appears from vision than could be basedon the structural description alone; to be accessible would it appear not to be available in a generally accessibleconceptual system yet it cannot be used to realize naming. because By contrast, the accounts presented for these disorders fit naturally with those beginning to be developedwithin developmentalpsychology for image schemasat a level of abstraction higher than the structural description and yet not simply subsumable within verbal knowledge (seeMandler , chapter 9, this volume) . However, to esshould be limited to what is required for argue that the such visual semanticprocess ' visual identification alone in Chertkow and Bub s ( 1990 ) visual identification procedure between the structural the this is that and lying only system subsystem not fit well for either does semantic core an amodal and system description system the sensory quality what is lost are condition syndrome. In the herpesencephalitis , according to Miller and Johnsonaspectsof the item, while identification procedures Laird ( 1976 ), require primarily functional property information as well as structural

-to-Object Perception Interface The Language

547

, one possibility to explain the syndrome might be analysis. Turning to optic aphasia to view it as arising from a disconnectionbetweenthe visual identification procedures and the core semanticsystem . However, a task like Pyramids and Palm Treesinvolves the utilization of shared context. The Bub and Chertkow theory holds that inferred context is stored in the amodal core semanticsystem , so that an optic aphasicwould be not expectedto perform well on such tasks for words that could not be named. Patients A .G. (Manning and Campbell 1992 ) and LiE .W. (Druks and Shallice 1995 ) show the opposite pattern, namely, intact performance on this task, together with grossly impaired naming. There are, however, certain problems in explaining the two types of syndrome in terms of the functional /sensoryquality and visual/ verbal dichotomies. The concepts are orthogonal . The information available in a visually or sensory quality - based semanticsystem , as inferred by the information lost in the herpesencephaliticpatient is not the only information accessible from the visual modality in the optic aphasic . Certain , A .G. and LiE .W., can access patient optic aphasic patients, for example types of information from vision that would be in the functional or encyclopedic parts of the semantic systemon a simple all -or -none multiple store view. Moreover, within the semantic dementia literature there are striking echoesof this visual input predominance extending outside the purely sensory quality domain in the performance 2 of patient T .O.B. (McCarthy and Warrington 1988 ). When a picture was presentedto TO .B. his identification was more than 90% accurate for both types of material, but he identified verbal input artefacts much better than living things (89% " vs. 33% ) . Thus when the word dolphin was presented , the patient could say only, A " " fish or a bird , but when presentedwith the picture, he said, Livesjn water . . . they are trained to jump up and come out. . . . In America during the war they started to " get this particular animal to go through to look into ships. McCarthy and War rington have argued that this patient has an impairment that affects the stored information itself rather than an input pathway becauseof the consistency with which particular items were or were not identifiable (see for rationale Warrington and Shallice 1979 ; Shallice 1987 ) . Thus contrasting both optic aphasia and semantic dementia with herpes simplex encephalitis , it would appear that the putative lines of cleavagewithin the semanticsystemsuggested by the syndromesdiffer. One possibility is to postulate category specificsystemsthat are themselves specific to particular modalities (McCarthy and Warrington 1988 ) . However, explanations provided for certain secondaryaspectsof the syndromessuggestan alternative direction in which a more economical solution might lie. A connectionist simulation of Farah and McClelland ( 1991 ) can account for certain otherwise most recalcitrant , the counterintuitive findings about category specific disorders. For optic aphasia error pattern associatedwith the disorder is in turn explicable on a connectionist

548

TimShallice

) . Thus adding a connectionist dimension to simulation of Plaut and Shallice ( 1993a the theoretical framework used to account for the characteristics of the syndromes enablesa much fuller explanation of the detailed nature of the deficits to be provided . approach provides an account Adding sucha connectionist dimension to a subsystem the last ten years or so, that over made closely related to presimulation suggestions neural network with associative the semanticsystemhas as its material basis a large , depending different concepts being representedin different combinations of its subregions on the specific subsetof input and output systemsgenerally used to address ; and Saffran ; Shallice 1988b them (seeAllport 1985 ; Warrington and McCarthy 1987 semantic of processingwould be and Schwartz 1994 ) . How the rule governedaspects . dealt with on this type of account has not been addressedby neuropsychologists However, the use of a connectionist network framework for explaining aspectsof ruleneuropsychological disorders does not preclude the possibility of explaining , provided additional elementsare added to governed aspectsof semantic processing ; and Miikkulainen ; Derthick 1988 the basic network (seeTouretsky and Hinton 1988 1993 /conceptual system postulated by Jackendoff ) . On this account the semantic a as realized be would needto complex neural network. As yet, though, no implementation adequatelyexplains the rich and highly counterintuitive evidencethat detailed study of individual neurological patients provides.
Notes

level is not the caseif the mappingfrom the visual to the semantic I . This is especially of visual for I 993a Shallice and Plaut see presentation ); ( , as it is in language orthogonal . correlated are semantic the and visual the representations , objects
2. A simple peripheral explanation of the phonological word -form being damagedcan also be excluded.

References . In S. K. anddysphasia subsystems , modular memory ). Distributed , D. A. (1985 Allport ill : . in EdinburghChurch .), Current and R. Epstein Newman perspectivesdysphasia (Eds . stone Living . andlanguage vision between of interaction : A process Beauvois ). Opticaphasia , M. F. (1982 47 . 33 B298 London the , , , Transactions Society Royal of Philosophical and of pictures encoding , A. ( I 988 ). Semantic S. Bub, D., Black , E., and Kerkesy , , Hampson
. Cognitive , 5, 27- 66. observations Neuropsychology : Some words neuropsychological in alexiawithout -White, T. ( 1974 ). Cueingand memorydysfunction , L., and Hedley Capian . 262 251 97 . Brain : A case , , report agraphia

-to -Object Perception Interface The Language

549

: deficit hypothesis Caramazza , H. H. ( 1982 , R. S., and Brownell ). The semantic , A., Berndt . BrainandLanguage , 161 , 15 patients by aphasic Perceptual parsingandobjectclassification 189 . Caramazza , C. ( 1990 hypothesis ). Themultiplesemantics , B. C., Romani , A., Hillis, A. E., Rapp - 189 . ? Cognitive : Multiple confusions , 7, 161 Neuropsychology dessignes et et isolee la visionmentale Charcot , J. W. ( 1883 brusque ). Un casde suppression - 571 . Medical et couleurs desobjets(formes , 568 , 11 ). Progres 's . of Alzheimer Chertkow type , H., and Bub, D. ( 1990 ). Semantic memorylossin dementia 417 . Brain , 397 , 113 . In M. Coltheart : A right hemisphere Coltheart , K. E. , M. ( 1980a hypothesis ). Deepdyslexia . . London : . Marshall Eds and J. C. Patterson Routledge , ), Deep dyslexia ( . In M. Coltheart : A . Coltheart , K. E. , M. ( 1980b ) Deepdyslexia reviewof the syndrome . . London : Routledge .), Deep Patterson , andJ. C. Marshall(Eds dyslexia andreading Coslett , EM . ( 1989 comprehension , H. B., andSaffran ). Preserved objectrecognition - 1110 . . Brain in optic aphasia , 112 , 1091 : Replication and the right hemisphere Coslett , EM . ( 1992 , H. B., and Saffran ). Opticaphasia 161 . 43 148 . BrainandLanguage andextension , , : A review of past studiesand a R. and De Bleser Davidoff J. , ). Optic aphasia , , ( 1993 . 7 135 154 . , , reappraisalAphasiology : of objectconcepts Dector , H (in press ). Multiple representations , M., Bub, D., andChertkow -specific . . Cognitive from category Evidence Neuropsychology aphasia in the De Renzi , F. ( 1994 separately represented , E., and Lucchelli ). Are semantic systems . Cortex . of living category brain ? Thecase impairment ., . PhiD. diss Derthick , M. ( 1988 ). Mundanereasoning by parallel constraint satisfaction . Mellon , Pittsburgh University Carnegie of visualidentificationand action namingin T. Druks J. and Shallice , ( 1995 , , ). Preservation Conference Annual British at the . , Society Neuropsychological Paper presented optic aphasia London , March. Farah , G. ( 1989 , Z., and Ratcliff ). Categoryspecificity , K . H., Mehta , M. J., Hammond - 200 . . Neuropsychologia in semantic , 27, 193 memory modalityspecificity Farah memory , J. L. ( 1991 ). A computationalmodelof semantic , M. J., and McClelland . Journalof Experimental and : emergent categoryspecificity impairment Modality specificity - 357 . : General , 120 , 339 Psychology of living thingsbe Farah , MM . ( 1991 , P. A., and Meyer ). Can recognition , M. J., McMullen - 194 . ? Neuropsychologia , 29, 185 selectively impaired und . ArchivfUr Psychiatrie Freund , D. C. ( 1889 Aphasieund Seelenblindheit ). ()ber optische ' . 276 297 2O Nervenkrankheiten , ,

550

TimShallice

of knowledge ? Unfamiliaraspects of living and Funnell , E., andSheirden ,J. ( 1992 ). Categories - 153 . . Cognitive , 9, 135 Neuropsychology nonlivingthings -specific visualagnosia for living GaITan , C. A. ( 1993 , D., and Heywood ). A spurious category . Journalof Cognitive Neuroscience , 5, 118 primates thingsin normalhumanand nonhuman . 128 - 180 . Cognition of lexicalselection . Garrett , M. ( 1992 , 42, 143 ). Disorders Gil, R., Pluchon , D., Rogew , C., Toullat, G., Michenau , R., Lefevre , J. P. ( 1985 ). Disconnexion -verbale visuo , lesimages , lescouleurs , et lesvisages ) pour lesobjects (aphasie optique - 349 avecalexie abstractive . Neuropsychologia . , 23, 333 -specific Hart, J., Berndt , A. ( 1985 , R. S., andCaramazza ). A category namingdeficitfollowing . Nature . cerebral infarction 316 439 440 , , . B. ( 1992 . Nature Hart, J., and Gordon for objectknowledge , 359 , 60- 64. ). Neuralsystems : A testof semantic andpalm trees access Howard , F., Patterson , K. E. ( 1992 ). Pyramids from . and words . Thames pictures Valley Iorio, L., Falango and , A., Fragassi , N. A., and Grossi , D. ( 1992 ). Visualassociative agnosia : A single . Cortex case , 28, 23- 37. optic aphasia studyanda reviewof the syndromes Jackendorff : The relation of linguisticand visualinformation . , R. ( 1987 ). On beyondzebra - 114 . , 26, 89 Cognition . AnnualReview , I. ( 1979 ). Modulatary actionsof neurotransmitters of Neuroscience Kupferman - 465 . , 2, 447 -Faure Laurent , B., Foyatier , N., andPellat , , B., Allegri, R. F., MichelD ., Trillet, M., Naegele : au a unilaterale Etude J. ( 1990 ). Encephalites herpetiques predominance neuropsychologique - 681 . Revue . , 146 , 671 longcoursde9 cas Neurologique L hermit : Reportof te, F., and Beauvois disconnexion , M. F. ( 1973 ). A visualspeech syndrome - 714 with optic aphasia . Brain . a case alexia , 96, 695 , agnosic , andcolouragnosia with spared actionnaming : A description , L., and Campbell , R. ( 1992 ). Optic aphasia Manning - 592 . Neuropsychologia . and possible loci of impairment , 30, 587 . SanFrancisco : Freeman . Marr, D. ( 1982 ). Vision -specificmeaning for modality , E. K. ( 1988 , R. A., and Warrington ). Evidence McCarthy - 430 in the brain. Nature . , 334 , 428 systems of sentence : Assigning McClelland , J. L., andKawamoto , A. H. ( 1986 ). Mechanisms production of sentences . In J. L. McClelland andDE . Rumelhart .), Parallel rolesto constituents (Eds - 325 . Vol. 2, 272 . distributedprocessing : Explorationsin the microstructure of cognition MA: MIT Press . , Cambridge -role analysis with embedded clauses . Miikkulainen case , R. ( 1993 ). Subsymbolic of sentences -202 . Austin: Universityof Texas Press . Technical reportAI 93 -Laird, P. N. ( 1976 . Cambridge Miller, G. A., and Johnson and perception : ). Language . Cambridge UniversityPress

Tim Shallice

Shallice within the semantic . Cognitive , T. ( 1988b , 5, ). Specialization system Neuropsychology - 142 . 133 : Whoseconfusions ? Cognitive Shallice , T. ( 1993 , , 10 ). Multiple semantics Neuropsychology - 261 251 . -semantic Sheridan , J., and Hymphreys , G. W. ( 1993 ). A verbal category specific recognition . Cognitive 10 143 184 . , , impairment Neuropsychology Silver i , M. C., and Gainotti, G. ( 1988 visionand language ). Interactionbetween incategory - 709 semantic . Cognitive . , 3, 677 specific impairment Neuropsychology Snedecor . 6th ed. Ames : Iowa State , G. W., and Cochran , W. G. ( 1967 ). Statisticalmethods Press . : Normsfor setof 260pictures , J. G., and Vanderwart , M. ( 1980 Snodgrass ). A standardized name and visual . Journal , , agreement image agreement familiarity, complexity of Experimental - 215 : HumanLearning andMemory . , 6, 174 Psychology Stewart , F., Parkin , A. J., and Hunkin, N. M. ( 1992 ). Namingimpairment followingrecovery -specific from herpes : Category ? QuarterlyJournalof Experimental simplexencephalitis Psychology - 284 . , 44a , 261 Swales with semantic loss : Cantheyrelearn lost , M., andJohnson , R. ( 1992 ). Patients memory '295 ? Rehabilitation 2 305 . , , conceptsNeuropsychological Yon der Marlsburg . NeuralNetworks , C. ( 1988 ). Patternrecognitionby labeledgraphmatching - 148 . , 1, 141 of semantic . Quarterly Journal , E. K. ( 1975 ). Theselective Warrington impairments memory of - 657 . , 27, 635 Experimental Psychology -specific access . Brain , E. K., and McCarthy , R. ( 1983 , 106 , ). Category Warrington dysphasia - 878 . 859 of knowledge : Furtherfractionation , R. ( 1987 , E. K., and McCarthy ). Categories Warrington - 1296 and an attempted . Brain . , 110 , 1273 integration . Brain access , E. K., and Shallice , T. ( 1979 , 102 , 43- 63. ). Semantic Warrington dyslexia -specificsemantic . Brain , T. ( 1984 , , E. K., and Shallice ). Category impairments Warrington - 854 . 107 , 829 . of objectrecognition , E. K., and Taylor, A. M. ( 1978 ). Two categorical Warrington stages - 705 . , 7, 695 Perception - 377 . Fortschritte . Wernicke Arbeitenfiber Aphasie derMedizin , C. ( 1886 , 4, 371 ). Die neuren classand contexteffects in a case of , L. B., and Berndt , R. S. ( 1988 ). Grammatical Zingeser . 5, anomia : for models of , language Neuropsychology pure Implications production Cognitive - 516 473 .

Chapter 14 The Language - to- Object Perception Interface : Evidence from Neuropsychology Tim Shallice

Cognitive neuropsychology has as its principal aim the elucidation of the organization of the cognitive system through the analysis of the difficulties experiencedby neurological patients with selectivecognitive difficulties. As far as the relation between vision and language is concerned , the area that has been most extensively investigatedconcernsthe semanticrepresentationof objects. By contrast, the relation between how representationsof space are accessed from vision and how they are accessedfrom language has been little touched; spatial operations have not been subject to much cognitive neuropsychologyinvestigation. If we consider objects, then the Gibsonian tradition teachesus that the richnessof information available in the visual field is such that many of their properties may be inferred fairly directly from the visual array . Yet there are many other aspectsof the visual world that cannot be inferred from the information in the visual field alonethe structural aspectsof an object that are hidden from the present viewpoint, the potential behavior of an object and of the other objects likely to be found in its vicinity or that go with it in some other way. There are also wider properties of an such as the perceptual features it has when experienced object that may be accessed through other modalities, how it is used and by whom, what its function is, what types of thought processit triggers, and what intentions it may help to create. How are the process es involved in accessingthese properties of an object when it is presented when it is presentedverbally? visually related to the way they are accessed This issue has been the subject of considerable controversy in cognitive neuropsychology in recent years for two reasons . A number of striking syndromesseemto relate very directly to it . In addition , the theory that most directly reflectsthe surface manifestations of the disorders differs from the standard theory in other fields where the issuehas beenaddressed . A model widely referred to in this book and in current cognitive scienceis that of Jackendoff( 1987 ). Languageis viewedas involving three main types of representation - phonological structures, syntactic structures, and semantic /conceptual structures.

532

TimShallice

As far as the semantic , meaningshave internal /conceptual structures are concerned and of from a set built principles of combination , one of primitives up organization " However in addition to its " . the the primitives being , phonological, entity thing contain specifically syntactic and conceptual structures the representationof a word may visual structures. The visual structures involved are, however, explicitly identified with the 3-D structural description level of Marr ( 1982 ). 's was concerned Jackendoff specifically with words and their theorizing Although its in esand particular position on the organization of , the issuesit address meanings the cognitive systemsmediating semantic processingare closely related to issuesrecently . A topic on which there has much debated by cognitive neuropsychologists beenmuch cognitive neuropsychologyresearchin recent years is whether thesemanwhen a word is being comprehendedare the sameas those used tic systemsaccessed in the identification of an object, given that its structural description has already been determined. Somecognitive neuropsychologistshave argued that they are the same , but others have claimed that they differ at least in part . ' Approachesclosely related to Jackendoff s have beenadopted by certain cognitive ; Riddoch and , Berndt, and Brownell 1982 neuropsychologists (e.g., Caramazza of a theory account current . The best 1987 neuropsychological developed ) Humphreys of this type is the organized unitary content hypothesis (OUCH ) of Caramazza . More et al. ( 1990 ), which utilizes a feature basedtheory of semanticrepresentations " an to a semanticrepresentationthrough object will specifically, it holds that access necessarilyprivilege just those perceptual predicatesthat are perceptually salient in an object" . Thus while many elementsof the semantic representation are as easily from visual as from verbal input , someaspectsof the semanticrepresentation accessible from its structural description than from its phonologiare more easily accessed ' . Accessproperties can be asymmetrical. The authors rationale for cal representation assumingan asymmetric relation derives from consideration of certain conditions to be discussed shortly . There is an older tradition in neuropsychology , however, which can be traced back at leastas far as Charcot ( 1883 ) . Certain syndromessuggestthat ) and Wernicke ( 1886 be based partly separablefrom verbally based knowledge. knowledge may visually This perspective has been explicitly adopted more recently by a group of neuropsychologists ; and McCarthy ; Shallice 1987 ; Beauvois 1982 (e.g., Warrington 1975 and Warrington 1988 , ) using the terminology visual semanticsand verbal semantics not been has two of basis of the the clearly representation types conceptual although articulated (see Caramazza et al. 1990 ; and ; Rapp, Hillis , and Caramazza 1993 Shallice 1993 ). An intermediate position has beenadvocatedby Bub et al. ( 1988 ) and by Chertkow and Bub ( 1990 ), they argue that a spe ) . Following Miller and Johnson-Laird ( 1976

-to-Object TheLanguage Interface Perception

533

cific stage intervenes between attaining the structural description and accessingthe amodal " core concept" of an object. Accurate identification of object is held to ' require more than just a characterization of an object s structure, but must involve criteria which are more functional than structural. They therefore argue for the existenceof a subsystemthat contains only the application of the functional and perceptual criteria necessaryfor object identification , receiving the output from the structural description systemand sendingoutput to the core amodal semanticsystem . " is reduced Thus " visual semantics . very considerably in its scope We thus have one position in cognitive neuropsychology (Caramazzaet al. 1990 ) that is entirely compatible with Jackendoff' s perspectivein holding that there is a . In addition it , namely the Caramazzaet al. perspective /conceptual system single semantic certain aspectsof the semantic representation can be , holds that accessing easier from the structural description than from phonology. Two other positions, ' ; Chertkow and Bub 1990 ( Warrington 1975 ) hold that Jackendoff s view is too gross a characterization of the subdivisions of the cognitive system involved in semantic , and that more than one semantic processing /conceptual system exists. A fourth which has to be articulated , holds that semantic representations position , yet formally are processedthrough a connectionist network of which different regions are more specializedfor different types of semantic subprocess , but neither subprocess nor region can be characterizedin an all -or -none fashion (see , for example , Allport 1985 . ; Shallice 1988a ) Two main types of syndrome have beenusedto argue that the semantic - conceptual - those system is not in fact unitary but contains a number of types of subsystem , in involving some form of category specificity, and the modality specific aphasias . I will in review the evidence from each turn and then relate , particular optic aphasia them to the alternative theories. A third syndrome- selectiveprogressiveaphasia will also be addressed .

14 .1 Category Specificity The first groupof syndromes for the plausibilityof the positionthat the responsible semantic is not unitary but composed of a numberof subsystems are those system so -calledcategoryspecificity . The performance of the patient for some manifesting of knowledge is far betterthan for others . Of particularrelevance is the categories in four with described ( War syndrome originally patients herpes simplex encephalitis ). Thesepatientshad a selective rington and Shallice1984 problem in identifying -madeartefactsmuch animals , plants , and foods , while beingable to identify man . For example better , one of thesepatients , JiB.R., could nameonly 6% of living -madeobjects and 20% of foods but could name 54% of man . Moreover , if the things

534

TimShallice

whether a description of a line drawing of the object " grasped the judges assessed core concept," the contrast was even greater (living things, 6% ; foods, 20% ; but man-made objects, 80% ) . A similar effect was found when the patient was asked to ' as to whether the give the meaning of the object s name and this, too , was assessed man made core concept was grasped(living things, 8% ; foods, 30% ; objects, 78% ) . Similar effectshave now been obtained with other patients with the sameetiology ; Silveri and Gainotti 1988 ; Laurent et al. ; Sartori and Job 1988 (Pietrini et al. 1988 and 1993 Sartori et al. 1993 1990 Swales and Johnson 1992 Sheridan ; ; ; ; Humphreys a rash . in the last few there have been De Renzi and Lucchelli 1994 However , ) years of claims that these dissociations are essentially a result of characteristics of the stimulus set rather than evidencefor a particular type of underlying organization of . the semanticsystem Funnell and Sheridan ( 1992 ) initially claimed that the dissociations might arise because words matched for word frequency as used , say, by Warrington and Shallice ) may not be matched for visual familiarity . Indeed, McCarthy and Shallice(see ( 1984 ) had shown that living things were less familiar to Warrington and Shallice 1984 than artefacts when matched for word frequency. Warrington and Shallice subjects ( 1984 ) had dealt with this problem by showing that the dissociationswere still present when differencesin familiarity were taken out as a covariate. Moreover this explanation does not account for the way that the impairment of the patients involved foods as well as living things, as McCarthy and Shallice found foods to be more familiar than artefacts when word frequency is control led. A stronger argument was presentedby Stewart; Parkin , and Hunkin ( 1992 ), who found that the category-specific dissociation of a herpessimplex patient, H .O., disappeared when word frequency, familiarity , and visual complexity were all control led simultaneously. However, the basic dissociation, while statistically significant, was much weaker in H .O. than in some of the patients describedearlier. Moreover, the , volcano, and waterfall instead nonliving category included objects like swamp, geyser of being composedsolely of artefacts. Most critically , Sartori , Miozzo , and Job , ( 1993 ) usedstimuli matched on thesethree variableswith their patient Michaelangelo who showed a clear and significant category-specific effect of artefacts over living things on two different stimulus sets(living things, 30% and 40% ; artefacts 70% and 76% ) . Yet another possible artifact has been suggested ), by Gaffan and Heywood ( 1993 who argued that a critical variable was the density of exemplarswithin a category, which they held to be greater for living things than for artefacts. Becauseliving things are more similar to each other and so less discriminable than artefacts, any discriminability problem would have a greater effect in the category of living things.

-to-Object TheLanguage Interface Perception

535

Riddoch and Humphreys ( 1987 ) had made a similar point previously and shown that there was more overlap betweenline drawings of animals than betweenline drawings of artefacts. Gaffan and Heywood buttress their position on the difficulty in discriminating betweenliving things, as opposed to artefacts, by considering the identification perfonnance of three groups of subjects using the Snodgrassand Vanderwart ( 1980 ) stimuli . The first group were two patients of Farah, McMullen , and Meyer ( 1991 ), who showedstandard category-specificeffects ; the secondwere nonnal subjects , who, however, were given only a 20 ms exposure and the third used six ; monkeys, who were tested on how well they could decide which of two presenteditems was in a previously trained set. All three groups of subjectsin their very different tasks showed an advantageof man-made objects over living things. " Gaffan and Heywood ( 1993 ) argue These results from monkeys are contrary to Warrington and Shallices conjecture . . . that a specific system for identification of man-made objects has evolved in the human brain ; if Warrington and Shallices conjecture were correct, monkeys would show relatively greater difficulty in discriminating among inanimate objects than among living things, compared to human observers ." It is not apparent, however, how such a comparison can be made because the tasks carried out were so different. Moreover, for the monkeys, most of the stimuli would presumably be meaninglessobjects; therefore what should be critical would indeed be raw discriminability . If , however, discriminability were a key factor underlying the perfonnance of both the monkeys and the patients, then one would expect a positive correlation within each of the living and nonliving sets of stimuli between the results of the two group of subjects . In fact, there was no correlation betweenthe items the monkeys found difficult and those the patients found difficult in either the living or the nonliving sets . ' Gaffan and Heywood s work , like that in the other critical studies , used the Snod) stimuli , for which nonns are available on a number of grassand Vanderwart ( 1980 relevant variables. In this set of stimuli the animals, in particular , tend to be rather similar to other membersof their category. Warrington and Shallice( 1984 ), however, also used the so-called Ladybird stimuli , large clear colored pictures designed for preschool children, with three of their patients. Shallice and Cinan have obtained ratings of structural complexity, familiarity , and discriminability from nonnal subjects for the Ladybird stimulus set and used theseto reanalyzethe findings of War rington and Shallice. With these ratings, no difference was found betweenall three categories of stimuli (animals, artefacts, foods) for either familiarity or discriminability , but the animals remained structurally more complex than the other two categories . Becausethe task the patients carried out with this stimulus set had involved

536

TimShallice

word -picture matching using a four -alternative forced-choice task, the relevant degree of discriminability on the Gaffan-Heywood hypothesis was that within each set of five; this is what the subjects of Shallice and Cinan rated. However, with these stimuli two of the three original Warrington and Shallice patients on whom the test had been used performed significantly more poorly on foods than on artefacts with the third showing a strong trend in the same direction. Moreover, on a regression analysis using the ratings obtained by Shallice and Cinan, all three patients showed a significant effect cf category and no effect of the other three variables. Thus it would appear that thesecategory specificity findings cannot just be reducedto somecombination of differencesin word frequency, visual familiarity , structural complexity, and within -category discriminability . In this respect , the work of Shallice and Cinan corroborated an earlier finding of Farah, McMullen , and Meyer ( 1991 ), who used the Snodgrassand Vanderwart ( 1980 ) stimuli with two patients exhibiting the standard category-specific dissociations . In a regressionanalysison picture recognition performance, Farah, McMullen , and Meyer showed that neither name frequency, name specificity, similarity to other objects, structural complexity, nor object familiarity had any significant effect. The . The absenceof a significant only factor to have such an effect was category membership effect of other factors in the presenceof a significant effect of category makes implausible even one final convoluted artifactual explanation put forward by Gaffan and Heywood ( 1993 ) . These authors suggestedthat the category difference arises through performance on items differing in a way dependentupon someother dimension ; following Snedecor and Cochran ( 1967 ), they pointed out that measurement errors on the other dimension can lead to an apparent difference in performance acrosscategorieseven when the differenceson the other variables are allowed for as a covariate. However, what would then be expectedis that there would be a basic effect of some other dimensions ; this was not in fact found in either study. Thus it would appear that the basiccategory-specificeffectscannot be reducedjust to an artifact of somecombination of differencesin word frequency, visual familiarity .A , structural complexity, and within - category discriminability acrosscategories secondtype of finding that supports the conclusion that all neuropsychologicaldissociations in this domain cannot simply be attributed to some artifact of differencesin , namely a presemantic factors is the existenceof the complementary phenomenon in in and two studies foods) some of subjects living things ( superior performance over artefacts ( Warrington and McCarthy 1983 , 1987 ; Hillis and Caramazza 1991 ; Sacchettand Humphreys 1992 ). The first two studies involved global aphasicswho could only be tested by word-picture matching using, for instance , the Ladybird stimuli discussedabove. However, the subjectsin the last two studies were not glob-

-to -Object Perception Interface The Language

537

, C.W . ; thus naming to visual confrontation could be used (for instance ally aphasic in Sacchettand Humphreys 1992scored 19 / 20 on naming animals; but only 7/ 20 on ' naming artefacts). Interestingly, the location of C.W . s lesion (left frontoparietal ) differed from that characteristic of the herpes simplex encephalitis cases(for all of whom the left temporal lobe was involved) . Much the most plausible conclusion is that the category-specificeffectsdo not arise at a presemanticlevel due to some difference in difficulty betweenthe categoriesbut . reflect some qualitative difference in the semantic representationsof the categories in it was terms When the herpesencephalitis syndrome was first described , explained of a contrast betweenstimuli primarily differentiable in terms of their sensoryqualities and those more saliently differentiable in terms of their function . -madeobjects . Theevolutionary functions haveclearlydefined Unlike mostplantsandanimals , man finer functional differentiations of tool has led to finer and of using development . Individualinanimate havespecific functions artefacts for an increasing objects rangeof purposes . Consider for activitiesappropriateto their function and are designed , for instance , and but and are all used for chalk , crayon , ; they drawing writing, they havesubtly pencil . . . . Similarly differentfunctions , , jar, jug, and vaseare identifiedin termsof their function features of each canvaryconsider , but thesensory , to hold a particulartypeof object namely , functional attributescontributeminimally to the identificationof living ably. By contrast attributesprovidethe definitivecharacteristics e. . lion , tiger, andleopard ), whereas sensory things( g , ice 1984 and Shall , 849 , or spotted ) ). ( Warrington (e.g., plain, striped A closely related position was taken to explain the complementary syndrome to be .) discussedlater (seeWarrington and McCarthy 1983 Dector , Dub, and Chertkow (in press ) take a somewhat related position basedon their study of a patient, E.L .M ., who suffered from bilateral temporal lobe strokes. On tests of perceptual knowledge of objects he performed normally , but he was grossly impaired at many tests involving the perceptual characteristics of animals. Dector , Dub, and Chertkow argue that the difference between the superiority of artefacts over animals arises becausedifferent tokens of the sameman-made object may show a considerablevariation in the shapeof its parts but a consistent function that allows for a unique interpretation , thus echoing the Warrington -Shallice position . However, they then argue that artefacts " can be uniquely identified at the basic level through a functional interpretation of their parts" and this is why they are relatively preserved(seeDe Renzi and Lucchelli 1994for a related position) . Many artefacts with a unique function do indeed have a unique organization of distinctly functioning parts; take a lamp, for example. However, others, such as a table tennis ball , do not. As yet it remains unclear to what extent the relative sparing of artefacts depends upon their unique organization of distinctly functioning parts or on the unique functions of the whole.

538

Tim Shallice

14 .2 Sensory of Dift'erentCategories Quality andFunctional Aspects


The position just developed attributes differences in performance across different categories to the way that identification in some categories depends critically on sensory quality information but for others functional information is more critical . One can, however, consider how well different semanticaspectsof the samecategory are understood by patients who show this category-specific pattern . When this is done, knowledge of functional aspectsof biological categories tends to be much better preserved than knowledge of sensory quality aspects (Silveri and Gainotti ' 1988 ) . In a related fashion, Dector, Bub, and Chertkow s (in press ) patient E.L .M . " " was much better at answering encyclopedic questionsabout animals such as " Does a camel live in the jungle or the desert ?" (85% ) than visual ones such as " Does a ' " camel have horns or no horns ? where he was at chance(55% ) . However, the effects are not completely clear-cut. The performance of E.L .M ., say, on functional aspects of animals was still well below that of normal controls, who scored 99% . This was not due just to a general problem with carrying out semanticoperations on concrete objects; when asked to identify artefacts he performed at ceiling. A more dramatic example is given by De Renzi and Lucchelli' s ( 1994 ) herpes encephalitic patient, Felicia. In explaining the perceptual difference betweenpairs of animals, for example , goat and sheep , or paired fruits or vegetables , for example , and she far worse than the worst controls cherry strawberry, performed ( 15% vs. 90% ; 49% vs. 85% ) . However, in explaining the visual difference between paired , lightning rod and TV antenna, she was somewhat better than objects, for example the normal mean (90% vs. 85 % ) . Analogous results have been reported in a number of other studies (e.g., Silveri and Gainotti 1988 ; Sartori and Job 1988 ; Farah et al. 1989 ) although at least one patient, Giuletta (Sartori et al. 1993 ), answerednonvisual questions about animals almost perfectly (seealso Hart and Gordon 1992 ), while at the other extremeS.B. (Sheridan and Humphreys 1993 almost as poorly ) performed on visual as on nonvisual questionsabout animals (70% and 65% , respectively ). Why should the category specific impairment generally recur if in a milder form when the patient is responding to questionsabout animals or foods which appear not to be basedon accessing sensoryqualities? Does it not undermine the explanation of effects outlined earlier, namely, that they arise from damageaffecting category specific ? If one articulates the theory developedthus far sensoryquality representations in a connectionist form , then the problem can be resolved . Farah and McClelland .1) in which somesemanticunits represented ( 1991 ) investigateda model (seefigure 14 the functional roles taken by an item, while others representedits visual qualities. Each of the semantic units was connected (bidirectionally ) to the others, to units representing structural descriptions, and to units representing phonological word -

- to-Object Perception Interface The Language


VISUAL FUNCTIONAL

539

.1 Fig8re14 ' s 1991modelfor -specific of artefact Farahand McClelland ( ) category preservation explaining from Farah and McClelland1991 and naming(reproduced ). by permission comprehension
fonDs . The number of units in the two subsets of semantic representations was deter -

. Subjectsrated the description of mined through an experiment on normal subjects in definitions of both and each item nonliving things in the American Heritage living Dictionary as to whether it described the visual appearanceof the item, what the item did , or what it was for . On averagethere were 2.13 visual descriptions and 0.73 functional ones , but the ratio betweenthe two types was 7.7: 1 for living things and 1.4 1 : for the nonliving things. Thesevalueswere then realized in the representations only of living things and artefacts used for training . The network was trained using an error correction procedure based on the delta rule (Rumelhart, Hinton and McClelland 1986 ) applied after the network had beenallowed to settle for ten cycles following presentation of each input pattern. In each of four additional variants of the basic network , one particular parameter was altered so as to establish the robustnessof any effect obtained. The most basic finding was that lesioning the " visual" semanticunits led to greater impairment for living things than for artefacts with the opposite pattern shown for the lesioning of the functional semantic units. Thus the standard double dissociation was obtained due to " identification " of living things relying more on the visual

VISUAL

PERIPHERAL INPUT SYSTEMS


VERBAL

I I I I I I I I I SYSTEMS SEMANTIC

540

Tim Shallice

semantic units and " identification " of artefacts depending more on the functional semanticunits. More interestingly, if one examineshow closea match occursover the functional semanticunits when a lesion is made to the visualsemanticunits, then there is a difference between the two types of item. The functional representationsof the living things were less adequately retained than those of artefacts. In the original " , the attainment of the representation in one of the semantic subsystems learning process " helps to support the learning of the complementary representation in the other; the richer the representationis in one of the systems , the more use is made of it in learning the complementary representation . Thus the most typical relation between functional and visual impairments with living things is explained. Whether the full range of relations observedcan be explained remains to be investigated . There are two uncomfortable findings that the model would appear well designed to explain. First , the living /nonliving distinction is not absolute. Thus Y .0 .T . was one of the global aphasic patients who performed very much better on word -picture matching with living things and foods than with artefacts( Warringtonand McCarthy 1987 , the impairment did not extend to large man-made objects ) . In Y .O.Tis case such as bridges or windmills . Patient J.J. of Hillis and Caramazza( 1991 ), who had a selective sparing of animal naming like Y .0 .T ., also had the naming of means of transportation spared. Complementarily, the problems of herpes encephalitic patients extendedto gemstonesand fabrics. The semantic representationof all these , especially subcategories may well consist more of visual units than of functional ones if function has to be linked with a specificaction. Second , the living/nonliving distinction is graded. Thus patients have been described in whom the deficit is limited , say, to animals alone (e.g., Hart , Berndt, and Caramazza 1985 ; Hart and Gordon 1992 ) . The sensory quality/ function contrast would seemlikely to be more extreme for animals than foods, say, so that for more minor damage to sensory quality units only the least functional of the semantic categorieswould be affected. Overall, this group of category-specificdisorders fits with the idea that knowledge of the characteristicsof objects is basedon representationsin more than one type of . Realizing the different systemsas setsof units in a connectionist model allows system certain anomaliesin the basic subsystemapproach to be explained. The nature of the mediatedby eachof the systemsremainsunclear, however . The deficit representations appearsnot to correspond simply to damageto visual units. Thus one of the patients studied by Warrington and Shallice ( 1984 ) was unable to identify foods by taste as well as by sight. Moreover, in three of the patients where it has been assessed ; E.L .M . in Dector , Bub, and Chertkow , in (Michaelangelo in Sartori and Job 1988 in ; and SiB. Sheridan and Humphreys 1993 press ), relative sizejudgments could be

-to -Object Perception Interface The Language

541

made fairly accurately , suggesting that even the visual deficit does not extend to all

visualcharacteristics remaIns open . The issue . 14 .3 Optic Aphasia


A secondsyndrome that suggests the needto refine the conceptual / structural description contrast of Jackendoff ( 1987 ) is optic aphasia. First describedby Freund ( 1889 ), optic aphasia refers to an impairment where the patient is unable to name objects presentedvisually but at the sametime givesevidenceof knowing what theseobjects are, for instance , by producing an appropriate mime. Moreover, the problem is not one of just naming; the patient is able to produce the name to a description or on or tactile presentation. A considerable number of patients have been described auditory who roughly fit the pattern (seeBeauvois 1982 ; Iorio et al. 1992 ; Davidoff and De Bleser 1993for reviews ). If one limits consideration to patients who do not appear to have any impairment in accessing the structural description because stimulus quality doesnot affect naming ability , Davidoff and De Bleser( 1993 ) list fourteen patients who have been formally described . Certain of thesepatients performed perfectly in gesturing the use of visually presentedstimuli they could not name (Lhennitte and Beauvois 1973 ; Capian and Hedley-White 1974 ; Gil et al. 1985 ). This apparent preservation of knowledge of the visually presentedobject when it cannot be named has beenexplained most simply by assumingthat the optic aphasic " and " verbal " suffers from a disconnection between " visual semantics semantics , with the name only being accessiblefrom verbal semantics( Beauvois 1982 ; Shallice 1987 at the semanticlevel appearsto differ from ) . The distinction betweensubsystems the one drawn in the previous section betweensystemsrepresentatingfunctional and visual or sensoryquality types of information . I will addressthis issuein more detail later. In any case , a number of authors have contested the claim (seeRiddoch and 1987 ; Garrett 1992 Humphreys ; Rapp, Hillis , and Caramazza 1993 ), holding that the miming could simply be basedon an affordance, that is, an action characteristically -modal associationof sensoryand motor induced by the shapeof the object, or a cross schemas , either of which might in turn be basedonly on an intact structural description . Alternatively , miming might require accessingonly restricted parts of the semantic , in particular those parts most strongly realized from the structural system description because they are also representedin it explicitly , for example , the tines of forks ; this is the privileged accesstheory account of Caramazza et al. ( 1990 ) and Rapp, Hillis , and Caramazza( 1993 ) . A similar explanation might also be given for the preserveddrawing from memory shown in patients such as J.F. ( Lhennitte and Beauvois 1973 ).

542

TimShallice

However, accessto other types of infonnation can be present in these patients ) gave their patient when they cannot name. For instance , Coslett and Saffran ( 1992 ) in which the EM2 a task based on one devised by Warrington and Taylor ( 1978 for similar are items of three which , example, zipper, functionally patient has to judge scoredat 97% . EM2 1989 ) button , coin (seealso patient C.B. in Coslett and Saffran on this task, with the control mean being 94% . Becausethe affordancesof a zipper and a button are not similar , it is difficult to seehow the use of affordancesmight be the basis for this good perfonnance; indeed, there are no subcomponentsof the two ), in confronting structural descriptions that are related. Rapp, Hillis , and Caramazza( 1993 a the argument that such a pattern of perfonnance presents difficulty for " their privileged access ), merely respond by saying, difficulty position (Shallice 1993 naming visually presenteditems in the face of demonstratedintact comprehensionof someaspectof the visual structures, however, indicates that the full semanticdescription required to support naming has not beenactivated from a 3 D representationof " that nonnal perfonnance on the function the stimulus. This argument presupposes matching test can be obtained when activation of the relevant semantic representation . . This claim is merely assertedby Rapp, Hillis , and Caramazza is reduced basic with rather test , However, becausethe task is a three alternative forced choice semantic infonnation being required about each item- concerning its function the assertionhas someplausibility . ) on Similar results have, however, beenobtained by Manning and Campbell ( 1992 Two . more much to be demanding patient A .G. on semantic tasks which appear and Palm Trees was the first . The with these were used test of Pyramids patients types test of Howard and Patterson ( 1992 ) . In a typical item of this test, the patient has to with a pyramid . The stimuli can be presented best fir tree decide which (palm, ) goes either visually, verbally, or in mixed visual-verbal fonnat . In the second test, the patient has to answer sets of questions about each item, (e.g., What is it made of ?) both when the item is presentedvisually and when it is presentedauditorily . A .G. % in perfonned at only 40% 50% in naming objects from drawings, but at 100 thus stimuli , showing naming to description and at 91% in naming tactilely presented a specific naming defect with visual stimuli . However, A .Gis perfonnance on the Pyramids and Palm Trees test, while not at ceiling, was virtually identical acrossthe visual and verbal modalities of presentation (82% vs. 84% ) and in both caseswas . A similar within one standard deviation of the mean of nonnal control subjects . vs. 91% ) Druks and pattern was observed for the question answering test (88% Shallices ( 1995 ) patient LiE .W . behavedin the sameway for both types of test. That no differenceand were not at ceiling on tests of auditory and verbal showed patients ' comprehension seemsimpossible to account for in Rapp, Hillis , and Caramazzas

- to -Object Perception Interface The Language

543

( 1993 ) version of the privileged access . By theory, which involves a unitary semantics contrast, theseresults fit well with the multiple semanticsystemposition . Coslett and Saffran ( 1992 ), on the other hand, presentan interesting variant of the store . multiple position They agreethat two semanticstores do exist and that one is disconnected from the language production mechanismsin optic aphasic patients, but they argue that the stores are primarily distinguished by hemisphere , with the right -hemispheresemantic systembeing disconnectedfrom the languageproduction . However, the patients described by Manning and systemsin the left hemisphere 1992 a Campbell ( ) present difficulty for this position . In the acute condition immediately after a suddenonset lesion (e.g., vascular), the right hemisphereis supposedby right -hemisphere theorists such as Coslett and Saffran not to have accessto any phonological lexicon, although they hold that over time a phonological lexicon becomes available to a semantic system in the right hemisphere(Coslett and Saffran 1989 . This semantic systemor the variety of output phonological word -forms that ) can be accessed from it is then seento have an effectivecontent corresponding to that of the words readable in deep dyslexia (Coltheart 1980a ; Saffran et al. 1980 ; Coslett and Saffran 1989 . In however ) deepdyslexia, , concrete nouns can be read reason ably well but verbs present severeproblems (Coltheart I 980b) . Yet while patients A .G. and LiE .W . were severely impaired in naming objects, which they could identify nonverbally, they could name actions very well. Thus A .G. was 95% correct at naming actions- the samelevel as controls- but worse than 50% at naming objects. This contrast in easeof accessingoutput phonological word -forms from an intact semantic representation is the opposite of what would be expectedaccording to the right -hemispheretheory, where one would assumethat objects should be more easily nameable than actions. The basic multiple semantic store position can perhaps explain the obtained effect by assumingthe existenceof another semanticsubsystem one controlling actions (Druks and Shallice 1995 ); being an essentially high-level output system but accessiblefrom perceptual input , it would have connections to verbal semanticsdistinct from those used by the visual semantic representationsof objects. This, however, remains a highly speculativeaccount. There remains one other counterintuitive aspect of optic aphasia. Many of the patients characterizedas optic aphasicthrough their pattern of success and failure on naming and comprehensiontestsexhibit a strangeset of errors when they fail to name correctly . Of the optic aphasic patients reviewedby Iorio et al. ( 1992 ), who generally ' correspond with Davidoff and De Blesers ( 1993 ) group 2 optic aphasics , nearly all made both semanticand perseverativeerrors, with lessthan half also making visual errors. Moreover, in the most detailed analysisof sucherrors- that of L hermit teand Beauvois ( 1973 ) of their patient J.F.- the authors consider the interaction between

544 Table 14.1 Errors Made by J. F . in Two Experiments Type of error Example 100 pictures

Tim Shallice

30 objects

Horizontal errors Semantic Visual Mixed visual-and-semantic Vertical errors Item and coordinate perseveration

" shoe=- " hat " " coffeebeans=- hazel nuts " " orange=- lemon " T26 . . . =- " wristwatch =- " wristwatch" T27 scissors " " T44 . . . ~ newspaper " " T45 case=- two books " T43 . . . =- " chair " " T47 basket=- cane chair " T53 string =- strand of " weavedcane

9 2 6

3 1 I

Mixed horizontal/ vertical errors

. : Lhennitte and Beauvois 1973 Source

" es(temporally what they call " horizontal errors, understood strictly in terms of the process the and the stimulus of , and between responses presentation ) intervening " " occur. or stimuli of effects where responses what they call vertical errors, preceding It is clear from this analysis that the perseverativeand the semantic errors combine in a complex way (seetable 14.1) . ? Why might such a strangecombination of errors be characteristic of optic aphasia the to dimension a connectionist be given by adding Again a possible answer can pathway models. Plaut and Shallice ( 1993a ) considereda network which had a direct " " a had It also . ones into semantic cleanup pathway visual representations mapping " " that involved recurrent connections from the semantic units to the cleanup units and back (seefigure 14.2) . The network used an iterative version of the backpropagation learning algorithm known as backpropagation through time (Rurnelhart, Hinton , and Williams 1986 ) . Training with an algorithm of this type in such a recurrent network leads to its developing a so-called attractor structure; the effect of the operation of the cleanup pathway is to move a noisy first -passrepresentation at the semanticlevel toward one of the representationsit has beentrained to produce as an output , given that the initial representationis in the vicinity of the trained one. The network contained one other major difference from other networks well ' ) . In known in cognitive psychology, such as Seidenbergand McClelland s ( 1989

-to -Object Perception Interface The Language


C ~

S4S

V~ I

. .2 Figure14 allices ( 1993 Plaut and Sh ) model for explainingthe typical error pattern found in optic 1993a ). from Plautand Shallice by permission (reproduced aphasia

occur at in synapticefficiencyat a single synapse the nervoussystem , changes connection of additional . The 1979 incorporation ) (Kupferman manydifferenttime scales those than in standardly weightsthat changemuch more rapidly training valuable ; it allowsfor temporal is alsocomputationally usedin connectionist modeling ) and into a whole(e.g., von der Marlsburg1988 elements bindingof neighboring and McClelland in described communication Hinton facilitatesrecursion , personal ( a standard combined , therefore network in the Kawamoto 1986 ) . Eachconnection based term weight , short , long term weightwith a rapidly altering slowly changing the activitiesof its input and output units. on thecorrelationbetween A network havingboth typesof weightstendsto reflectin its behaviorboth its the analogue activity; it contains historyand its mostrecent long-term reinforcement to respond trained was of both long term learningand of priming. The network of forty different levelto the structuralrepresentations at the semantic appropriately , it produceda few visual errors but . Whereverthe network was lesioned objects and errors consider typicallymorewith both visualand semantic ably moresemantic to aspect . More critically, therewasa strongperservative similarity to the stimulus well could associates or one of its semantic . The previousresponse the responses to the error pattern well This . stimulus to the error corresponds an occuras present . occurringin optic aphasia allowsthe error pattern to the modeltherefore dimension Adding a connectionist model we usedas a . The informationprocessing to be explained of the syndrome to thoseof Riddochand Humsimulations corresponds basisfor the connectionist et al. ( 1990 ), which wereheld to be unsatisfactory ) and Caramazza phreys( 1987 - and of the simulationis that if short . However , the essence earlierin this chapter influences , the errorswill reflectboth perseverative long-term weightsarecombined .1Thus the obtained at of level the and representation whichstrongattractorsoccur were dimension connectionist if an analogous error patternwould alsobe expected

C~ S

S~ I

units

semantic

86 units up

clean 40

546

Tim Shallice

addedto the multiple semantic system models , provided that one or more of the
semantic systems had analogous attractor properties . 14.4 Conclusion

In the sections 14.1 and 14.2 certain syndromes were discussedinvolving category, specific impairments, particularly those associatedwith herpes simplex encephalitis made arte of man identification where large differencesin performanceexist between facts on the one hand and of living things and foods on the other. Explanations in terms of differencesbetweenthe categorieson a number of potentially confounding that partially . The favored explanation assumes dimensionswere consideredand rejected and of the functional of semantic representations separablesystemsunderlie the the sensory quality properties of stimuli . In section 14.3 another syndrome optic - was considered ; here it was argued that the most plausible explanation aphasia " " " " " . involved disconnecting" visual and verbal or lexical semanticrepresentations that a the view for difficulties The evidencepresentedin all three sections poses , together with a structural description systemthat can also single conceptual system from above, is a sufficient material basefor representingsemanticoperations be addressed . The sensory quality component of the semantic systemcannot be conftated with the structural description system becausevariables relevant to disorders of the latter system , presentation of items from unusual views ( Warrington , for example and Taylor 1978 ) , do not predict the stimuli that are difficult for patients with ; Dector , Bub, and impairments to the former system( Warrington and Shallice 1984 of from the clearer is even . The issue Chertkow in press perspective the secondset of ) disorders. In certain optic aphasicpatients much more semanticinformation appears from vision than could be basedon the structural description alone; to be accessible would it appear not to be available in a generally accessibleconceptual system yet it cannot be used to realize naming. because By contrast, the accounts presented for these disorders fit naturally with those beginning to be developedwithin developmentalpsychology for image schemasat a level of abstraction higher than the structural description and yet not simply subsumable within verbal knowledge (seeMandler , chapter 9, this volume) . However, to esshould be limited to what is required for argue that the such visual semanticprocess ' visual identification alone in Chertkow and Bub s ( 1990 ) visual identification procedure between the structural the this is that and lying only system subsystem not fit well for either does semantic core an amodal and system description system the sensory quality what is lost are condition syndrome. In the herpesencephalitis , according to Miller and Johnsonaspectsof the item, while identification procedures Laird ( 1976 ), require primarily functional property information as well as structural

-to-Object Perception Interface The Language

547

, one possibility to explain the syndrome might be analysis. Turning to optic aphasia to view it as arising from a disconnectionbetweenthe visual identification procedures and the core semanticsystem . However, a task like Pyramids and Palm Treesinvolves the utilization of shared context. The Bub and Chertkow theory holds that inferred context is stored in the amodal core semanticsystem , so that an optic aphasicwould be not expectedto perform well on such tasks for words that could not be named. Patients A .G. (Manning and Campbell 1992 ) and LiE .W. (Druks and Shallice 1995 ) show the opposite pattern, namely, intact performance on this task, together with grossly impaired naming. There are, however, certain problems in explaining the two types of syndrome in terms of the functional /sensoryquality and visual/ verbal dichotomies. The concepts are orthogonal . The information available in a visually or sensory quality - based semanticsystem , as inferred by the information lost in the herpesencephaliticpatient is not the only information accessible from the visual modality in the optic aphasic . Certain , A .G. and LiE .W., can access patient optic aphasic patients, for example types of information from vision that would be in the functional or encyclopedic parts of the semantic systemon a simple all -or -none multiple store view. Moreover, within the semantic dementia literature there are striking echoesof this visual input predominance extending outside the purely sensory quality domain in the performance 2 of patient T .O.B. (McCarthy and Warrington 1988 ). When a picture was presentedto TO .B. his identification was more than 90% accurate for both types of material, but he identified verbal input artefacts much better than living things (89% " vs. 33% ) . Thus when the word dolphin was presented , the patient could say only, A " " fish or a bird , but when presentedwith the picture, he said, Livesjn water . . . they are trained to jump up and come out. . . . In America during the war they started to " get this particular animal to go through to look into ships. McCarthy and War rington have argued that this patient has an impairment that affects the stored information itself rather than an input pathway becauseof the consistency with which particular items were or were not identifiable (see for rationale Warrington and Shallice 1979 ; Shallice 1987 ) . Thus contrasting both optic aphasia and semantic dementia with herpes simplex encephalitis , it would appear that the putative lines of cleavagewithin the semanticsystemsuggested by the syndromesdiffer. One possibility is to postulate category specificsystemsthat are themselves specific to particular modalities (McCarthy and Warrington 1988 ) . However, explanations provided for certain secondaryaspectsof the syndromessuggestan alternative direction in which a more economical solution might lie. A connectionist simulation of Farah and McClelland ( 1991 ) can account for certain otherwise most recalcitrant , the counterintuitive findings about category specific disorders. For optic aphasia error pattern associatedwith the disorder is in turn explicable on a connectionist

548

TimShallice

) . Thus adding a connectionist dimension to simulation of Plaut and Shallice ( 1993a the theoretical framework used to account for the characteristics of the syndromes enablesa much fuller explanation of the detailed nature of the deficits to be provided . approach provides an account Adding sucha connectionist dimension to a subsystem the last ten years or so, that over made closely related to presimulation suggestions neural network with associative the semanticsystemhas as its material basis a large , depending different concepts being representedin different combinations of its subregions on the specific subsetof input and output systemsgenerally used to address ; and Saffran ; Shallice 1988b them (seeAllport 1985 ; Warrington and McCarthy 1987 semantic of processingwould be and Schwartz 1994 ) . How the rule governedaspects . dealt with on this type of account has not been addressedby neuropsychologists However, the use of a connectionist network framework for explaining aspectsof ruleneuropsychological disorders does not preclude the possibility of explaining , provided additional elementsare added to governed aspectsof semantic processing ; and Miikkulainen ; Derthick 1988 the basic network (seeTouretsky and Hinton 1988 1993 /conceptual system postulated by Jackendoff ) . On this account the semantic a as realized be would needto complex neural network. As yet, though, no implementation adequatelyexplains the rich and highly counterintuitive evidencethat detailed study of individual neurological patients provides.
Notes

level is not the caseif the mappingfrom the visual to the semantic I . This is especially of visual for I 993a Shallice and Plaut see presentation ); ( , as it is in language orthogonal . correlated are semantic the and visual the representations , objects
2. A simple peripheral explanation of the phonological word -form being damagedcan also be excluded.

References . In S. K. anddysphasia subsystems , modular memory ). Distributed , D. A. (1985 Allport ill : . in EdinburghChurch .), Current and R. Epstein Newman perspectivesdysphasia (Eds . stone Living . andlanguage vision between of interaction : A process Beauvois ). Opticaphasia , M. F. (1982 47 . 33 B298 London the , , , Transactions Society Royal of Philosophical and of pictures encoding , A. ( I 988 ). Semantic S. Bub, D., Black , E., and Kerkesy , , Hampson
. Cognitive , 5, 27- 66. observations Neuropsychology : Some words neuropsychological in alexiawithout -White, T. ( 1974 ). Cueingand memorydysfunction , L., and Hedley Capian . 262 251 97 . Brain : A case , , report agraphia

-to -Object Perception Interface The Language

549

: deficit hypothesis Caramazza , H. H. ( 1982 , R. S., and Brownell ). The semantic , A., Berndt . BrainandLanguage , 161 , 15 patients by aphasic Perceptual parsingandobjectclassification 189 . Caramazza , C. ( 1990 hypothesis ). Themultiplesemantics , B. C., Romani , A., Hillis, A. E., Rapp - 189 . ? Cognitive : Multiple confusions , 7, 161 Neuropsychology dessignes et et isolee la visionmentale Charcot , J. W. ( 1883 brusque ). Un casde suppression - 571 . Medical et couleurs desobjets(formes , 568 , 11 ). Progres 's . of Alzheimer Chertkow type , H., and Bub, D. ( 1990 ). Semantic memorylossin dementia 417 . Brain , 397 , 113 . In M. Coltheart : A right hemisphere Coltheart , K. E. , M. ( 1980a hypothesis ). Deepdyslexia . . London : . Marshall Eds and J. C. Patterson Routledge , ), Deep dyslexia ( . In M. Coltheart : A . Coltheart , K. E. , M. ( 1980b ) Deepdyslexia reviewof the syndrome . . London : Routledge .), Deep Patterson , andJ. C. Marshall(Eds dyslexia andreading Coslett , EM . ( 1989 comprehension , H. B., andSaffran ). Preserved objectrecognition - 1110 . . Brain in optic aphasia , 112 , 1091 : Replication and the right hemisphere Coslett , EM . ( 1992 , H. B., and Saffran ). Opticaphasia 161 . 43 148 . BrainandLanguage andextension , , : A review of past studiesand a R. and De Bleser Davidoff J. , ). Optic aphasia , , ( 1993 . 7 135 154 . , , reappraisalAphasiology : of objectconcepts Dector , H (in press ). Multiple representations , M., Bub, D., andChertkow -specific . . Cognitive from category Evidence Neuropsychology aphasia in the De Renzi , F. ( 1994 separately represented , E., and Lucchelli ). Are semantic systems . Cortex . of living category brain ? Thecase impairment ., . PhiD. diss Derthick , M. ( 1988 ). Mundanereasoning by parallel constraint satisfaction . Mellon , Pittsburgh University Carnegie of visualidentificationand action namingin T. Druks J. and Shallice , ( 1995 , , ). Preservation Conference Annual British at the . , Society Neuropsychological Paper presented optic aphasia London , March. Farah , G. ( 1989 , Z., and Ratcliff ). Categoryspecificity , K . H., Mehta , M. J., Hammond - 200 . . Neuropsychologia in semantic , 27, 193 memory modalityspecificity Farah memory , J. L. ( 1991 ). A computationalmodelof semantic , M. J., and McClelland . Journalof Experimental and : emergent categoryspecificity impairment Modality specificity - 357 . : General , 120 , 339 Psychology of living thingsbe Farah , MM . ( 1991 , P. A., and Meyer ). Can recognition , M. J., McMullen - 194 . ? Neuropsychologia , 29, 185 selectively impaired und . ArchivfUr Psychiatrie Freund , D. C. ( 1889 Aphasieund Seelenblindheit ). ()ber optische ' . 276 297 2O Nervenkrankheiten , ,

550

TimShallice

of knowledge ? Unfamiliaraspects of living and Funnell , E., andSheirden ,J. ( 1992 ). Categories - 153 . . Cognitive , 9, 135 Neuropsychology nonlivingthings -specific visualagnosia for living GaITan , C. A. ( 1993 , D., and Heywood ). A spurious category . Journalof Cognitive Neuroscience , 5, 118 primates thingsin normalhumanand nonhuman . 128 - 180 . Cognition of lexicalselection . Garrett , M. ( 1992 , 42, 143 ). Disorders Gil, R., Pluchon , D., Rogew , C., Toullat, G., Michenau , R., Lefevre , J. P. ( 1985 ). Disconnexion -verbale visuo , lesimages , lescouleurs , et lesvisages ) pour lesobjects (aphasie optique - 349 avecalexie abstractive . Neuropsychologia . , 23, 333 -specific Hart, J., Berndt , A. ( 1985 , R. S., andCaramazza ). A category namingdeficitfollowing . Nature . cerebral infarction 316 439 440 , , . B. ( 1992 . Nature Hart, J., and Gordon for objectknowledge , 359 , 60- 64. ). Neuralsystems : A testof semantic andpalm trees access Howard , F., Patterson , K. E. ( 1992 ). Pyramids from . and words . Thames pictures Valley Iorio, L., Falango and , A., Fragassi , N. A., and Grossi , D. ( 1992 ). Visualassociative agnosia : A single . Cortex case , 28, 23- 37. optic aphasia studyanda reviewof the syndromes Jackendorff : The relation of linguisticand visualinformation . , R. ( 1987 ). On beyondzebra - 114 . , 26, 89 Cognition . AnnualReview , I. ( 1979 ). Modulatary actionsof neurotransmitters of Neuroscience Kupferman - 465 . , 2, 447 -Faure Laurent , B., Foyatier , N., andPellat , , B., Allegri, R. F., MichelD ., Trillet, M., Naegele : au a unilaterale Etude J. ( 1990 ). Encephalites herpetiques predominance neuropsychologique - 681 . Revue . , 146 , 671 longcoursde9 cas Neurologique L hermit : Reportof te, F., and Beauvois disconnexion , M. F. ( 1973 ). A visualspeech syndrome - 714 with optic aphasia . Brain . a case alexia , 96, 695 , agnosic , andcolouragnosia with spared actionnaming : A description , L., and Campbell , R. ( 1992 ). Optic aphasia Manning - 592 . Neuropsychologia . and possible loci of impairment , 30, 587 . SanFrancisco : Freeman . Marr, D. ( 1982 ). Vision -specificmeaning for modality , E. K. ( 1988 , R. A., and Warrington ). Evidence McCarthy - 430 in the brain. Nature . , 334 , 428 systems of sentence : Assigning McClelland , J. L., andKawamoto , A. H. ( 1986 ). Mechanisms production of sentences . In J. L. McClelland andDE . Rumelhart .), Parallel rolesto constituents (Eds - 325 . Vol. 2, 272 . distributedprocessing : Explorationsin the microstructure of cognition MA: MIT Press . , Cambridge -role analysis with embedded clauses . Miikkulainen case , R. ( 1993 ). Subsymbolic of sentences -202 . Austin: Universityof Texas Press . Technical reportAI 93 -Laird, P. N. ( 1976 . Cambridge Miller, G. A., and Johnson and perception : ). Language . Cambridge UniversityPress

Tim Shallice

Shallice within the semantic . Cognitive , T. ( 1988b , 5, ). Specialization system Neuropsychology - 142 . 133 : Whoseconfusions ? Cognitive Shallice , T. ( 1993 , , 10 ). Multiple semantics Neuropsychology - 261 251 . -semantic Sheridan , J., and Hymphreys , G. W. ( 1993 ). A verbal category specific recognition . Cognitive 10 143 184 . , , impairment Neuropsychology Silver i , M. C., and Gainotti, G. ( 1988 visionand language ). Interactionbetween incategory - 709 semantic . Cognitive . , 3, 677 specific impairment Neuropsychology Snedecor . 6th ed. Ames : Iowa State , G. W., and Cochran , W. G. ( 1967 ). Statisticalmethods Press . : Normsfor setof 260pictures , J. G., and Vanderwart , M. ( 1980 Snodgrass ). A standardized name and visual . Journal , , agreement image agreement familiarity, complexity of Experimental - 215 : HumanLearning andMemory . , 6, 174 Psychology Stewart , F., Parkin , A. J., and Hunkin, N. M. ( 1992 ). Namingimpairment followingrecovery -specific from herpes : Category ? QuarterlyJournalof Experimental simplexencephalitis Psychology - 284 . , 44a , 261 Swales with semantic loss : Cantheyrelearn lost , M., andJohnson , R. ( 1992 ). Patients memory '295 ? Rehabilitation 2 305 . , , conceptsNeuropsychological Yon der Marlsburg . NeuralNetworks , C. ( 1988 ). Patternrecognitionby labeledgraphmatching - 148 . , 1, 141 of semantic . Quarterly Journal , E. K. ( 1975 ). Theselective Warrington impairments memory of - 657 . , 27, 635 Experimental Psychology -specific access . Brain , E. K., and McCarthy , R. ( 1983 , 106 , ). Category Warrington dysphasia - 878 . 859 of knowledge : Furtherfractionation , R. ( 1987 , E. K., and McCarthy ). Categories Warrington - 1296 and an attempted . Brain . , 110 , 1273 integration . Brain access , E. K., and Shallice , T. ( 1979 , 102 , 43- 63. ). Semantic Warrington dyslexia -specificsemantic . Brain , T. ( 1984 , , E. K., and Shallice ). Category impairments Warrington - 854 . 107 , 829 . of objectrecognition , E. K., and Taylor, A. M. ( 1978 ). Two categorical Warrington stages - 705 . , 7, 695 Perception - 377 . Fortschritte . Wernicke Arbeitenfiber Aphasie derMedizin , C. ( 1886 , 4, 371 ). Die neuren classand contexteffects in a case of , L. B., and Berndt , R. S. ( 1988 ). Grammatical Zingeser . 5, anomia : for models of , language Neuropsychology pure Implications production Cognitive - 516 473 .

Chapter

IS

Space and Language

, and Merrill F. Garrett , Paul Bloom , Lynn Nadel Mary A. Peterson

15.1

Introduction

Functioning effectively in space is essential to survival, and sophisticated spatial of . In humans, the emergence cognitive systemsare evident in a wide range of species language adds another level of complexity to the organization of spatial cognition . We use languagefor many purposes , not least of which is the conveying of information about where important things are located (food , safety, enemies ) and how to get ' to and from theseplaces(for discussionof theseevolutionary issues , seeO Keefe and Nadel 1978 , Pinker and Bloom 1990 ) . Given the fundamental nature and importance of spatial cognition , it is of considerable interest to determine the ways in which it . The hope that study of such connections might shed light on connects to language both the spatial cognitive faculty and on the languagefaculty has generatedconsiderable ." interest in the domain of " languageand space We are interestedin how people talk about spaceand what they can and do choose to say about it . By exploring the boundaries of thesecognitive domains, we hope to uncover their structure and to elucidate the ways in which they can relate to one another. By considering the role of developmentand culture in shaping the language spaceinteraction, we hope to discover the extent to which fundamental aspectsof spatial cognition are given a priori and the extent to which spatial cognition can be . And by analyzing the ways in which neural systemsorganize altered by experience and linguistic knowledge, we hope to shed light on how thesetwo capacities spatial relate to one another. In the present chapter we analyze what we take to be the consensuallyaccepted framework within which the relations betweenlanguageand spacehave beenconsidered . Basedon this framework , we critically discusssome influential proposals as to the precise nature of this relationship. Finally , we return to the set of issuesand . questionswith which we began, and reach some tentative conclusions

554 15.2 Framework

M . A . Peterson , L . Nadel, P. Bloom, and M . F. Garrett

The framework we adopt here for how we talk about spaceis basedon the proposal " ' " ) languageof thought hypothesis , 1987 ), who took Fodor s ( 1975 by Jackendoff ( 1983 as a starting assumption. Fodor argued convincingly that one cannot learn a language unlessone already has an original languageto structure the learning process " " ; he referred to this original languageas the languageof thought . The language of thought includes the building blocks from which our conceptsare constructed. ' , Jackendoffhas argued for somethingalong the lines of Extending Fodor s analysis in the situation represented figure 15.1. There exist language representations(LRs), spatial representations(SRs), and conceptual representations(CRs) . LRs include all aspectsof languagestructure, including the lexicon and the grammar; SRs include all aspectsof spatial structure as it is representedin the brain ; and CRs are primitives that form the componentsof meaning, both linguistic (CRL) and spatial (CRs) mean-

LANGUAG SPATIAL As As I \ SPATIAL CAs CAs


Fi~ 15.1 A schematicdepiction of Jackendoff' s analysis of the relationship betweenlanguagerepresentations (Language Rs), spatial representations(Spatial Rs), and conceptual representations . CRs of ) both languageand space (

Spaceand Language

555

ing . It is by virtue of some interface between CRL and CRs that we can talk (using LR ) about space(SR) . We can leaveopen whether this interface correspondsto a set of mappings betweentwo distinct systemsor to an actual sharedconceptual representation . In either case , as shown in figure 15.1, it is likely that only certain aspectsof CRL and CRs can participate in this interface. That is, it is likely that someaspectsof spatial meaning cannot be expressedlinguistically, just as some aspectsof language do not correspond to spatial notions. If we accept this view, then it follows that the " " study of the languageof space cannot fully illuminate our faculty of spatial cognition , some aspectsof which we would not , according to this view, be able to talk about. Nevertheless , given the necessityof someinterface, Jackendoff had the insight that one might gain substantial knowledge about the nature of at least some of the spatial CRs by analyzing linguistic spatial tenDs. Jackendoff' s framework thus makes obvious the following four questions : (I) Which aspectsof spacecan we talk about and which not , and why might this be so? (2) Which aspectsof languagereflect particular spatial attributes and which do not? ? (4) What light can the study of (3) Are spatial CRs changedby linguistic experience ? The answersto spaceand languageshedon the nature of conceptual representations these questions invite analyses from a number of different perspectives , many of which are representedby the authors in this volume. IS.3 Spaceand Spatial Information " ? It is clear What do we mean by " space that spacecan contain objects and events , ' but it neednot . Empty space and , , exist. O Keefe and Nadel ( 1978 , unoccupiedplaces 86) note that " the notions of place and spaceare logical and conceptual primitives which cannot be reduced to , or defined in terms of , other entities. Specifically , place and spacesare not , in our view, defined in terms of objects or the relations between " objects. On the other hand, objects and events both occupy spatial locations and have intrinsic spatial properties. Objects are partly defined by the spatial relations among their parts, and events are partly defined by the spatial relations among the various entities (e.g., objects, people, actions) that compose them. Are the logical distinctions among objects, events , and spacesmaintained in language , and are they ? In order to determine which aspects paralleled by dissociableneural representations of spatial representation are transparent to language , it would be useful to know about the various in which is something ways space representedin the mind / brain. Is there a single multimodal or amodal representation of spaceper se ? Or are there a number of independent modules for spatial representation ? Can the study of these representationshelp us understand space , language , and the interface between the two , and can it shed light on the nature of the " spatial" primitives in CR?

556

M . A . Peterson , L . Nadel, P. Bloom, and M . F. Garrett

15.3.1 Evidencefrom Studiesof the Brain that a relatively large number of distinct A vailable neurobiological evidencesuggests " " , various representationsor maps of spaceand spatial information exist. For example visual motor of , discussed have , space , space space auditory maps investigators of far near ; maps egocentricspace and haptic space , , space space ; maps of body space . Quite and allocentric space ; and, maps of categorical space and coordinate space separaterepresentationsfor the spatial features of objects and their locations have . Neuroscientists have linked a variety of brain structures also been investigated another of these spatial representations or to one , providing converging and systems forms of spatial of these of evidencefor some degreeof independence many information . Neurobiological evidencealso suggeststhe existenceof multi modal spatial representations . It is known , for example , that the mammalian superior colliculus integrates sensoryspatial maps and motor maps, such that nearby neurons are activated either by sensory inputs from , or motor outputs directed to , a particular part of ) . Indeed, this ; Meredith and Stein 1983 egocentricspace(cf. Gaithier and Stein 1979 is ally; comparable integration of sensory integrative system quite primitive phylo genetic " and thermal " spatial maps are where visual , spatial maps occurs in snakes ) brought into register in the superior colliculus (Newman and Hartline 1981 There is also evidencedemonstrating the presenceof multimodal sensory spatial maps in various areasof the mammalian neocortex, including especiallyparts of the ) . All of thesemultimodal maps seemto ; Kolb et al. 1994 parietal cortex (Pohl 1973 share the critical feature of representingspaceegocentrically, that is, with reference to the organism, or some part of the organism (e.g., hand, eye, head, torso) . The " " , as well as those in unimodal maps in these multimodal brain association areas cortex visual and , are all laid out in topographic somatosensory regions such as the fashion, such that neighboring regions of neural spacerepresentneighboring regions of the ego-centeredworld . In addition to these data demonstrating the existenceof various unimodal and multi modal ego-centered spatial maps, there is evidence for a superordinate allocentric amodal or multi modal spatial representation that subsumesor somehow integrates the spatial representations(SRs) provided by each of the various spatial a spatial maps. It is now well establishedthat the vertebrate hippocampus subserves is external that and allocentric multimodal both that is , ; spaceis mapping function of the in terms of the organism, representedindependentof the momentary position relations betweenobjects and the placesthey occupy in what appearsto be an objective ' ' ; and seeO Keefe, chapter 7, this , absolute framework (O Keefe and Nadel 1978 , and direction. The volume) . This systemcontains information about place, distance ' ' ' " cells first identified " and . O Keefe ; O Keefe Dostrovsky 1971 by O Keefe (e.g , place

Spaceand Language

557

' 1976 ; O Keefe and Nadel 1978 ) are active when an animal is in a given location in , as defined by the relationship betweenthat spatial location and other placesin space the environment. O ' Keefe and Nadel ( 1978 ) have postulated that information about distance is provided to the hippocampus via the septal region, and a pattern of . More activation termed theta, driven by inputs from brain stem movement systems 1992 Ranck b 1990a ; Taube, Muller , and , ) have recently, Taube and others (Taube " " described a population of head direction neurons in the dorsal subiculum and thalamus of the rat ; thesecells are active when an animal facesin a particular direction in the environment, whatever its specific location. Existing data show that the place cells and the head direction cells are tightly linked together (Knierim et al. 1993 ) . The representationof allocentric spacecreatedin the hippocampus usesmultimodal information from cortical systemsincluding the parietal and temporal regions, . as well as inputs conveying egocentric information about directions and distances this allocentric is not Interestingly, spatial representation neurally topographic in the way the egocentricrepresentationsare. As far as we can tell , neighboring regions of hippocampal neural space do not represent neighboring regions of the external world . While this hippocampal system has been postulated to provide the basis for certain spatial primitives (e.g., places ), it does not appear to be necessaryfor a wide range of nonallocentric spatial representations , such as those subservedby the superior colliculus and neocortical regions already noted. The spatial maps in the superior colliculus, parietal cortex, and hippocampus are thought to represent space without regard for the exact nature of the objects occupying . The spatial (and other) properties of any part of the representedspace . Thus considerableneuroobjects appear to be captured in separateneural systems biological evidencesuggeststhe existenceof two streamsof visual processingabout objects: a ventral pathway, incorporating regions of the temporal lobe that is concerned with what an object is, and a dorsal pathway, incorporating regions of the parietal cortex that is concernedwith where an object is located with respect to the ). Neuropsychological investigations organism (e.g., Ungerleider and Mishkin 1982 " of brain -damaged individuals have been taken as support for this " what versus " where" distinction in the of , as in the well-known cases object representationsystem " where " for the presence and nature subjectsexpressa lack of awareness blindsight, " of an object, all the while demonstrating by their behavior that they know " where the object is located (e.g., Weiskrantz 1986 ) . However, recent evidenceindicates that the ventral and dorsal processing streams are not nearly as isolated as originally , Anderson, and Fellman 1992 ) . Nor is the neuropsychological conjectured ( Van Essen evidence for independent streams completely convincing; some neuropsychologists now argue for notions like degreesof modularity (for review, seeShallice 1988 ). both to Whatever the status of these visual processingstreams , they provide inputs

558

M . A . Peterson , L . Nadel, P. Bloom , and M . F. Garrett

the hippocampal system , presumably contributing to its ability to construct the allocentric representationdiscussedabove. that ( I ) there are a variety of The evidencefrom studies of the brain thus suggests spatial maps in the brain , which makesit unlikely that there is a single amodal spatial representation that gives rise to the entire set of spatial primitives; (2) at least some neural representationsof spacedo not include detailed representationsof objects, reflecting the logical distinction betweenenvironmental spaceand the spatial aspects of objects discussed above; and (3) there is some , but not total , separation within the systemsrepresentingobjects betweenthose representingwhat an object is, and those representingwhere it is located. 15.3.2 Evidencefrom Studiesof Perception Behavioral evidence is consistent with the idea of a variety of spatial maps. For , consider the elegant study conducted by Loomis et al. ( 1992 ), who showed example observerstwo targets located at different distancesfrom the observerin an open field. The observers performed two tasks with respect to these targets. In one task, observers used a matching responseto report about the perceiveddistance betweenthe two targets at different distances(i.e., they adjusted the apparent horizontal distance between two objects located at a standard distance to match the apparent depth interval betweenthe two test targets) . In the secondtask, observersviewed the display from the samestationary vantagepoint used for the first task and, closing their eyes , walked first to one distal target and then to the other (the targets were removed once the observersclosed their eyesand began walking) . Thus the distance between two distal objects that had been visually apprehended from the same vantage point as usedin the first task was motorically expressed in the secondtask, once the observers had walked to the location of the first object. Loomis et al. ( 1992 ) found a dissociation betweenthesetwo different estimatesof the distance betweenthe two objects, with the walking responses suggestinga more . The latter reflected the veridical perception of distance than the matching responses operation of organizing factors; the error in the perceiveddistance betweenthe two distal objects appeared to increasesystematically as a function of the distance from ' 's the observer vantage point to the objects (this effect may be an instance of Gogel s 1977equidistance tendency ) . On the other hand, no such increaseas a function of distance from the stationary vantage point was observed when the interobject distance . The experiment by Loomis et al. demonstrates was assessed via walking responses that although certain distance representationsare veridical, in that they can support accurate navigating to the two targets in turn , other representationsof the distance between the two objects are systematically distorted by the operation of perceptual organizing factors.

andLanguage Space

559

.... .

... .2 . Figure15 -motion displayin which a smalldot is enclosed An illustrationof the Dunckerinduced in a . A stationary is shown in (A). A movingdisplayis shownin (B), where largerectangle display the actualmotion of the rectangle is indicatedby a solid arrow pointingto the right, and the motionof thedot is indicated arrow pointingto the left. perceived by a dashed Similarly, Bridgeman, Kirch , and Sperling ( 1981 ) demonstrateda striking dissociation between the location of a small target relative to an enclosing frame and its location relative to the observer. These investigators showed their observers a Duncker display, containing a small target enclosed within a larger rectangular frame, like that shown in figure 15.2. In displays such as this one, an " induced motion " illusion occurswhen the frame is displacedabruptly in one direction , say, to the right . In a number of situations, abrupt displacementshave beenshown to mimic motion signals to the visual system , and consequently , to result in apparent motion . The unique characteristic of the Duncker induced-motion illusion is that observers perceivemotion (or displacement ) of the stationary dot , rather than of the displaced frame. For example in the , display shown in figure 15.2, rather than perceiving the displacedrectangular frame as moving to the right , observersperceivethe stationary target as moving to the left , that is, in the direction opposite to the direction in which the frame was displaced . The induced illusory motion of the small target is very compelling visually, and it can be canceled(i.e., the small target can be made to appear stationary when the frame is displaced to the right) by the addition of a real displacement of the small target in the samedirection as the frame displacement(in this example , to the right) . Thus, as with many other kinds of illusory motion (e.g., Gogel 1982 ; Peterson and Shyi 1988 ), induced motion and real motion add perceptually and may be indistinguishable.

S60

M. F. Garrett . and M . A . Peterson , L . Nadel, P. Bloom

about the location Bridgeman, Kirch , and Sperling gathered two kinds of responses of the small target in the Duncker display. One responsewas a cancellation . By this measure , and by self-report about what they saw, the , as described response observersin their experiment indicated perceiving the (actually stationary) target as having moved from its original location. The magnitude of the change in location was about half the distance through which inferred from the cancellation responses the frame had been displaced. On another block of trials , Bridgeman, Kirch , and Sperling asked the same observersto point , with an unseenhand, to the final perceived location of the target after viewing the induced-motion display, which disappeared . Surprisingly, at least from sight before they made the pointing response the same with respectto the cancellation responses , the magnitude subjects given by of the illusion measuredby the pointing responsewas negligible. Under theseconditions , observerspointed much closer to the actual location of the target than to the ). perceivedlocation of the target (as inferred from the cancellation responses Thus the experiment revealed a distinction between the spatial representations mediating the cancellation response(and presumably, visually perceived location) and the spatial representations mediating the pointing response (and perhaps , reflects in general . It clear that the location motoric responses is ) visually perceived visual organizing factors- in this case , an organization that dependson the enclosing relationship betweenthe frame and the target- whereasthe representation of location . ( Wewill accessed by the motor responseseemsrelatively free of such effects return to this point below.) Although it may not be clear how best to characterizethis distinction (seeLoomis et al. 1992for a lucid discussion ), the results of Bridgeman, Kirch , and Sperling and those of Loomis et al. strongly imply that the maps mediating visually perceived spatial relationships differ from those mediating movementdistances , locations, and/ or directions. expressed In addition to the behavioral evidencefor differential encoding of egocentricversus allocentric spaces , and for locations versusdirections, there is evidencethat spatial can reflect the combination of inputs from different modalities. For example experience , Lackner and his colleagueshave shown that auditory , visual, and kinesthetic ' inputs regarding an observers orientation in spaceare combined to yield a perceived spacethat doesnot correspond to the spacesignaledby anyone input (for review, see Lackner 1985 ) . The behavioral evidenceis therefore consistentwith the idea of multimodal , as well as with the idea of various unimodal spatial spatial representations . maps of object and spatial What do perceptual studies indicate about the independence of thesetwo systemshas simply ? To a certain extent, the independence representations been assumed(seeMarr 1982 and Wallach 1949 for explicit statementsof this

561
assumption ) . Consistent with this assumption , object recognition does appear to exhibit location invariance ( see Biederman and Cooper 1991 for recent evidence), and accurate distance perception is clearly possible for novel objects . On the other hand , behavioral evidence has occasionally suggested that these two systems may influence one another . For example , Carlson and Tassone ( 1971) found that the perceived egocentric distance to objects in naturalistic settings is influenced by the familiarity of the objects . The initial experiments could not rule out a number of alternative explanations based on response tendencies or differences in the complexity of familiar and unfamiliar objects , but subsequent work excluded these possibilities ( Predebon 1990) . Similarly , object recognition may not be completely independent of location : recognition accuracy for individual objects located within contextually appropriate scenes is reduced when the objects are presented in inappropriate locations (e.g ., a fire hydrant in a street scene is less likely to be recognized when it is located inappropriately on top of a mailbox than when it is located appropriately at street level ; Biederman 1981) . Recent evidence indicates that figure -ground organization , which entails the perception of the relative distance between two adjacent regions in the visual field , is influenced by the familiarity (or recognizability ) of the regions ( Peterson 1994; Peterson and Gibson 1993, 1994) . These findings have led Peterson and her colleagues " " to propose that a rapid object recognition process ( a what process) operates before " where " the detennination of depth segregation (a classic process) and that the former exerts an influence on the latter in combination with more traditional depth cues, such as binocular disparity ( see Peterson 1994; Peterson and Gibson 1994) . Similarly , Shiffrar and Freyd ( 1990; Shiffrar 1994) have shown that perceived direction of motion " " through space ( where ) is constrained by the types of movements that are " " possible given the nature of the objects in motion ( what ) . In addition to these effects of object identity on perceived spatial organization , there is evidence that another type of spatial information is fundamental for object identity . Object identification fails when the parts of the object are spatially rearranged ( see, for example , Biedennan 1987; Hummel and Biedennan 1992; and Peterson , Harvey , and Weidenbacher 1991), and is delayed when a picture depicts an object misoriented with respect to its typical upright orientation (Gibson and Peterson 1994; Jolicoeur 1988; Tarr and Pinker 1989) . In sum , and consistent with conclusions drawn from neurobiological analysis , the study of perception shows that ( I ) a variety of independent modules for spatial representation exist ; and ( 2) some representations deal with objects , some with spaces, and some with the interaction between the two . What can be said about how , if at all , language hooks up with each of these proposed modules ?

562

M . A . Peterson , L . Nadel, P. Bloom , and M . F. Garrett

15.3.3 Talking about Spaceand Spatial Relations , Taking the existence of independent spatial modules some dealing with spaces somedealing with objects, and somedealing with the interactions betweenthe two as a starting assumption, one can posethe following question. Does languageexpress ? And are the information available in all , or only some , of these spatial modules particular parts of languageused to expressspecificforms of spatial information ? Landau and Jackendoff ( 1993 ; Jackendoff, chapter I , this volume; Landau, chapter 8, this volume) have recently taken a modular position on the question of how language relates to representationsof objects versus spatial relations between objects. the differencesbetween Their position rests on a linguistic analysis that emphasizes . For example code for spatial relations and for objects the manner in which languages , in English, objects are describedby nouns, which are open-classlinguistic elements , -class whereas spatial relationships are described by prepositions, which are closed . The prepositions of English- and membersof the corresponding linguistic elements - may be specializedfor speakingabout space , grammatical classin other languages as opposed to speaking about objects, in that they may be applied with few constraints to objects of different sizes and different sorts (e.g., see Talmy 1983 ) . In coded to be between tend addition , spatial relationships by languages coarsely objects , at least in comparison to object identities. That is, spatial prepositions such as near and far would not support accurate motor behavior of the type studied by ) . Landau and ) and by Loomis et al. ( 1992 Bridgeman, Kirch , and Sperling ( 1981 number of the fact that the also stress Jackendoff ( 1993 spatial prepositions in ) English (around 75) is quite small relative to the number of object names(30,000 or so, according to a count by Biederman 1987 ). differences between prepositions and took these Landau and Jackendoff ( 1993 ) nouns as evidencethat prepositions and nouns mapped onto different sorts of spatial -classlinguistic spatial that closed . In particular , their proposal suggests representations terms might map to a subsetof CRss that are about the spatial relations between objects, or betweenan observerand objects (i.e., relations that representthe locations of objects without regard for the specific properties of the objects occupying those locations), whereasnouns might enjoy a privileged mapping to a subsetof CRss that are specializedfor object representation(e.g., the 3-D object models of Marr 1982or Biederman 1987 ) . Landau and Jackendoff pointed out that the linguistic distinctions in the meaningsof nouns and prepositions fit nicely with neurobiological and compu" tational evidenceindicating that " what" and " where are representedindependently ; Rueckl, Cave, and Kosslyn 1989 ). (e.g., Ungerleider and Mishkin 1982 " what" and " where" about modem research and , systems theory By incorporating ' s 1983 ' on Jackendoff builds Landau and Jackendoff s ( 1993 ) ( ) approach usefully insight that we can learn about spatial conceptual representationsby studying how

Spaceand Language

563

we talk about space . However, in our view, researchprograms that attempt simply to identify subdivisions of languagewith neural spatial systemswill not be able to fully elucidate the nature of spatial conceptual representations(seealso Bierwisch, Chapter 2, this volume) . This follows from the fact that words expressabstract conceptual notions that do not appear to be captured in any one-to-one fashion by sensory , . The literature on word learning illustrates this perceptual, or neural representations point . In much of the discussionof the relationship betweenlanguageand space , particularly by developmental psychologists , it is assumed that nouns are equivalent to object names and that objects are equivalent to entities generalizedon the basis of . But neither of theseequivalenciesexists. For adults, only a minority of nouns shape refer to material objects; most nouns are like day, family , joke, factors, information, and so on. Nouns that do not refer to objects are also used , with appropriate syntax and meaning, by two - and three-year-old children, and there is considerableexperimental evidenceshowing that children have no specialproblems learning such words (see Bloom 1994 , in press CRs that do not correspond ). Even infants appear to possess to objects: infants six months of ageare capableof mentally representing , and counting, discretesounds(Starkey, Spelke , and Gelman 1990 ) and individual actions, such as the jump of a puppet ( Wynn 1995 ). What about the claim that " just for nouns that do name objects," a property such as shape , which can be derived from sensory /perceptual inputs, is criterial? Even this is too strong. Children learn superordinates , like animal and furniture , that refer to that share no common shape categories , relationship tenDs like doctor and sister, and functional tenDs like clock and switch (seeSola, Carey, and Spelke 1992for further discussion ) . In fact, evenfor those objects that do havecharacteristic shapes , children know that shapeis not criterial . If you alter a porcupine so that it has the shapeof a cactus, three- and four -year-old children insist it remains a porcupine; they view onto logical boundaries (e.g., you cannot transfonn an animal into a plant) as more significant than shape(Keil 1989 ). Rather than assumingthat nouns map directly onto a " what" systemthat encodes objects in tenDs of shape , an alternative outlined in Bloom ( 1994 , in press ) assumes that nouns map onto CRs that are nonspatial (and thus can include notions like joke and day) . This is not to deny that shape is important for learning object names , as demonstratedby Landau and her colleagues(seeLandau, chapter 8, this volume) . In some cases , considerations of shape are relevant for detennining category membership , implying an interface between CRs and the shape of an object. For instance , there is evidencethat children and adults have an essentialistnotion of natural kind , so that the CR for porcupine is, roughly, " everything that has the same concepts ' " e. . internal stuff ' as previously encounteredporcupines ( g , Putnam 1975 ; Kei I1989) .

564

M. A. Peterson , andM . F. Garrett , L. Nadel , P. Bloom

- is unobservable But since internal stuff- essence , we nonnally use an observable - to detennine whethersomewith essence is correlated that shape highly property thing is a porcupine. Shape also correlates with membership in certain functional kinds; given the purpose for which chairs are designed , they are likely to have a certain configuration (i.e., they are likely to have shapesthat afford sitting) . As noted above, however, there are many words and semanticcategoriesfor which no such correlation with shape exists. Jokes and days have no shapes , doctors and fundamentalists do have typical shapes but not ones that distinguish them from , and although categories like animals and furniture refer to lawyers and agnostics . These entities with shapes the entities fonning the category all have different shapes , considerations show that although there is a relationship between the category of nouns and the notion of object shape , it is not direct. Rather, it is mediated through a more abstract conceptual systemof CRs. As a result, the link betweenlanguageand the shapesof objects is, at least for open-classcategoriessuch as nouns, nowhere near assumeit to be. as direct as many researchers Similarly , it is clear that spatial tenDScannot be derived simply from an interface between language and a set of sensory jperceptual maps. Consider what is actually used to describethe relationship betweenthe the conveyedby spatial representation : in 15.3A and the , captured by the following sentence jar figure butterfly The butterfly is in the jar . In this description, the relationship describedby the spatial preposition in cannot be reduced to one of mere surroundednessin the visual display. The butterfly in figure 15.3B is not in the tabletop, although the contours of the tabletop surround it , as the contours of the jar surround the butterfly in figure 15.3A ; whereasthe butterfly in figure 15.3C is correctly described as in the canyon, although the contours of the canyon do not surround the butterfly . Clearly, the meaning of the spatial preposition in cannot be defined by appealing to attributes of a sensory spatial representation only . Instead, one must to appeal to someabstract relationship, such as a capacity for containmentthat jars and canyonsshare , but tabletops do not. The abstract notion of containment may be one of the conceptual representationslinking space and language , and it may be by virtue of this CR that canyons and jars can be categorized , but the notion of containment simply cannot be accountedfor similarly by language es of sensed infonnation (for discussion , seeBowennan, chapter 10, this by complex volume, and Mandler , chapter 9, this volume) . Even though CRs will not map to complexes of sensory infonnation , can similarities in the way linguistic tenDSand spatial maps are characterizedhelp us identify an underlying isomorphism betweena linguistic category such as prepositions and a " " ? For example , might the categorizing role played particular spatial representation

Spaceand Language

S6S

,~
a

.3 Figure15 in doesnot map simply to surof the spatialpreposition that the meaning Demonstrations of the butterflyis surrounded . The image in thevisualdisplay roundedness by thecontoursof of the the contours but not in b the table in a and both thejar ( ) canyonin (c). Yet the by ( ), relativeto the jar in the of location the describes in term butterfly spatial correctly spatial the spatialterm on (rather than in) appliesto the spatial (a) and the canyonin (c), whereas locationof the butterflyrelativeto the tablein (b). imply that by the tenD in in the situations illustrated in figures "15.3A and 15.3C " coordinate" " to as to have a opposed categorical privileged mapping spatial tenDs " " , such as those postulated by spatial representations within the where system ; Kosslyn et al. 1992 )? Such a solution Kosslyn and his colleagues(Kosslyn et al. 1989 would appeal to the surface similarity implied by the use of the tenD categorical in thesetwo cases , but the similarity may not go beyond the surface. Categorical maps treat a set of spatial locations as equivalent; that is, these maps have low resolution . Yet tenDS such as in could as easily reflect high-resolution as low -resolution . spatial representations Those situations in which spatial prepositional usagecan depend upon the entities representations being related may be more revealing about the nature of conceptual " " " " than those situations that fit within a what versus where dichotomy . There are a number of examplesin addition to the one illustrated in figure 15.3, showing that

566

M . A . Peterson , L . Nadel, P. Bloom, and M . F. Garrett

, the considerable nonspatial factors govern the use of spatial terms in English. For example variability in applying the terms front and back to churches implicates functional factors as relevant to axial descriptions, given that the front of a church can be defined by functional factors (e.g., the direction people attending a service face) as well as by structural factors coded in an object representation . (SeeVandeloise 1991for a discussionof functional meaningsof French prepositions) . Current , supported by computational evidence , indicates that neuropsychological evidence functional representationsmay be critically important attributes of object meaning , this volume) . Thus the nonspatial semanticsof an object (see Shallice, chapter 14 . Other nonspatial semantic factors such as salience may govern prepositional usage are relevant when we say whether something is " near" or " far " ; these semantic factors are evident in some spatial memories as well (for review, see McNamara 1991 ). A last example that language does not divide up in ways that map directly onto particular neural spatial representationscan be found in linguistic directional terms. Directions are coded and lexicalized by various languages in deictic (egocentric ) terms; in intrinsic (object-centered in terms or absolute cardinal terms. Does this ; ) ( ) variability imply that languageinterfaces directly with spatial representationsof all these types, at least with regard to direction? Would this count as a case of direct ? linkage betweenparticular linguistic elementsand specificspatial representations 's Consider the fact that the terms right/left can be used to denote both the speaker ), or the right and left of some other object or person. right/ left (e.g., egocentric use Yet spacesare apprehendedegocentrically by an exploring animal, either by the act of moving through them or by the behavior of visually scanningthem. In either case , the inputs are initially coded in terms of egocentric relations between the observer and entities such as placesand objects in the environment. Spatial relations of distance and direction , for instance , and even of the arrangement of parts within an " " entity, are computed by the organism from its various egocentric inputs . Human factors work has demonstrated the primacy or importance of egocentric coding as well. For example , if one has to discriminate which of two adjacent objects on a screen is display brighter (larger, more familiar ), the best responsemapping is one in which a choice of the object on the right is indicated by a right -hand key press , and a choice of the object on the left by a left -hand key press . There is much evidencefrom developmental work showing that egocentric knowledge about spatial location precedes allocentric knowledge (e.g., Mangan and Nadel 1992 ; Wilcox , Rosser , and In Nadel 1994 addition , the neuropsychological deficit of neglect points to the ). primacy of egocentric right/ left coding. Individuals who sustain damageto the right " " . For example , when there are objects in parietal lobe often neglect left hemispace their right hemispace in their left hemispace(Hellman , neglectpatients ignore objects

Spaceand Language

567

1979 ) . Likewise, when neglect patients are ; Volpe, LeDoux , and Gazzaniga 1979 , they are asked to imagine a scene , well known to them before their brain damage unable to imagine those objects that lie to their left from their imagined vantage point , and the objects omitted changeas the imagined vantagepoint changes( Bisiach and Luzzatti 1978 ) . Thus right/ left egocentric relationships appear to be coded early in processingand appear to be critically important in spatial understanding. Notwithstanding the evidenceattesting to the importance of egocentrically based spatial information , many normal individuals have severe difficulties in mapping linguistic terms onto egocentric relationships. Part of the problem in using the terms egocentric spacesdepend on the direction the individual right/left may arise because is facing; that is, the regions of spacethat lie to the right and left are interchangedby a 1800rotation . For a speaker and a listener who face each other, the spaceto the right of the speakerlies to the left of the listener. Even for speakerswho evidenceno overt difficulty in using the terms right and left , considerable effort is required to translate the frame of the speakerinto that of the listener (seeTversky, chapter 12, this volume) . Those sameindividuals who have difficulties mapping linguistic terms onto egocentric relationships have no trouble in reaching to the right or left to catch an object that right/ left terms do not map directly falling off a table. This dissociation suggests . Consistent with the possibility that the to motoric egocentric neural representations perspectivetaking evident in language use does not simply reflect the use of a per spectivizedspatial map, Levelt (chapter 3, this volume) demonstratesthat the spatial for speakingabout arrays of dots affording either an egocentric representationsaccessed or an intrinsic description are not already coded for the egocentric or instrinsic . In our view, the underlying spatial representations directions speakerschooseto express allocentric be spatial representationssuch as those found in the may instead ' ). hippocampus (seeO Keefe and Nadel 1978 The discussion of linguistic evidencehas thus far focused on objects and spatial relations, including distancesand directions. We have yet to touch upon a critical ? In contrast to the aspectof spaceand that is place. How does languagetreat places described by open-classelements spatial relations of distanceand direction , placesare . Someplace namesare count nouns, like center , rather than by closed classelements same in much the instances novel to can be extended these and and border basement , , , and country . Others are proper way as names for kinds of entities like ball , house which behavemuch like the individual names , or the Equator , , like Paris, Times Square names Bill , Joan, and the Salvation Army , and are certainly as informative . We know of no count of the number of places that can be named in English to stack up against the number of spatial prepositions counted by Landau and Jackendoff ; ; n = 75) or against the number of object namesestimatedby Biederman( 1987 ( 1993

568

M . A . Peterson , L . Nadel, P. Bloom, and M . F. Garrett

n = 30,000), but certainly the number of placesthat can be named is several orders of magnitude larger than 75. This fact suggeststo us a rather different way of imagining the relation between -classelementsand aspectsof spatial aspectsof language such as open- and closed and . ( object) representations Systemsconcerned with both space per se and with objects contain information about entities and about relations betweenentities. The linguistic evidencecould be taken to suggestthat nouns are usedin the caseof entities , objects, or other things- seebelow), and that prepositions are used (be they places in the caseof relations (be they about places , objects, or other entities) . In this view, prepositions are not limited to describing the spatial relationships between objects. Even putting aside more abstract usagesof prepositions (as in " John went from rich to poor " ; see , for example , Jackendoff 1990 ), sentencessuch as the following are : perfectly acceptable The mist hovered over the sea . John put the poison into the soup. A swarm of beesflew into the forest. There was an explosion next to my house . Boston is near New York . He swept the spacein front of the fireplace. In these examples , there is no problem whatsoever using prepositions to describe spatial relationships between and among substances , collections of objects, events , locations, and even empty spaceitself. In sum, studies of languageprovide no reason to go beyond the basic framework spelled out at the outset, in particular , to propose privileged one-to-one mappings betweenparts of languageand particular spatial representationsystems . While it may be the casethat the information carried by some spatial maps can be talked about, and the information carried by others cannot, we believethis is not a result of connections betweenspecifictypes of languageelementsand specificspatial maps. This does not mean that we reject the idea that neurobiological and perceptual / cognitive research can shed light on the nature of the spatial conceptual representations . Rather, we suspect that investigations of the ways in which CRs interact with the maps identified by neurobiological and behavioral researchwill be fruitful in elucidating the nature of spatial conceptual representations . This leads us to the following modest proposal. Some , but not all , of the spatial maps identified by neurobiological and behavioral researchimposea structure that goesbeyond, and in consequence alters, our interpre-

Spaceand Language

569

tation of the information available in the input alone. For example , the hippocampus ' Euclidean onto non framework a Euclidean to (O Keefe and inputs appears impose Nadel 1978 , who seein this processthe instantiation of a Kantian a priori notion of absolute space ) . Other examplesare revealedby the organizing factors that structure - factors like the equidistancetendency(Gogel, somebehavioral representations 1977 ; Gogel and Teitz 1977 ) and the constraints due to gravity identified by Shepard " " 1994 see for ) . We propose that in distorting the sensoryinputs, Shepard ( summary, thesespatial maps may impose an order and a structure that our spatial conceptual , studiesof languageuseand other spatial representationsrequire. If this were the case behaviors that revealedthe operation of theseorganizing factors might lead to some . understanding of the CRs themselves in this sort of Before engaging analysis it is important to first look at the ways in which the mappings between language , behavior and spacevary across cultures. If there are internally imposed structures that reflect primitive spatial CRs, one would expectto seethesestructurespreservedacrosscultural and linguistic boundaries. This follows from the assumption, central to our guiding framework , that the CRs are part " . of a universal " languageof thought that makesunderstandingof the world possible If , on the other hand, spatial frameworks and perception itself can be shown to vary acrossculture, their utility as stable indicators of the nature of spatial CRs is . questionable
15.4 EfFects of Experience

Speakersof Tzeltal code spatial relations with respect to absolute directions; they simply do not use egocentric terms to speak about space or objects (see Levinson, chapter 4, this volume). In this respect they differ from speakers of Dutch and English, for example. The critical feature of absolute directions is that they remain . In Tzeltal, the absolute directions that are used invariant as vantage point changes - uphill / downhill - are applied even when the environment in of originate a feature the geographicalfeature is out of sight. A tremendousamount of effort is required to , these directions seemto be well keep track of the absolute directions; nevertheless events and scenes in memories of the experiencedby the speakersof Tzeltal preserved 1993 Levinson Levinson Brown and , chapter 4, this volume) . This certainly raises ; ( the question that led us to consider the effects of experiencein the first place. Are there differencesin the CRsSbetween speakersof Tzeltal and speakersof English, Dutch , or other languagesthat lexicalize egocentric relations rather than cardinal directions? This possibility is difficult to address , but Levinson and his collaborators have shown that speakersof Tzeltal and speakersof Dutch behavedifferently in old/ new

570

M. A. Peterson , L. Nadel , P. Bloom , andM. F. Garrett

perceptual recognition tasks, problem-solving tasks, and memory tasks (Levinson, chapter 4, this volume) . Furthermore, gestures employed by speakers recounting rememberedscenesand events are different. The gesturesemployed by speakersof Tzeltal indicate absolute directions, and the gesturesemployed by speakersof Dutch and English indicate relative directions (Haviland 1993 ). Does this mean that the , or the culture in which one lives, can change the nature of the languageone speaks ? Or does it support the lessradical claim that the culture in which underlying CRsS one lives, and the language one speaks , affects the availability of different CRsS becauseof differential degreesof practice utilizing them? And in either case , do such findings imply that the conceptual representationsat the interface betweenlanguage and thought are themselves different? It seems clear that different languagesand/ or cultures can utilize different cognitive skills to different degrees . For example , Emmorey and her colleagues(seeEmmorey, chapter 5, this volume) have shown that sign languagemay engagemental rotation skills, and consequently , may improve theseskills due to practice. Of course , differential at mental rotation does not that the are different. Nor is this prowess imply CRsS differences in on implied by performance memory tasks and problem solving tasks, such as those discussedby Levinson (chapter 4, this volume), although the existence of such differencesprovides evidencerelevant to theories of perception, memory, and ' , Levinson s finding (chapter 4, this volume) that differential problem solving. For example encoding of absolute versusegocentricdirections by speakersof Tzeltal and of Dutch , respectively , is evident in problem-solving tasks is consistentwith psychological evidencethat world knowledge (which differs from speakerto speaker ) influences . problem-solving behavior (Murphy and Medin 1985 ) What about Levinson' s claim that differential encoding of absolute directions versus egocentric directions is also evident in performance on old/ new perceptual ? If the perceptual representationsaccessed recognition memory tests , for example , by speakersof two languagesweredifferent, by virtue of the differential attention each had paid to particular aspectsof the situation at encoding, that would be consistent with recent evidencethat knowledge influences perception more than traditionally assumed(for summaries , seePeterson 1994 ; and Shiffrar 1994 ) . Note that the effects of knowledge on perceptual organization may be highly constrained in that the relevant structural , semantic , or functional representationsmediating such knowledge must be accessedwithin the normal time course of perceptual processing (see ; Gibson and Peterson 1994 Carpenter and Grossberg 1987 ; and Peterson , Harvey, ' and Weidenbacher1991 ) . Thus, if Levinson s findings are shown to reflect differences in perceptual representations per se , they might justify a search for correlates of absolute directions in perceptual input . !

Spaceand Language

571

Alternatively , the tasks employed by Levinson may reflect differencesin semantic representationsbetweenspeakersof different languagesand/ or membersof different cultures. It has always been supposedthat different languagesor different cultures or less might combine primitive CRs differently so that certain meanings are more 2 ) . Neuropsychological ; Slobin 1995 salient to speakers of a given language (Bowerman 1989 that volume 14 this Shallice discussed , ), suggest (chapter by findings of identifying course in the be accessed qualitatively different semanticrepresentationsmay artifacts and living things (seealso Farah and McClelland 1991 ) . Similarly, and eventscould vary in their emphasis semanticrepresentationsof rememberedscenes ' . on absolute or egocentricdirections, dependingupon one s culture and experience ' different are that semantic mean s results Thus, while Levinson representations might of different languages(seealso Bowerman, chapter 10 , this volume), they for speakers would not entail that the primitive CRs themselves , from which the semantic representations are constructed, have beenchangedby language(or culture) . It is important not to overemphasizethe differencesbetweenspeakersof different : it is clear that spatial cognition is not necessarilyconstrained by the language languages that one knows. For example , speakersof languagesthat do not habitually so when theseterms are suited to the task (see do terms can direction absolute employ In addition . volume 12 this , speakersof Tzeltal may useegocentric ) Tversky, chapter , not overshadowedby absolute direction are (or deictic) relations, especiallywhen these relations ( Brown and Levinson 1993 ) . More generally, it follows as a matter of logic that some understanding of spatial relationships must be available prior to the ; otherwise, it would be impossiblefor spatial language acquisition of spatial language to be acquired in the first place. While this leavesopen the possibility that exposure to different languagescan engagecertain aspectsof spatial cognition to a greater ' extent than others, it does not support the strongest Whorflan hypothesis that one s manner of thinking about spaceis entirely determined by the language one learns. .) (For more generaldiscussionof this point , seeFodor 1975 The precedingdiscussion , and much of the evidencein this volume, implies that the exact mappings betweenCRs and CRL are plastic in that they can look rather different . Need this have implications for the structure of CRs and in different languages answer is no. One need not assumethat how one talks ? The short CRL themselves ' about one s spatial conceptsnecessarilyinfluencesthose conceptsin somefundamental way. What we can talk about is certainly less than what we know , and what we know consciously , whether we can state it preciselyor not , is certainly lessthan what in . Cultures toto we know may influence how we chooseto refer to spatial attributes, and even which spatial attributes we chooseto refer to , but there is little support for the view that they, or the languagesthey use , fundamentally alter our spatial understanding of the world . Under the assumption that experiencedoesnot fundamentally

572

M . A . Peterson , L . Nadel, P. Bloom, andM. F. Garrett

alter perceptual and cognitive processes, the study of intrinsic organizing factors should offer a window upon the underlying conceptual representations . By comparing how we use language to refer to space and spatial relations with how we behave in space, we can gain insight into some of theso " distortions ."

15 .5 Conct _ oIB
At the outset we posed four central questions in the study of spaceand language : ( 1) Which aspectsof space can we talk about? (2) Which aspectsof language reflect ? (4) What particular spatial attributes? (3) Are spatial CRs changed by experience can the of and shed on the nature of light study space language conceptual ? representations There are aspectsof spatial knowledge we cannot naturally talk about (for example , absolute distance between two objects or between an observer and an object), and aspectsof spatial knowledge we can talk about (for example , spatial relations), but we cannot at presentprovide a satisfying distinction betweenthesetwo classes of spatial knowledge. Although the distinctions can be described by terms like precise and coarse spatial representations , we do not believe that those terms accurately the CRs that underlie the distinction . The suggestionby Landau and express might Jackendoff ( 1993 that nouns and ) prepositions describeobjects and spatial relations, , is an important start to the project of understandinghow languagemaps respectively to space , but we suspectthat a broader view, namely, that nouns describe entities (including, but not limited to , placesand objects) and prepositions describerelations, is likely to be closer to the truth . There is evidencethat different cultures refer to spacein different ways, but there is no reasonto supposethis involves a changein the , as long as the distinction between CRs and underlying conceptual representations semanticrepresentationsis kept clear. We pointed out the importance of a careful analysis of the intrinsic " organizing factors" that interact with environmental information to structure our knowledge of the spatial world . These organizing factors act as a kind of " syntax" in accord with which inputs to spatial systemsare ordered, and in so doing they contribute meaning to the spatial representationsthemselves . This is perhaps clearest in the allocentric . It is our map observedin the hippocampus, but it is also observablein other cases view that careful study of the way language reflects these organizations or " distortions " should help illuminate the CRs. By itself, however, such a study will not the entire task. The relationship betweenspatial languageand other aspects accomplish of cognitive processing , such as our intuitive understanding of motion (Talmy, chapter 6, this volume), the on-line recognition of spatial relationships (Logan and

Spaceand Language

573

Sadier, chapter 13, this volume), and our deductive inferencesabout theserelationships (Johnson- Laird , chapter II , this volume), must also be carefully unpacked if we are to derive maximum benefit from the study of language and space . Progressin these areas should improve our understanding of the relations between space and . , which in turn could illuminate the nature of conceptual representation language
Acknowledgments We thank the McDonnell -Pew Cognitive NeuroscienceProgram for fostering our collaboration on this chapter, which was written while Mary Petersonand Lynn Nadel were on sabbatical from , and supported by, the University of Arizona .
Notes

' s claimsreston the I . Levinson that performance on old/newrecognition assumption memory tests reflects differences in perceptual or processing . See predominantly organization Hochberg andPeterson and Hochberg . (1987 ) ; Peterson ) for criticismof this assumption ( 1983 - 86 2. Both Fodor( 1975 theprimitive , 85 , 17 ) andJackendoff ( 1983 ) allowthat, by combining CRsdifferently a givenideamoreor less salient to thespeakers , differentlanguages mayrender of thoselanguages .
Refere~

Biedennan, I . ( 1981 . In M . Kubovy and J. R. ) . On the semantics of a glance at a scene Pomerantz ( Eds.), Perceptualorganization , 213- 253. Hillsdale, NJ: Erlbaum. Biedennan, I . ( 1987 : A theory of human image understanding. ) . Recognition by components Review 94 147 . 115 , , Psychological Biedennan, I ., and Cooper, E. E. ( 1991 ) . Evidencefor complete translational and reflectional invariance in visual object priming . Perception , 20, 585- 593. Bisiach, F., and Luzzatti , C. ( 1978 . Cortex, 14, ). Unilateral neglect of representational space - 133 129 . Bloom, P. ( 1994 : The role of syntax-semanticsmappings in the acquisition of ). Possiblenames nominals. Lingua, 92, 297- 329. Bloom, P. (in press . ) . Theories of word learning: Rationalist alternatives to associationism In T . K . Bhatia and W. C. Ritchie (Eds.), Handbook of language acquisition . New York : Academic Press . Bowennan, M . ( 1989 : What role do cognitive predispositions ) . Learning a semantic system - 169 . , 133 play? In M . L . Rice and R. L . Schiefellbusch( Eds.), The teachability of language Baltimore: Brooks. Bridgeman, B., Kirch , M ., and Sperling, A . ( 1981 ). Segregationof cognitive and motor aspects of visual function using induced motion . Perceptionand Psychophysics , 29, 336- 342.

574

M. A. Peterson , andM. F. Garrett , P. Bloom , L. Nadel

: Brown , S. ( 1993 ). Linguistic and nonlinguisticcoding of spatial arrays , P., and Levinson Research no. 24 . Working paper , CognitiveAnthropology in Mayancognition Explorations . , Nijmegen Group, Max PlanckInstitutefor Psycholinguistics : A theoretical Carlson ). Familiar versusunfamiliar size , E. P. ( 1971 , V. R., and Tassone - 115 . , 87, 109 . Journalof Experimental Psychology derivationandtest for a selforganizing parallel architecture , S. ( 1987 ). A massively , G. A., and Grossberg Carpenter and Vision , . , Graphics Image neural pattern recognition machine Computer - 115 . , 37, 54 Processing model of semantic memory Farah , J. L. ( 1991 ). A computational , M. J., and McClelland . Journal and of Experimental : specificity category emergent impairment Modality specificity - 357 . : General , 339 , 120 Psychology . . Cambridge , MA: HarvardUniversityPress Fodor, J. A. ( 1975 of thought ). Thelanguage usesimilarsensory and mammals organizations Gaithier ). Reptiles , BE . ( 1979 , N. S., and Stein - 597 . . Science in themidbrain , 595 , 205 -independent objectrecognition Gibson , M. A. ( 1994 ). Doesorientation , B. S., and Peterson . Journal of a cueingparadigm from ? Evidence orientationdependent recognition precede - 316 . 299 20 and Human , , : Performance Perception Psychology Experimental . In W. Epstein(Ed.), Stabilityandconstancy W. C. , ). The metricof visualspace ( 1977 Gogel - 181 . . New York: Wiley Interscience es andprocess : Mechanisms , 129 in visual perception with a lateralmotion of motion concomitant , W. C. ( 1982 ). Analysisof the perception Gogel . 250 241 32 andPsychophysics . Perception , , of the head . 1977 D. ) Eye fixation and attentionas modifiersof perceived ( , W. C., and Tietz, J. Gogel - 362 . andMotor Skills . Perceptual distance , 45, 343 , and orientation in Guugu Yimithirr pointing Haviland , iconicity , J. ( 1993 ). Anchoring . Journal , 3, 3- 45. Anthropology Linguistic of gestures . In K. M. Hellmanand E. Valenstein Hellman ). Neglectand relateddisorders , K. M. ( 1979 . Oxford . New York: UniversityPress .), Clinicalneuropsychology , 268 307 (Eds and cognitivecomponents organization ). Piecemeal , M. A. ( 1987 , J., and Peterson Hochberg . Journalof Experimental to : in objectperceptionPerceptually responses movingobjects coupled . 380 370 116 : General , , Psychology Biederman and E. J. Hummel , I . ( 1992 ). Dynamicbinding in a neural network for shape , , - 517 . 99 480 Review . , , Psychological recognition . . Cambridge andcognition , MA: MIT Press Jackendoff , R. ( 1983 ). Semantics . : The relation of linguisticand visual information Jackendoff , R. ( 1987 ). On beyondzebra . , 26, 89 114 Cognition . . Cambridge structures , MA: MIT Press Jackendoff , R. ( 1990 ). Semantic . Canadian objects Jolicoeur , P. ( 1988 ). Mental rotation and the identificationof disoriented . 478 461 42 Journalof Psychology , ,

Spaceand Language

575

Keil, F. C. ( 1989 . Cambridge , kinds , andcognitive ). Concepts . development , MA: MIT Press Knierim, J. J., McNaught on, B. L., Duffield , C., and Bliss , J. ( 1993 ). On the binding of fieldsto the inertialorientationsystem . Society hippocampal place , Abstracts for Neuroscience , 19 . , 795 Kolb, B., Buhrman , K., McDonald , R., and Sutherland of the , R. J. ( 1994 ). Dissociation medialprefrontal , posteriorparietal , and posteriortemporalcortexfor spatialnavigation and - 680 Cortex . recognition memoryin the rat. Cerebral , 46, 664 , S. M., Chabris , C. F., Marsolek Kosslyn , C. J., and Koenig versus , O. ( 1992 ). Categorical coordinatespatial relations : Computational and computersimulations . Journalof analyses - 577 : HumanPerception andPerformance Experimental . Psychology , 18 , 562 , S. M., Koenig , 0 ., Barrett Kosslyn , A., Cave , C. B., Tang , J., and Gabrielli , J. D. E. ( 1989 ). Evidence for two typesof spatialrepresentations : Hemispheric for categorical specialization and coordinate . Journalof Experimental relations : HumanPerception andPerformance Psychology - 735 . , 15 , 723 -motor adaptation Lackner , J. R. ( 1985 to the terrestrialforceenvironment ). Humansensory . In D. J. Ingle rod, and D. N. Lee (Eds .), Brain mechanisms , M. Jeanne and spatial vision , - 209 175 . Dordrecht : Nijoff. " " " " Landau , B., and Jackendoff , R. ( 1993 and spatial ). What and where in spatiallanguage . Behavioral and Brain Sciences . cognition , 16 , 217 265 Loomis , J. M , DaSilva , J. A., Fujita, N., and Fukusima , S. S. ( 1992 ). Visualspace perception and visuallydirected . Journalof Experimental action : HumanPerception andPerformance Psychology - 921 . , 18 , 906 , P., and Nadel Mangan , L. ( 1992 and development ). Spatialmemorydevelopment of the -fifth International formationin Down syndrome . Paper at theTwenty hippocampal presented of Psychology , Brussels Congress , July. Marr, D. ( 1982 . SanFrancisco : Freeman . ). Vision ' McNamara . In G. H. Bower(Ed.), Thepsychology , T. ( 1991 ). Memorys view of space of - 186 andmotivation . New York: Academic learning , vol. 27, 147 Press . Meredith . ( 1983 , M. A., andStein , BE ). Interactions among converging sensory inputsin the - 391 colliculus . Science . superior , 221 , 389 in conceptual Murphy, G. L., andMedin , D. L. ( 1985 coherence . Psychological ). Theroleof theories - 316 Review . , 92 , 289 Newman , E. A., and Hartline , P. H. ( 1981 ). Integrationof visualand infraredinformationin - 791 bimodalneurons of the rattlesnake . Science . , 213 optic tectum , 789 ' O Keefe . Place units in the , J. ( 1976 of the ) hippo campus freely moving rat. Experimental . , 51, 78- 109 Neurology O' Keefe , J., and Dostrovsky , J. ( 1971 : Preliminary ). The hippo campusas a cognitivemap - 175 evidence from unit activity in the freelymovingrat. BrainResearch . , 34, 171

576

M. A. Peterson , andM. F. Garrett , L. Nadel , P. Bloom

, map. Oxford : Clarendon O' Keefe, J., and Nadel, L . ( 1978 ) . The hippocampusas a cognitive . Press

-groundorganization cananddoesoccurbeforefigure Peterson , M. A. ( 1994 ). Shape recognition - 110 . Science in Psychological . CurrentDirections , 3, 105 -ground contributionsto figure Peterson , BS . ( 1993 recognition , M. A., and Gibson ). Shape . . Cognitive in threedimensional , 25, 383 429 Psychology display organization Peterson object , BS . ( 1994 , M. A., and Gibson ). Must figuregroundorganization precede - 259 . in peril. Psychological Science ? An assumption , 5, 253 recognition Peterson , H. L. ( 1991 , E. H., andWeidenbacher , M. A., Harvey ). Shape recognition inputsto : Human counts ? Journal : Which route of Experimental Psychology figuregroundorganization - 1089 . andPerformance , 17 , 1075 Perception -setmeasurement : A quantitative Peterson , J. ( 1983 , M. A., and Hochberg ). Opposed procedure . Journalof Experimental in andintention form perception of the role of localcues analysis - 193 . andPerformance : HumanPerception , 9, 183 Psychology of real and illusoryconcomitant Peterson , M. A., and Shyi , G. C.-W. ( 1988 ). The perception - 42. . Perception andPsychophysics cube rotation in a three -dimensional , 44, 31 . Behavioral andBrain andnaturalselection Pinker , S., andBloom , P. ( 1990 ). Natural language - 642 . Sciences , 13 , 585 deficitsfollowingfrontal and parietal of spatialdiscrimination Pohl , W. ( 1973 ). Dissociation - 239 . andPhysiological . Journalof Comparative in monkeys lesions , 82, 227 Psychology Predebon , J. ( 1990 ). Relativedistance judgmentsof familiar and unfamiliar objectsviewed - 348 . . Perception andPsychophysics naturalconditions underrepresentatively , 47, 342 " " . In H. Putnam(Ed.), Mind, language of meaning Putnam , and , H. ( 1975 ). The meaning - 271 . . Cambridge : Cambridge : Philosophical , vol. 2, 215 UniversityPress papers reality " " " " Rueckl , S. M. ( 1989 , K. R., andKosslyn , J. G., Cave ). Why are what and where processed Neuroscience . Journalof Cognitive ?A computational corticalvisualsystems investigation by separate 1 171 186 . , , : Cambridge University . Cambridge to mental structure Shallice , T. ( 1988 ). Fromneuropsychology Press . of theworld. Psychonomic asreflections - cognitive universals , R. N. ( 1994 ). Perceptual Shepard BulletinandReview , 1, 2- 28. in Psychological Research . CurrentDirections where Shiffrar , 3, , M. ( 1994 ). Whenwhat meets . 96- 100 Shiffrar , J. J. ( 1990 , M., and Freyd ). Apparentmotion of the humanbody. Psychological . 1 265 Science 257 , , : Nativelanguage to t ~ink for speaking globin, D. ( 1995 , and theoretical , cognition ). Learning : . Cambridge .), Rethinking . In J. J. Gumperz and S. C. Levinson linguisticrelativity (Eds style . Cambridge UniversityPress

and Language Space

, Spelke , N. N., Carey , S., Sola , E. S. ( 1992 ), Perception , ontology, and word meaning. - 107 . , 45, 101 Cognition , P., Spelke , E. S., and Gelman . Starkey , R. ( 1990 ). Numericalabstraction by humaninfants . , 36, 97- 127 Cognition
structures . In H. Pick and L. Acredolo(Eds , L. ( 1983 .), Spatial Talmy ). How language space - 282 orientation : Theory . NewYork: Plenum , research , andapplication Press . , 225 Tarr, M. J., and PinkerS . ( 1989 in shape ). Mental rotation and orientationdependence - 282 . Cognitive . recognition , 21, 233 Psychology Taube of headdirectioncellsrecorded , J. S. ( 1992 in the rat anterior ). Qualitativeanalysis thalamus. Societyfor Neuroscience , 18, 108. Taube, J. S., Muller , R. U ., and Ranck, J. B., Jr. ( l990a) . Head direction cells recorded from the postsubiculum in freely moving rats. I . Description and quantitative analysis . Journal of Neuroscience , 10, 420- 435. Taube, J. S., Muller , R. U ., and Ranck, J. B., Jr. ( 1990b ) . Head direction cells recorded from the postsubiculum in freely moving rats. 2. Effects of environmental manipulations. Journal of Neuroscience , 10, 436- 441.

. In D. J. Ingie , L. G., and Mishkin, M. ( 1982 Ungerleider ). Two corticalvisualsystems , M. A. - 586 Goodale .), Analysis behavior , andR. J. W. Mansfield . Cambridge (Eds , 549 of visual , MA: MIT Press . Van Essen , D., Anderson , C., and Felleman in the primate , D. ( 1992 ). Infonnationprocessing - 423 visualcortex : An integrated . Science . systems , 255 , 419 perspective Vandeloise / prepositions : A case . Chicago , C. ( 1991 : Universityof ). Spatia study from French . Press Chicago , B. T., LeDoux , J. E., and Gazzaniga in an Volpe , MS . ( 1979 ). Infonnation processing " " - 724 . extinguishedvisualfield. Nature , 282 , 722 Wallach the relation between , H. ( 1949 ). Someconsiderations concerning perceptionand . Journa / of Personality . cognition , 18 , 6- 13 Weiskrantz . Oxford: Clarendon Press . ,L.( 1986 ). B/indsight

Wilcox of object , T., Rosser in 6.5-month location , R., andNadel , L. (1994 ). Representation oldinfants . Cognitive 9 193 209 . , , Development - 60 K. 1995 . of numerical , Wynn ( . ) Origins knowledge. Mathematical Cognition,, 1 , 35

Name Index

Acredolo , L., 129 , 130 Allan, K., 358 Allen, R., 443 Allport, D., 533 Anderson , C., 355 , 557 Anderson , J., 280 Anderson , S., 172 , 469 , 479 Antell, S., 388 , 392

Antinucci , F., 388 Atkinson , M., 391

, H., 113 , 125 Baayen Babcock , M., 244

, E., 15 , 386 , 422 Bialystok Biedennan , I., 8, 9, 13 , 24 , 46 , 317 , 347 , 495 , 498 , 561 , 562 , 567 - 73 Bierwisch , M., 8, 15 , 19 , 31 , 32 , 37 , 55 , 58 , 65 , 134 , 358 , 386 , 402 , 422 , 424 , 563 Binford , 0 ., 317 Bisiach , F., 567 Black , J., 469 - 573 Bloom , P., 14 , 357 , 389 , 390 , 426 , 553 , 563

Battison , R., 172 Bauer , M., 456 Bauer , P., 369 , 375 , G., 387 Baumgartner , G., 460 Baylor Beauvois , M., 532 , 541 , 543 Becher , A., 349 -Chadha Behl , G., 388 , U., 171 , 195 , 196 , 200 , 201 , 175 , 194 Bellugi BennettD., 280 Berlin , B., 56 Bennan , R., 40I , 402 Berndt , R., 532 , 540 Bertenthal , B., 370 Berthoz , A., 152 , 196 Bettger

, R., 367 , 371 , 378 , 388 , 389 , 391 , 392 Baillargeon Barwise . J.. . 455 . 443

Bomba , P., 388 Bomstein , M., 344 Bower , G., 469 , 470 , 471 Bower , T., 392 Bowerman - 426 , M., 7, 192 , 203 , 379 , 385 , 387 , 389 , 391 , 402 , 404 , 405 , 409 , 416 , 418 , 419 , 420 , 421 , 424 , 428n , 498 , 526 , 564 , 571 , P., 234 Boyer Breedin ., 355 ,S , 358 Brewer , B., 126 , B., 559 , 560 , 562 Bridgeman Brown , P., 70 , 80 , 101 , 102 , 103 , 104 , 110 , Ill , 114 , 122 , 148 , Brown , R., 389 , 403 , 422 Brownell , H., 532 , C., 378 Brugman , 473 , 478 , 503 BryantD., 471 Bub , D., 532 , 533 , 537 , 538 , 540 , 546 , 547 Bucher , N., 231 B\ihler , K., 147 , 465 Bull , W., 449 , W., 465 Burroughs , R., 46 , 80 , 437 , 443 , 446 , 449 , 460 Byrne

Cara Dla12a , A., 532 , 533 , 536 , 540 , 542 , 545 ., 235 ,S , 336 , 346 , 372 , 389 , 424 , 425 , 563 Carey Carlson , V., 561 -Radvansky Carlson , L., 89 , 93 , 94 , 132 , 133 , 325 , 496 , 499 , 500 , 524 , 525 , 526 Caron , A., 388 , 392 Caron , R., 392 , G., 570 Carpenter , P., 131 , 503 , 521 Carpenter Carramazza , A., 532 Carroll , J., 424 Cave , K., 562 Charcot , J., 532

, J., 129 , 130 Campbell Campbell , R., 542 , 543 , 547 Capian . L.. 541

580 Chase , W., 502 Chertkow , 547 , 540 , 546 , 537 , 538 , 533 , H., 532 , S., 519 Chipman Choi, S., 351 , 420 , , 405 , 409 , 418 , 379 , 404 , 377 421 , 423 , 35 , 41 , N., 33 Chomsky Cienki , A., 401 Cinan , 536 , S., 535 Clark, E., 344 , 424 , 391 , 418 , 389 Clark, H., 110 , 422 , , 144 , 386 , 134 , 135 , 128 , 131 470 , 521 , 524 , 496 , 502 , 503 , 493 Cochran , W., 536 Cohen , D., 131 Coltheart , M., 543 , 526 , 523 , 524 , B., 496 , 521 Compton Comrie , 302 , B., 301 Cook , W., 280 Coon , 473 , 476 , V., 184 , E., 561 Cooper , 500 , L., 471 Cooper Corballis , M., 500 Corina , 194 , 175 , D., 172 Coslett , 543 , 358 , 542 , H., 355 Couclelis , H., 465 Coulter , G., 172 Craik, K., 437 Cramazza , A., 541 Culicover , P., 8

Name Index Fara , M., 355 Farah , 547 , 571 , 535 , 536 , 538 , M., 10 , 355 Farrell , W., 470 Felleman , D., 355 Fellman , D., 557 Fillmore , 471 , 465 , 466 , 467 , 374 , C., IS, 131 Fodor , 368 , 554 , 571 , 231 , J., 41, SO Francis , W., 336 Franklin , SO3 , 476 , 478 , 470 , 471 , 473 , N., 184 Frederick , R., 128 Freeman , N., 389 , 244 , 561 , J., 232 Freyd Friederici , 89 , 133 , A., 88 Frisk, V., 278 Funnell , E., 534

, 148 , 125 , 131 , E., 113 Danziger Davidoff , 543 , J., 541 DeBleser , 543 , R., 541 Dector , 546 , 540 , 538 , M., 537 deLeOn , 418 , L., 404 Denis , M., 469 DeRenzi , 537 , 538 , E., 534 Derthick , M., 548 DeValois , K., 387 DeValois , R., 387 , J., 37 Dolling Donna , D., 201 , J., 556 Dostrovsky , 448 , D., 14 Dowty Druks , J., 543 , 547 Dunker , K., 559 Ehrich , 484 , V., 89 , 469 Eilan , N., 152 Elmas , 388 , P., 368 - 204 , , 194 , 195 , 175 , 185 , K., 124 , 171 Emmorey 196 , 570 , 201 , 203 -Pedersen , E., 185 Engberg , S., 256 Engel Eriksen , C., SO3 , J., 455 Etchemendy Evans , J., 449

Gaffan , 536 , D., 534 , 535 Gainotti , 538 , G., 534 Gaithier , N., 556 Garnham , 493 , 494 , 524 , A., 89 , 437 - 573 Garrett , 553 , M., 541 Garrod , 479 , S., 469 , M., 567 Gazzaniga Gee , J., 185 Gelman , R., 563 Gentner , 403 , 404 , D., 378 Gibson , 570 , B., 561 Gibson , E., 389 Gleitman , L., 404 , A., 470 Glenberg , 569 , W., 559 Gogel Golinkoff , R., 376 Goodhart , W., 185 , 424 , 392 , 393 , A., 391 Gopnik Gordon , 540 , B., 538 Grabowski , J., 80 , 88 ., 470 ,S , 471 Greenspan , R., 152 Gregory Grice , H., 451 Griffiths , P., 391 , S., 570 Grossberg Gruber , 305 , 22 , 48 , 278 , J., 13 Gruendel , J., 390 , J., 404 Gumperz Hafner , J., 443 , G., 447 Hagert Hale , K., 35 Hankamer , J., 95 Hamard , 373 , S., 72 Hart , 540 , J., 538 Hart , R., 465 Hartline , P., 556 , 570 , E., 561 Harvey Haude , R., 198 Haviland , 134 , J., 113 , 125

Name Index . 526 . W.. 343 . 496 . 506 . 524 . 525 Hayward . M.. 464 Hazelfigg -White . T.. 541 Hedley Hellman . K.. 566 Heine . B.. 396 . 400 Herrmann . 88 . T.. 80 Herskovits . . 142 . 191 . 265 . 322 . 399 . A.. 128 . 134 493 . 494 . 496 . 524 . 526 . 536 . C.. 534 . 535 Heywood Hickok. G.. 201 Hill . A.. 89 Hill . C.. 143 . 339 . 474 . 400 Hillis. A.. 532 . 540 . 541 . 542 . 536 Hinrichs . E.. 14 Hinton. G.. 539 . 544 . 548 Hirtle. S.. 465 . L.. 61. 62 Hjelmslev Hockett . C.. 147 Hoffman . D.. 347 Hoiting. N.. 185 Howard . F.. 542 Hseih . 331 Hubel . D.. 355 Hummel . J.. 561 . 536 . 537 . 538 . 540 . . G.. 532 . 534 . 535 Humphreys 541 . 545 Hunkin. N.. 534 Hurwitz. S.. 83. 471

581
Jones , S., 344 , 346 , 376 Jonides , J., 465 Just , M., 131 , 503 , 521 Kahneman , D., 443 , H., 50 Kamp Kant , E., 128 -Smith Kanniloff , A., 367 Katz , J., 35 Kawamoto , A., 545 , P., 56 Kay Keil . F.. . 368 . 392 . 404 . 424 . 563 . 235 , A., 448 Kenny ., 35 ,S Keyser Kintsch , W., 471 Kirch , M., 559 , 560 , 562 , E., 6 Kittay , R., 152 , 153 Klatsky Klima , 200 , 201 , E., 171 , 175 , 194 Knierim , J., 557 Kolb 556 , B., . Kolstad , V., 371

., 45 , 47 , 130 , 195 , 196 , 198 , 199 , 471 , ,S Kosslyn 496 , 562 , 565 Koster , C., 469 , 484 Kruskal , J., 514
, M., 131 Kubovy Kucera . , n .. , 336 ., 336 ,S , 339 Kuczaj Kuhn , D., 443 , I., 545 Kupferman

lIan , A., 196 Inhelder , 319 , 326 , 335 , 387 , B., 129 Iorio , L., 541 Irwin , 132 , 133 , 325 , 496 , 499 , 500 , , D., 89 , 93 , 94 524 , 526 , 525 - 27 Jackendoff , 6, 7, 8, 9, 10 , II , 13 , 14 , 15 , , R., 1 22 , 47 , 48 , 49 , 63 , 74n , 77 , , 26n , 35 , 39 , 40 , 44 104 , 281 , 322 , , 215 , 268 , 272n , 278 , 130 , 176 , 200 353 , 399 , 405 , 422 , 423 , 493 , , 354 , 355 , 378 , 386 494 , 532 , 533 , 541 , 554 , 555 , 562 , , 499 , 524 , 531 567 , 568 , 572 Jammer , M., 128 Janis , W., 174 ., 464 Jankovic ,1 , 471 , 479 rod Jeanne , M., 10 , A., 256 Jepson Job , R., 534 , 538 , 540 Johnson , M., 268 , 269 , 375
Johnson , R., 534 -Laird, P., 5, 6, 15 Johnson , 47, 81 , 87 , 110 , 128 , 132 , 159n , 162 , 203 , 322 , 386 , , 134 , 137 , 140 , 144 - 460 422 , 465 , 438 , 443 , 445 , 449 , 456 , 460 , , 437 466 , 524 , 532 , , 472 , 493 , 494 , 496 , 469 , 470 , 471 546 , 573 Johnston , 386 , 388 , 389 , 390 , J., 326 Jolicoeur , P., 500 , 521 , 561

Labov , W., 270 , 271 , 469 , 483 Lackner . l .. 560 Lakoff , , G., 13 , 22 , 268 , 269 , 373 , 402 , 420 , 498 499 , 506 - 358 Landau , 281 , 317 , , B., 10 , 15 , 130 , 176 , 268 327 , , 347 , 353 , 3590 , 378 , 386 , 399 , 329 , 333 , 346 405 , 499 , 524 , 562 , , 422 , 423 , 464 , 478 , 493 , 494 567 , E., 19 , 21 , 54 , 55 , 73 Lang , 215 , 365 , 402 , 420 , R., 10 , 13 , 22 , 62 Langacker Larkin , J., 455 Laurent , B., 534 Lederman ., 152 ,S , 153 le Doux , J., 567 Leech , G., 131 , M., 371 Legerstee Lehrer , A., 6 Leslie , 371 , A., 367 - 104 Leve1t , 19 , 77 , 78 , 81 , 83 , 88 , 89 , 93 , , W., 18 101 , 285 , , 103 , 117 , 132 , 133 , 134 , 144 , 146 , 160 325 , 484 , 335 , 465 , 466 , 469 , 472 , 478 , 482 , 493 ,
494 , 524 , 567 Levin , B., 6 Levine , D., 355

582
Levine , M., 464 , 471 , 479 Levine , S., 336 , 389 Levinson , S., 7, 80 , 88 , 101 , 102 , 103 , 104 , l04n , - 157 l06n , 109 , 111 , 114 , 113 , 117 , 122 , 125 , 131 , 132 , 134 , 145 , 148 , 158n , 160 , 176 , 179 , 184 , 254 , 285 , 321 , 324 , 325 , 335 , 352 , 386 , 402 , 404 , 465 , 466 , 468 , 470 , 482 , 486 , 569 , 570 , 571 , 573n Lewis , D., 144 , M., 242 , 335 , 347 , 353 Leyton Lhennitte , F., 541 , 543 ., 174 Liddell ,S , 175 , 198 Lillo-MartinD., 171 , 174 , 175 , 185 Linde , C., 270 , 271 , 483 , 469 Lindem , K., 470 , M., 355 Living stone , S., 389 Lloyd Locke , J., 152 Lockman , J., 465 Loew , R., 185 - 526 , G., 133 , 493 , 496 , 499 , 502 , 521 , 523 , Logan 524 , 526 , 572 Loomis , J., 558 , 560 , 562 Lowe , D., 317 Lucariello , J., 367 Lucchelli , F., 534 , 537 , 538 Luzzatti , C., 567 , K., 465 Lynch , J., 134 , 135 , 449 Lyons , L., 500 , 521 MacKay Mac , R., 396 Laury MacLean , D., 392 , S., 471 , 484 Mainwaring - 380 Mandler , J., 5, 26n , 203 , 365 , 366 , 369 , 370 , 371 . 372 . 373 . 374 . 375 . 376 , 381 , 420 , 421 , 546 , 564 , P., 566 Mangan Mani , K., 470 , 471 Manktelow , K., 449 , L., 542 , 543 , 547 Manning Maratsos , M., 336 , 339 , 367 Markman , E., 404 , C., 545 Marlsburg Marr , D., 8, 9, 13 , 21 , 24 , 46 , 130 , 141 , 151 , 252 , 317 , 335 , 347 , 437 , 455 , 464 , 460 , 495 , 497 , 498 , 499 , 532 , 560 , 562 Matsumoto , Y., 215 , R., 532 , 534 , 536 , 537 , 547 , 548 McCarthy McClelland , 544 , 545 , 547 , J., 538 , 539 , 571 , K., 124 McCullough -Nicholich McCune , L., 390 , L., 366 , 369 , 375 , 379 McDonough

Name Index

McNamara , T., 566 Medin , D., 424 , 570 Meier . R.. 171 Meltzotr , A., 152 , 391 , 392 Meredith , M., 556 Mervis , C., II , 368 Metzler , J., 130 , 131 , 196 , M., 470 , 535 , 536 Meyer Michotte , A., 11 , 343 Miikkulainen , R., 548 Miller , C., 185 Miller , G., 15 , 87 , 110 , 128 , 132 , 134 , 140 , 144 , 322 . 386 . 422 . 445 . 465 . 466 . 469 . 471 . 493 . 496 .
532 , 546 Miller, J., 196 Milner, B., 278 Miozzo , M., 534 Mishkin , M., 10 , 268 , 354 , 557 , 562 Moore , G., 465 Moravcsik , J., 51 Morrow, 0 ., 470 Muller, R., 280 , 557 , G., 424 , 570 Murphy

Nadel , L., 10 , 128 , 129 , 159n , 268 , 277 , 278 , 281 , - 573 294 , 305 , 308 , 313 , 465 , 553 , 555 , 556 , 557 , 566 , 567 , 569 - 21 Narasimhan , B., 19 , 29 , 335 Needham , A., 388 , 392 , 389 Neisser , U., 463 Nelson , K., 367 , 369 , 390 Newman , E., 556
, E., 171 Newport Newstead . S.. . 449

Nicbolicb , L., 386 , G., 463 Nigro Nishihara , H., 464 , 495 , 499 Ohlsson , S., 441 - 315 O'K~ fe , J., 10 , 128 , 129 , 144 , 1590 , 268 , 277 , 218 , 280 , 281 , 294 , 296 , 301 , 305 , 306 , 313 , 465 , 553 , 555 , 556 , 551 , 561 , 569 Oliva , A., 355 , 356
Olson , D., 15 , 386 , 422 Osherson , D., 443

Padden , C., 174 , 185 Paillard , J., 129 , M., 464 , 471 , 479 Palij Palmer ., 231 ,S , 460 , 495 Parisi , D., 388 Parkin , A., 534
Partee , Bo , 8, 449 Patterson , Ko , 542 Pears , Jo , 126

McIntire , M., 190 , 191 McMullen ., 535 ,P , 536

Name Index Pederson , E., 117 , 148 , 254 , 404 , 135 Perkins , D., 443 Perrett , D., 465 , W., 471 Perrig Peterhans , E., 387 - 573 Peterson , 561 , 570 , M., 495 , 553 , 559 , J., 129 , 319 , 326 , 335 , 365 , 366 , 367 , 387 , Piaget 391

583

Pick , H., 132 , 465 -LeBonniec Pieraut , G., 392 Pietrini , V., 534 Pinker . S.. 6. 8. 131 . 402 . 464 . 553 . 561 Plaut , D., 544 , 547 Pobl , W., 556 Poizner , H., 200 , 201 Poulin , C., 185 Predebon , J., 561 Presson , C., 464 Prior , A., 449 J,, 6, 7, 14 , 33 , 51 , 53 Putnam 11 , , 563 , Z., 460 , 494 , 497 , 500 Pylysbyn

Schiano D .,117 471 ,B ,484 Schmitt . , , ., Schneider 354 ,G M . Schober 183 , , ,463 ,468, 470, 484, 485, 486 M . Schuler 392 , , Schwartz M .,548 ,., P 355 , ,356 Schyns W Scoville . 277 , , .,514 ,J Seery .,544 ,M Seidenberg Senft G . 399 , , ., 521 ,J ,523 ,524 Sergent E . 443 Shafir , , -548 .531 Shallice .T .533 .534 .542 .544 .546 .547 .
548 , 566 , 571 , R., 83 , 130 , 131 , 196 , 464 , 471 , SOO , 519 , Shepard 569 -Kegl , J., 190 , 191 Shepard Sheridan , J., 534 , 538 , 540 Shiffrar , M., 355 , 561 , 570 , G., 559 Shyi i , M., 534 Silver , 538 Simon , H., 443 , 455 Sinha , C., 358 , 407 , 389 Sitskoom , M., 392 Slobin , D., 78 , 132 , 185 , 377 , 386 , 388 , 389 , 401 , 402 , 403 , 571 Smith , L., 344 , 376 , 345 , 346 Smith , M., 278 Smith ., 443 ,S Smitsman , A., 392 Snedecor , G., 536 , J., 535 , 536 Snodgrass , N., 346 , 563 Sola , R., 126 Sorabji , E., 346 , 367 , 388 , 392 , 563 Spelke , A., 559 , 562 , 560 Sperling . James St , J., 503 . John St , M., 185 , L., 465 Staplin , P., 563 Starkey Stecker , D., 327 , 329 , 422 Stein , B., 556 Stein , 152 , J., 129 Stewart , F., 534 Su brahman , K., 346 yam ., 185 ,S Supalla , T., 203 Supalla ., 128 Svorou ,S Swales , M., 534 Sweetser , E., 375 Szeminska , 335 , A., 326

Puste 10vsk ) '.. . H

, W., 368 , 443 , 449 Quine , P., 368 , 388 Quinn Ranck , J., 280 , 557 , T., 6 Rapoport , 541 , B., 532 , 542 Rapp Rescher , N., 449 , U., 50 Reyle

~hett C . 536 , , ,537 . Sadalla G 465 , , D . Sadier 133 343 493 526 573 ,, , , , , E . 355 358 542 543 548 Saffran , , , , , I. 95 , , Sag W . 172 Sandier , , Sartori G . 534 , , ,538 ,540 M . 534 Sartori , , . Schaeken ,W ,451

Richards , M., 424 Richards , W., 256 , 347 Richard son , J., 449 Riddoch , M., 532 , 535 , 541 , 545 Robin , F., 469 Rock , I., 126 Rosch , 353 , 368 , E., II , 344 Rosenkrantz , S., 368 Rosser , R., 566 Roth, J., 199 Rubin , J., 256 Rueckl , J., 562 Rumelhart , D., 539 , 544 , G., 448 Ryle

Tabossi , P., 449 Takano , Y., 131 , 144

584 Talbot , K., 198 , L., 10 , , II , 13 , 22 , 26n , 78 , 177 , 178 , 179 Talmy - 271 211 . 214 . 215 . 218 . 228 . 254 . 256 . 265 . 270 .
272n , 273n , 317 , 318 , 319 , 320 , 322 , 323 , 324 , 326 , 352 , 3590 , 365 , 375 , 386 , 399 , 404 , 405 , 420 , 422 , 423 , 426 , 4290 , 493 , 494 , 496 , 502 , 524 , 562 , 572 Tanz , C., 336 Tarr , M., 131 , 343 , 464 , 496 , 506 , 524 , 525 , 526 , 561 Tassone , E., 561 Taube , J., 280 , 557 , A., 542 Taylor , H., 88 , 184 , 470 , 479 , 482 , 483 Taylor Teitz , J., 569 Teller , P., 15 Tolman , E., 129 Toulmin , S., 443 , D., 548 Touretsky , E., 374 Traugott
d' Ydewal1e , G., 451 , F., 514 Young Zbrodoff , N., 502

Name Index

, E., 482 Tulving Turner , T., 469

, B., 5, 18 , 19 , 80 , 81 , 88 , 1050 , 133 , 160 , Tversky 184 -487 , 270 , 271 , 335 , 343 , 443 , 463 , 471 , 472 , 473 , 476 , 478 , 482 , 483 , 484 , 504 , 567 , 571 , M., 130 Tye Ullman Ol, 520 , S., 494 , 497 , SOO ,S , 521 -Ehrich Ullmer , V., 469 , 482 , 484 , L., 10 , 268 , 354 , 557 , 562 Ungerleider , A., 449 Urquhart Vaina , L., 347 Valvo , A., 152 VanCleve . J.. . 128 Vandaloise , C., 134 , 493 , 494 Vanderwart , M., 535 , 536 VanEssen , D., 355 , 557 , H., 14 Verkuyl , B., 567 Volpe vonderHeydt , R., 387

Wallach , H., 560 Warach , J., 355 Ward . T.. 349

, E., 532 , 533 , 534 , 535 , 537 , 540 , 542 , Warrington 546 , 547 , 548 Weidenbacher , H., 561 , 570 Weiskrantz , L., 557 Wernicke , C., 532 Wierzbicka , A., 422 Wilcox , T., 566 Wilford , J., 479 Wilkins , D., 399 Williams , R., 544 Wunderlich , D., 65 , 68 , 69

Subject Index

Aboriginal language , - - , frame of referencein , 125 -

184 - 300 Aboul , 299 , 304 - 90, 132 Above , 89 , 335 , 388 , 495 , 504 effecton figureobject , 323 to express socialstatus , 308 - 526 for, 497 , 505 template - 567 . Absolute directioncoding , 120 , 122 , 566 , 116 alsoFixedheading See system liabilitiesof, 146 in maprecall , 481 andperspective discourse , 486 preemption by, 94 - 570 in Tzeltal , 569 - 147 Absoluteframeof reference , 145 in American , 183 SignLanguage - 138 in horizontalspatialpatterns , 135 - 113 in Tzeltal , 111 - 90, 101 Absolute , 80 , 88 , 89 perspective -relative Absolute , theories , 128 space Absolute tense , 302 Abstraction - 260 in fictivity, 257 andI -language , 40 - 37 in lexicalsystems , 36 andmeaning formation , 376 neuralrepresentation of, 563 in perspective , 78 taking structural , 267 . See Abstractmotion alsoCaused motion , 215 ; Fictivemotiontheory ; Motion , in spatialcognitionSOl Aooeptability to consciousness , in fictivity, 248 Accessibility - 243 Access , 214 , 242 paths Across . 178 , 301 of, 258 concept effecton figureobject , 323 andobjectaxis , 327 of, 266 properties in Tzeltal , III

- 411 Action, 410 , 543 Actionability , 248 , 258 Active -detenninative , 226 principle Addressee , 259 , 470 Adjectives - 72 dimensional , 15 , 71 , 321 spatial - 242 Adventpaths , 214 , 241 Affixation , 173 African language , 396 , 400 After, 304 , 190 , 301 Against - 230 , 227 , 228 , 376 Agency , in fictivity, 227 sensory Agentive paths , in multipleperspectives , 89- 92 Alignment , in fictivity, 217 Alignment paths - 130 16 , 129 , 159n theory Allocentricity frame , 285 , 556 maps , 465 perspective , 560 space , 178 , 301 , 323 , 327 Along - 204 American , 171 , 183 SignLanguage Amid , 300 , 321 , 324 , 178 , 300 , 321 , 324 Among - digital ,5 , andCS Analog , 95, 174 Anaphora Anchorpoint, 142 , 148 . See entries under Axial Angles , 343 Animacy - 343 of, 342 , 349 judgment - 373 andmeaning formation , 372 in prelinguistic , 366 cognition - 371 Animals , 12 , 368 - 203 - 538 , 200 , 533 Aphasia , 211 , 237 Apparentmotion - 503 , 502 Apprehension - 526 , 493 , 524 Apprehension Aristotle boatcase , 126 , moored - 300 Around , 55 , 299 , 304

; " . = ~ 1 ,1 C I '= ~ " ~ ~ 0 ~ g . t c e a U o 8 . ~


. 8c 0 . 0 ~ tU ~

Arrernte language , 426n3 Articulation , 43 Art of the Fugue . 42 Aspect, 301 Associative ception fonn , 260- 261 At . 299- 300, 304, 324 , 7, 324 Atsugewi language Attachment, 319- 320, 351 Auditory -phonological interface, 4- 5 , temporal perception in , 173 Auditory system Auf- an. 320, 321 Awayfrom . properties of , 266 Axial basedlearning, 317- 360 - 19 Axial perspective , 474- 476 , 14 vertical, 135 Axial representation , 317- 319, 320, 325- 326, 358 for environmental comprehension , 472- 473 in mental models, 441 schematizingreferenceobject, 334- 344 Axial systemtheory, 14- 15, 19 , 63, 329

formationby, 370 motion , concept Biological


also . 130 . See reference

~ ~ f U . ~ Co = . I .

586

Index Subject

~ Deicti 395

- 95 Canonical , 92 position

Bach , 42 , J. S., musical example - 132 - 88 Back , 87 , 133 , 134 , 163n51 , 566 , 123 , 131 - 183 in American , 179 SignLanguage - 505 in apprehension , 504 theory -linguistic asnon , 389 concept - 143 relativeframeof reference , 142 in Tzeltal , 352 , 544 Backpropogation of, 146 Bali language , axes system Basic relations , 494 , asspatialindices - 57 Basic , 56 spatialtenDs - 147 frameof reference , 145 , in absolute Bearings , 299 , 304 Before - 299 Behind , 298 , 320 -linguisticperspective cross , 400 - 391 overextension , 390 - 336 schematization , 335 Behither , 298 Belhare , 27nl9 language - 289 Below , 287 , 335 , 388 , 495 - 313 causal relationuse , 311 effecton figureobject , 323 - 311 to express socialstatus , 306 in spatialapprehension , 504 - 526 for, 505 template - 287 in vectornotation , 282 - 311 - 294 Beneath , 293 , 306 Beside , 299 Between , 178 , 324 , 300 , 305 in American , 190 SignLanguage - 438 of, 437 concept - 298 , 296 , 305 Beyond tasks Bifurcation , 355 , andrepresentation , 12 , multimodalrepresentations Binding . See alsoStereopsis Binocular , 561 disparity

frame encounter framefor, 16 - 95 orientation . . . 561 - 16 - 89

- 343 determination of, 341

representationfor , 45 Cardinal direction , in , 466 - 567 Cardinal direction - - - - - - .09 , , 110 , 128 , 566 569- 570. Seealso Absolute direction coding in Guugu Yimithirr , 7 . SeeCardinal direction Cardinal direction system coding . SeeAbsolute direction Cardinal direction system coding Cardinal locations, 336, 339 Cartesian coordinate system , 187 , 193

perspective .. . coding

Case , 61 systems Case , 280 , 60- 62 theory - 370 . See also , 48 , 368 , 390 Categorization Relation judgment of, 563 child learning - 398 cross , 393 linguistic andmultiplestoretheory , 546 - 419 for, 411 patterns for recall , 482 semantic . 407 andshape , 563 - 420 - 398 , 402 , 403 , 393 spatial of visualobjects ,9 - 538 - 548 -specific , 547 , 533 system Category - 313 relations Causal , 311 , in semantic mapping . See alsoMotion;: Fictive 371 Caused . 374 - 410 - 420 for, 404 , 409 encoding in verbphrases , 377 - 261 , 214 , 245 , 260 Ceptiontheory , in fictivity, 248 Certainty Chamus , 400 language , 110 , Mexico Chiapas

Index Subject
Children. SeeLearners Chineselanguage . 8. 61- 62. 73 Chomsky theory. 33. 41 Clarity . in fictivity . 246 Classification. Seealso Categorization of spatial knowledge. 395- 398 Classifier systems . 61 Closed-classforms. 259. 267- 268. 321. 405. See also Open-classforms in fictivity . 264- 265 Coarse representation . schematizingfor . 327- 329 Coercion principle . 7 Cognition definition . 437 . 317- 359 developmental with diagrammatic icons. 455- 460 multidimensional . 24 and perspectivetaking . 464- 465 prelinguistic. 365. 386 . 353- 354. 387. 553 spatial. 43- 50. 195 for . 427n8 . 77- 78. 104 speaking thinking Cognitive map theory. 277- 314 Cognitive organization. 211- 213. 231 Color . 44. 52. 354 . and semantic Complementary phenomenon theory. 536- 538 . construction of . Compositional semantics 444 Computational theory. 493- 526 Computer logic. 444. 450 . SeeCeption theory Conceptual development . 50 Conceptual-intentional system Conceptualization in fictivity . 245- 246 - 130 opposition theory in . 129 prelinguistic. 369 of space . 43- 50 Conceptual lexicon. 47 - 123 . 122 . 554 . 120 Conceptual representation in computational theory. 498 effect of linguistic coding. 117 and frame of reference . 125 and organizing factors. 572 preverbal. 365- 380 Conceptual structure. 5- 8 and axial specification . 21 color . 52 and I -language . 40 time. 52 . interface of . 44 Conceptual system Conftation . by young learners . 334 Conjunctive forms ( Takano) . 131 Connectionist theory. 547- 548 Consciousness . 248. 268 Constructional fictive motion . 215 Constructivism. 372

587
Contactconcept , 386 - 398 - 378 cross , 393 , 377 linguistic -schema for, 374 neonate image andspatialtemplates , 526 Containment , 564 , 388 , 392 - 398 - 378 cross , 393 , 377 linguistic -schema for, 374 image ofin -on. I90- 193 in Korean , 322 of, 371 neonatal , 386 concept andspatialtemplates , 526 - 248 -structure Content , in fictivity, 247 frame Contextual , 17 , III , of objects Contiguity Control , 311 Conversion - 157 modalities across , 153 81 in perspective , taking - 82 Coordinate system in American , 179 SignLanguage in Gestalttheory , 126 - 141 of, 138 differences language , 137 ternary Coordination , 83- 88 , andperspective sharing Coreconcept , 533 theory Coreference , 174 , in American SignLanguage alsoBrain . See Cortex ; Brain ; Brainstem , 556 cortex function ; Parietal ; Hemisphere damage , 354 spatialfunction - 14 -mass distinction Count , 13 Countnouns , 344 , 345 , 214 Coverage paths - 244 in fictivity, 243 - 13 Criterionof economy , 12 , 21 effect Criterionof grammatical , 22 , 13 . See alsoInterfacing Criterionof interfacing , 13 - 428 Cross , 7, 134 , 385 linguisticanalysis -modality Cross definition , 158nl - 157 andspatialrepresentation , 152 -SRinterface . See CS Interfacing - 89 Cues , 88 , for perspective taking - 503 in tasks , 502 , spatialapprehension Cuing Culturaldeterminants , 102 , 569

Deicticframeof reference , 15 - 138 of, 135 contradictions - 132 , 131 appearance developmental of, 138 redefinition

588 Deicticperspective - 466 definition , 465 - 567 directioncoding , 566 in narrative , 185 Deicticperspective , 96 , 571 , 77 by, 94 preemption - 495 I Deicticrelation , 526n , 494 Deixisam Phantasma , 105n3 Demonstrative , 227 , in fictivity, 219 paths Demonstratives , 185 , in perspective - 402 , 487 , 401 , 83- 86 Descriptions - 487 andperspective , 463 , 399 , aslocatives Descriptors - 460 tools , 455 , asreasoning Diagrams ,5 , andCS Digital-analog - 72 Dimensional , 71 , 57- 59 , 15 adjectives , 49 , and I -space Dimensionality - 567 terms Directional , 566 , 272n8 , 99- 101 in cognitive , 279 theory mapping of, 95- 102 ellipsis . See Directionsystem Perspective - 568 Discourse . 104 . 468 . 562

Index Subject
- 130 directioncoding , 560 , 566 , 129 Egocentric a/soDeicticdirectioncoding 567 . See in, 561 distance perception , 556 maps , 465 perspective forms( Takano ), 131 Elementary , 95- 99 Ellipsis , 99- 103 deep of, 164n64 mapping Emanation , 214 - 231 of, 226 basis of, 235 evil eyeconcept - 232 andperception , 231 217 216 of , properties , 26n7 Embedding , 533 , S46 Encephalitis of, 401 Encirclement , concept - 420 learners , 404 , in language Encoding 235 in emanation , , theory Energy of, 50 Enter , I -space concept Entities , 278 , 568 Environmental , , andperspective taking cognition - 465 464 - 18 frames Environmental , 132 , , 92 , 130 , 17 - 473 471 information , 277 Episodic model , 478 , 471 Equiavailability , 569 ), 558 (Gogel tendency Equidistance Essence , , andfunctionalkind memberships - 564 563 -process distinction Event , 14 Evidentials , 259 Evil eye , 235 , in emanation theory - 572 of, 569 , effects Experience , 49 Explicittransfer , 268 Explicitvisualperception stimulation , neonate by, 370 Exploration - 418 - 391 Extensions , 416 , 390 , 215 -, 377 under of, 341 Externalregion , determination Extrinsicframeof reference , 132 - 467 Extrinsicperspective , 479 , 466 Factive , 211 , defined representation - 273 Fictivemotiontheory , 211 - 231 basis of, 226 - 262 andceptiontheory , 260 - 273 anddynamism , 270 - 224 fire in, 221 - 217 , 214 linguistic in maprecall , 481 - 269 andmetaphor , 268 - 244 - 226 , 236 , 217 path types - 260 in, 244 representations fictivemotiontheory Fictivity. See -ground . See also , 321 , 141 , 93, 110 Figure Referent ; Relatum

, cognitive, 211- 213 Discrepancy Disjunctions, in deductive problem-solving, 455- 460 . SeeMotion Displacement Dissociation, in fictivity , 261- 262 Distance, 558. Seealso entriesfor eachpreposition prepositions, 294- 295 Domain -specificlearning, 422- 424 Dorsal subiculum (rat ), 557 Down. 142 , 287- 289, 301 in cardinal direction system , III nonphysical useof , 308 in Tzeltal, III . Seealso Gaze tour . 133 Driving tour perspective

~ i~ :(Paivio Dual codinghypoth ), 5 Duration , 375 , 450 . 304 During - 398 Dutchlanguage , 394 , 351 . , 77 bipartiteproximity 378 of , concept support , 498 representation conceptual directiontermselection , 102 , 351 earlyspatialconcepts , 570 gestures -schemas , 421 image recallmemory , 114 - 116 relative , 115 coding - 410 408 , encoding spatial inference transitive , 120 visionboundperspective , 88 - 271 biasfor, 213 , 270 , cognitive Dynamism 543 , Dyslexia

-based . See a/soAbsolute frame Earth , 254 directioncoding ; Cardinaldirectioncoding

Subject Index
- 178 , 177 asymmetry in language definition , I 59n9 in fictivity , 217, 220 organization of , 561 in spatial relationships, 317 in spatial semantics , 402 theory, 357 visual, 93 Figure object, 322, 326 axial information on, 329- 334 geometry of , 323- 324 Fine-grained representation , 320, 344- 3SO Finnish language , 8, 393- 398 Fire, in fictivity , 221- 224 - 147 . Seealso Fixed bearing system , 128 , 145 Absolute direction coding; Cardinal direction coding - 125 use of gesture , 124 Fodorian hypothesis , 1- 2, 25n6, 41, 5S4 Folk iconography, 267 For , 281, 294- 295 Force dynamics CS - SR concept of , II in mapping, 307 neonateimage - schemafor , 375 sensingof , 256 Frame-relative motion , 214, 237- 240 . See Frames of reference , 5S4 , 159n17 , 127table a/so Deictic frame of reference ; Egocentric direction coding; Object-centeredframe; Perspective absolute, 556 allocentric, 285 - 184 in American Sign Language , 179 - 19 and axial vocabulary, 14 binary-ternary, 496- 497 computation for , 504- SO5 conceptual representationfor , 125 - 134 - discipline theory, 127 cross - 164 cross linguistic theory, 109 , 21 egocentric hierarchy of , 93- 94 orientation of , 92 - 127 spatial, 126 switching, 21 variability in , 463 Framework vertical constraint principle , 94 French language , conceptual representationin , 498 From, 288, 303, 304, 305 - 143 Front , 18 , , 163n51 , 142 , 133 , 134 , 87- 88, 123 352, 389 - 183 in American Sign Language , 179 in apprehensiontheory, 504- SO5 concept variability , 566 - 132 frame of referencetheory, 131

589
in maprecall , 481 in Nilotic cultures , 162n43 of, 217 properties Functionality andconceptual , 52 organization recallof, 542 - 541 for, 535 representation andSR hypothesis , 23
Gaze tour , 133 , 484 relative perspectiveof , 481 Generalization, 331- 333, 392- 393 Geographical frame, 17 of , , prepositional presence Geomagneticsense 315n3 Geometric frame, IS Geometric pattern, in fictivity , 248 Geometric representation , 317- 359 Geonic construction ( Biederman ), 8, 46 German language , 62, 63, 320, 321 concept of attachment, 351 concept of support, 378 dimensional adjectivesin , 73 direction equivalents , 324 , 351 early spatial concepts representationin , 498 Gestalt theory, 104 , 242, 243. Seealso Figureground dominant visual, 93 in fictivity , 269 frame of referencein , 126 influence on perspective , 102 Gesture, 156 - 125 and absolute direction coding, 124 , 570 in emanation theory, 234 Ghost physics , 234- 235, 388 Gibsonian tradition , 531 Global directional system , 296 Global framing, 238 Gloss, 204nI ' , 558, 569 Gogel s equidistancetendency Goodnessof fit , 50I Goodnessrating , 509- 514 Grammar acquisition of , 375 categoriesfor , 367 representationof , 554 spatial distinctions of , 59, 63 Gravity

of, 89 detennination of, 371 , 391 , 396 , 388 , 386 earlyconcept in spatialrepresentation , 472 , 281 , 17 , 132 in vectornotation , 285 Ground , 137 in, 479 Guidebooks , perspective

S90

Subject Index

andspatialcognition , 500 function , , in American SignLanguage Indexing 174 Induced motion in, 103 , 211 , 240 , 559 spatialexpressions Infants . See Learners variations ; Neonates ,7 syntactic Inference basis of, 460 . See Handedness alsoLeft-right , 131 -schemas schema for, 438(see alsoImage , 44, 78 ) Hapticinformation , 10 - 123 transitive Hausalanguage , 144 , 117 , 17 Inferentialpotential Headdirectionneurons , 81 83 , 557 Inferentialsatisficing Hebrew , 443 , 405 language - 203 -occipital Infixation . 173 Hematoma , right parietal , 201 - 311 - 203 of, 306 Influence function , metaphoric , 200 , 543 expression Hemisphere - 336 - 250 In/ rontof, 92 Hermann , , 16In34 , 320 , 335 , 161n31 grid, andfictivity, 249 400 , 495 , 466 , 533 , 546 encephalitis Herpes simplex cross , 137 , 129 , 557 , 567 linguisticperspective campus Hippo - 391 overextension aUocentric , 390 mapin, 572 . See of thought Innerlanguage alsoLanguage in cognitive , 109 , 279 theory mapping Inside . 324 in, 569 framework Euclidean function , of mapsin mammals , 556 , 277 , 556 Integration mapping - 13 role in perspective , 2, 3- 5, 10 , 465 Interfacing - 138 of I-language Horizontalpatterns , 43 , 544 , 135 - 299 to external . See alsoentries Internalstates Horizontalprepositions , 375 , relationship , 296 space Into, 351 for each preposition Intransitive verbs of, 377 , acquisition , 61 Hungarian language - 567 Intrinsicdirectioncoding variations , 79 , 566 ,8 syntactic - 141 of reference 15 96 Intrinsic frame 455 , 110 , 140 , , , Hyperproof in fictivity, 261 - 138 - 234 in horizontalspatialpatterns , 135 , 233 , in fictivity, 230 Iconography - 132 in younglearners , 131 , in fictivity, 247 Identifiability Intrinsicperspective I -language . See a/soI -space , 479 , in guidebooks , 33 , 64 , 73 Intrinsicrelations abstraction , definition , 454 , 40 - 62 of, 59 , 375 Introspection grammar -schemas of, 443 . See a/soInference Invalidity , knowledge , 9- 10 , 420 , 422 Image . See I-space alsoI -language , 546 high level definitionof, 44- 48 neonatal , 380 representation - 375 domain , 67 , 373 spatialrepresentation enter a/so Representation , 50 , 47. See Imaging Italian language , 351 encoding Imagistic , 260 ceptionforms Jackendoff single store theory, 44, 532, 547 , 101 representation andSR, 9- 10 , 77, 358 Japanese language , 49 Judgment, speedof spatial, 94 Implicit transfer , 268 Implicit visualperception Kantiantheory In, 304 , 151 , 569 , 401 , 335 , 351 , 357 , 377 , 385 - 566 absolute abstraction of, 564 , 128 space theory - 193 infonnation alsoHaptic Kinesthetic American , 44. See , 190 Language - 398 infonnation perspective , 134 , 393 Sof, 259 and I -space , linguisticforDl , SO , 55 Kinship of, 565 , spatial ,8 Knowledge mapping - 268 Koreanlanguage of, 264 , 396 , 401 , 321 , 357 properties - 324 in, 73 dimensional schematization , 322 adjectives - 526 for, 505 , 351 earlyspatialconcepts template in, 421 - schemas Indexing image of, 379 in-on morphemes in relation , 502 judgement in, 69- 70 locative constructions , 504 , 524 spatial

SigIJ cross linguistic

Subject Index - 410 - 420 in, 404 , 408 spatialencoding 324 , through equivalents in, 377 verbphrases KYST system , 517 , in computation theory set , 535 Ladybirdstimulus Lak language , 61 . See also of thought(Fodor ), 6, 554 Language of themind Fodorianhypothesis ; Language of, 569 cross -culturalunderstanding : of the mind, 1. See of alsoLanguage Localists . See Locationisttheory , in fictivity, 247 Localizability - 329 Location , 561 , 328 Locationisttheory , 281 , 62, 280 Locatives , 65, 389 of, 388 acquisition in American , 175 SignLanguage in, 399 differences cross linguistic in Gerdian , 32 in spatialrepresentation , 44 in Tzeltal , 352 , 439 Loglish

591

Language thought J1 .tinn Laterali7 , .in humanbrain , 277 and SR ,9

, Layout . See alsoNeonates Learners , 343 - 360 in, 317 axialunderstanding by, 563 categorization - 393 in, 385 earlyspatialconcepts - 344 in, 317 , 327 objectrepresentation in, 335 mastery prepositional - 348 schematizing by, 344 - 268 in, 267 abstraction structural - 357 in, 356 universals of, 50 Leave . I-space concept , 567 Left-right. 92 in American , 172 SignLanguage - 505 , 504 theory apprehension axisfor, III , 123 , 163n51 , , 134 , 162n42 , 162n47 335 , 566 of, 196 , 566 567 concept , 95 ellipsis maprecallof, 481 in perspective , 485 , 470 taking - 115 recall , 114 relativeframeof reference , 142 in spatialapprehension , 504 - 526 for, 505 spatialtemplate Leibnizian , 151 theory Lexicalsystems choice , 95- 96 - 24 , 103 , 19 , 77 concepts items , 5, 12 - 548 , 554 , 532 representation - 36 - 73 semantics , 34 , 32 , 33 - 57 , 56 spatialconcepts in Tzeltal , III - 224 , 221 Light radiation - 218 Line, intangible , 217 Linearization , 101 andCS ,6 in spoken , 185 languages Line of sight , 220 -spatialinterface , 1- 27 Linguistic Link schemas , 374 LISP , 446 Localframing , 238

, 482 , 77 , 103 Macroplanning in emanation Magicbeams , depiction theory , 233 , 355 Magnosystem , 3, 104 , 270 Mapping - 344 of axialinfonnation , 343 differences in, 560 of earlylanguage , 377 in tictivity, 230 in frameof reference , 152 theory framework of, 555 in hippocampus , 277 in I -language , 33 - 13 interface , 3- 5, 10 - 563 elements of linguistic , 562 motoric , 567 , I33 perceptual - 95, 102 in, 80 , 479 perspective in, 556 topography - 11 . See also2-D Marr-Biedennan , 10 theory Sketch ; 3-D Sketch ; 2- 1 (Marr) /2 D Sketch Marrianabstractions , 252 Massnouns , 14 , 110 , 396 Mayanlanguage -part descriptions , 162n38 body . See alsoMeaning , 386 packages Meaning ,6 encoding requirements - 373 fonnationof, 371 - 35 in lexicalsystems , 34 - 444 andmentalmodelconstruction , 438 - 380 , 365 preverbal - 555 andprimitives , 554 andSR, 12 - 376 , 375 , 380 , 365 Meaning packages - 117 , 570 , 113 , 114 Memory in maprecall , 482 - 200 of, 195 Mentalimage , 198 , generation Mentalmaps , 129 Mentalmodels , 46, 437 of environment , 471 comprehension -based , 470 language - 444 for, 438 representations propositional Mentalrotationskills , 570 Mentaltours , 469

592
for, 77 , preparation Message Metaphor - 269 in fictivity, 268 linguistic , 214 functionas , 281 preposition - 311 in verticalprepositions , 306 , 77 , 103 , 482 Microplanning , 541 Miming, to identifyobjects - 558 . See alsoBrain Mind, 555 Mirror reversals , 196 Mixteclanguage , 395 , 398 , 378 - 124 Mnemonics , controllingfor, 123 . See alsoMultimodalrepresentation Modalities - 134 -discipline cross , 125 understanding andCS ,6 - 152 and frameof reference , 149 , 109 logicin, 449 in spatialrepresentation , 9, 46- 47 , 45 transmodality - 444 Model , 437 , 460 - 126 , 43, 125 Modularity Chomsky theoryof, 41 definitionof, 158nl , I- 3 representational andspatialcognition , 392 , 438 - 13 of, 12 specialization 's , 152 , 157 , 110 Molyneux question , discrimination ability of, 535 Monkeys Mood, 258 , 102 , 80 Mopanlanguage - 293 More . 290 Morphemes - 268 closed -class , 267 - 408 of, 405 earlyproduction of 420 , mapping andobjectrelationship , 40I , 64 , in I -language categories Morphological - 32 , 173 , 31 Morphology Motion, 6, 388 aftereffect , 240 , 370 biological , 376 categorization by neonates - 271 biasfor, 270 cognitive of, II CS -SRconcept of, 391 earlyunderstanding - 420 - 402 of, 401 encoding , 404 - 273 fictive . 211 framefor. 15 - 560 . 559 illusory andmagnosystem . 355 -motion self . 373 . 377 . 371 andspatialrepresentation . 44- 45 Motoric maps . . andleft-right representation 567 Movement . See Fictivemotion ; Motion Multimodalrepresentation . 556
Multiple store theory, 532- 548 Musical fonnulation , 78

Index Subject

, 344 Naming - 546 of, 358 , 533 , 547 impainnent - 21 Narasimhan , 19 figures - 479 Narrative , 472 , 184 , 196 , 278 Nativism , 372 , 389 - 564 Naturalkind concepts , 563 , in children - 295 - 526 Near , 294 , 324 , 505 in American , 190 SignLanguage , 399 Negative objectparts in, 556 Neocortex , mapintegration - 389 . See alsoLearners Neonates , 563 , 152 , 388 - 546 Networktheory , 544 , 152 Neurophysiology - 548 , 531 Neuropsychology Neutralperspective , 470 , 468 Newtonian , 128 , 151 theory - 526 Next to, 505 - 227 , 226 Nonagentive paths , 386 , 389 coding Nonlinguistic in Tzeltal , 113 - 566 factors , 374 , 564 , 13 , 22- 23 Nonspatial - 568 . See alsoCountnouns Nouns , 567 andabstraction , 563 - 376 , 375 acquisition role of, 562 , 572 , 321 spatial - 13 , 345 , 10 Object - 402 , 400 conceptualization , III contiguity - 359 , 317 earlyrepresentation - 564 , 8, 563 encoding , 388 functionality identification , 354 localization , 354 - 373 formation andmeaning , 371 358 deficit for , naming andperspective , 464 , 371 properties , 561 recognition - 55 for, 54 schemata - 350 , 344 , 533 , 47 , 176 spatialrepresentation - 558 546 , 557 - 256 structure , 252 371 , support - 567 directioncoding -centered , 566 Object - 131 frame -centered , 132 , , IS, 26n9 , 130 Object frame . See alsoAbsolute 254 ; Cardinaldirection ; Deicticframe coding , 273nll , 247 Objectivity fonnation Observation , 372 , andmeaning - 18 . See also frame -centered Observer , IS, 17 Deicticframeof reference ; Deicticperspective

Subject Index

593
anddistance , 558 representation - 232 andfictiveemanation , 231 - 246 in fictivity, 245 andprelinguistic , 368 categorization visual , 44, 211 , 10 , andSR Perceptual encoding - 548 interface , 531 Perceptual 1 , 573n Perceptual organization - 320 Permanence , in Piagetian , 317 theory - 546 Perseveration recall , in naming , 544 Person deixis , 466 - 106 . See direction alsoAbsolute , 88 , 77 Perspective ; Cardinaldirectioncoding ; Deictic coding directioncoding directioncoding ; Egocentric ; Frames of reference ; Intrinsicdirectioncoding - 184 in American , 179 SignLanguage - 469 of, 468 choice - 498 in computational , 497 theory - 487 of, 463 descriptions drivingtour, 133 for, 95 multiplesystems - 486 neutral , 484 - 478 - 484 , 482 , 473 switching visionbound , 88 - 34 Phonetic features , 33 4 172 543 , , , Phonology -syntaxinterface ,2 Phonology effect , in fictivity, 250 Phosphene - 130 - 320 , 335 , 343 , , 129 , 317 theory Piagetian - 367 - 388 365 , 391 , 387 . See 2-D sketch Pictorialrepresentation ; 2- 1 /2 D sketch ; 3-D sketch (Marr) Place , 26nl4 in cognitive , 278 theory mapping - 13 -SRconcept of, 10 CS ashigherlevelconcept , 26n14 of, 567 linguisticrepresentation andobjects , 321 Place cells , 556 Place deixis , 466 , , in American SignLanguage Pointingfunction 174 of reference Pointof view . See alsoFrames ; Orientation - 183 in American , 179 , 184 SignLanguage Polishlanguage , 40I choice Politeness , 485 , in perspective , 498 , 499 Polysemy Possession , 48, 259 Postsubiculum , 280 , 296 - 104 - 95 - 471 , 80 , 103 , 470 Pragmatics - 255 , 254 Precognition Predications , 95 , 89- 92 , in multipleperspectives Preemption 390 366 387 , , Prelinguistic cognition andspatialtemplates , 525

- 300 Omnidirectional , 299 prepositions - 398 On. 321 , 394 , 398 , 401 , 335 , 351 , 377 - 193 in American , 190 SignLanguage effecton figureobject , 323 - 526 for, 505 spatialtemplate - 52 , 48, 51 Ontology - 386 , 385 Open . See -class forms -class forms alsoClosed Open in fictivity, 264 , 405 morphological , 30I Opposite - 130 , 129 theory Opposition - 546 , 541 , 547 Opticaphasia - 239 Opticflow, 238 Orderpreservation ,5 (OUCH), Organized unitarycontenthypothesis 532 Orientation , 230 path, in fictivity, 217 Orientation theory - 157 frameof reference , 16 , 131 , 155 andI -space , 49 andspatialrepresentation , 45 of, 189 specification Originalsin, 368 , 403 OriginalWord Game Ostension , in fictivity, 247 OUCH theory , 532 Oul. 385 , 428nl6 of, 411 semantic classification - 526 Over . 351 , 525 , 505 - 391 - 418 . Seealso Overextensions , 215 , 390 , 416 Extensions ; Underextensions 17 Paillardframes of reference , 1590 - 214 - 248 -related , 245 , , 213 parameters Palpability - 251 249 Parietal cortex , 354 , 556 , 557 Partarticulation , 347 , linguisticformsof, 259 Particularity Parvosystem , 354 Past . 305 Path , 26n14 , 185 in cognitive , 280 theory mapping - 13 of, 10 concept of, 405 descriptions encoding - 226 in fictivity, 217 ashigherlevelconcept , 26n14 - 374 -schema for, 373 neonate image 255 256 of 253 , , sensing of concept , 376 , acquisition Patiency Patternpath , 214 - 237 in fictivity, 236 570 , Perception definition , 437

16 Off , 428n semantic classification of. 411

594 Prepositions, 65. Seealso entries undereach preposition acquisition of , 377- 378, 388 axial occurrences , 22 in conceptual representations , 498 as figure-ground indicators, 178 lexical representationsof , 68- 69

Subject Index frame Reference , 524 in computational , 499 theory - 254 of, 253 sensing - 505 in spatialapprehension , 504 500 andspatialcognition , Reference , 32 , 321 , 322 object - 178 in language , 177 asymmetry in coordinate , 126 system in fictivity, 217 , 220 - 326 of, 324 geometry Reference , 305 , with temporal prepositions points -ground . See alsoFigure Referent ; , 104 , 175 , 78 , 92 Relatum of acceptability , , andspatialcognition Regions - 497 496 Relation , 502 judgment - 526 - 400 , 398 , 493 Relationships - 129 ~ , theories Relative -absolute , 127 spa - 138 ~ , 110 Relative frameof referen , 142 , 135 ~ . See alsoDeicticframeof referen 144 , 179 Relative , 467 perspective Relative system , 79 Relative tense , 302 . SeeaboFigure Relatum , 126 , 137 , 78 , 104 ; Referent ground - 555 . See abo , 437 , 554 , 8- 10 Representation ; Templates system Representation for action , 543 in American , 186 SignLanguage in, 21 axialspecification criteria , 8- 9 -cultural cross , 571 - 357 in, 353 differences of distance , 558 - 473 for environmental , 470 comprehension - 350 -grained fine , 344 for infront of , 336 asinterface , 44 of I -space , 64 - 500 kindsof, 497 of, 81 independence language multimodal , 560 in neonates , 563 andobjectfunction , 541 andobjectrecognition , 463 , 133 perception - 373 - 380 , 372 , 365 preverbal - 444 , 438 propositional frame andreference , 499
retinotopic , 10 semantic , 39, 533 of sensoryquality , 538- 541 shapebias, 345- 347 - 195 in signedlanguages , 193

. . encoders asshape , 176 - 323 - 314 , 321 , 15 , 277 spatial andspatialcognition , 525 - 380 Preverbal , 365 learning Primes . See alsoprimitives , 54 conceptual in I -language , 59 , 73 semantic , 66 , 72 , 125 in UG, 34 - 555 - 375 - 140 Primitives , 557 , , 378 , 554 , 374 , 138 573n2 -intentional of conceptual , 50 system andCS ,5 in I -space , 64 in modularitymodel ,I - 425 role of, 422 semantic , 532 , 443 Probability judgments -solving Problem , 392 , 570 - 501 Process , 500 , in spatialcognition , 501- 503 , in spatialapprehension Programs relators , 134 Projective Deicticperspective . See Pronominal deixis - 79 formulation , 78 Propositional , 34 Prosody , in fictivity, 217 paths Prospect for mapping , , shortcomings image Prototype - 12 11 of, 388 , earlyconcept Proximity facets Pseudointrinsic , 144 , 133 , andframeof reference Psycholinguistics of language , , and frameof reference Psychology 132 ), 53 (Pustejovsky Qualiastructure ,6 , andCS Quantification

spatialS, 554 - 572 , cultures , 571 stabilityacross

Subject Index
for temporal relations, 448- 455 three types of , 531 visual, 101

595 Shape
- 329 , 330 , 327 ignoring andobjectnames , 563 of, 131 recognition - 347 for, 345 , 358 representations andSR , 8- 9 - 350 bias , 345 Shape and Metzlerparadigm , 131 Shepard - 225 . See a/soVisualsystem , 152 , 224 Sight - 205 . See alsoAm Pcrican , 171 Signed languages SignLanguage , 368 Similaritytheory - 398 cross linguisticstudyof, 393 - 520 , 514 rating , 542 testing Since . 304 , 305 - 242 Sitearrival . 241
-basedschematization . SeeAxial -based Skeletal learning 2- 1/ 2 D Sketch (Marr ), 2, 9- 10, 130 , 142 , , 134 160n20 2-D Sketch (Marr ), 133 , 156 , 323 3-D Sketch (Marr ), 2, 8- 9, 12 , 46, 48, 133 , 134 , 156 , 437, 532 in concept, 323 and object schemata , 55 and theory of vision, 130 , 151 , 455 Snakes , map integration in , 556 Social categories , 48, 468 Social cognition, 6, 23 Social determinants, 484, 487 Social relation , 8, 306- 311 , 68, 394- 398, 399, 405 Spanishlanguage concept of containment, 378 en, 321 in , 421 -schemas image lexical concepts , 77 , and conceptual organization, Spareconcepts 53- 54 -centeredsystem , 79 Speaker ' Speakers knowledge status, linguistic forms of , 259 , speedof articulation , 173 Speech SR-CS interface. SeeInterfacing Standard averageEuropean (SAE), 94, 105n8

in younglearners , 317 , 39- 40 , 1- 3, 24 modularity Representational . See alsoRepresentatiol1 Representation system - 337 for axialinformation , 335 109 and frameof reference , , 156 's - 157 and Molyneux , 152 question of, 42 multiplefunctions - 224 Reverse -direction , 223 conceptualization Richconcepts , , andconceptual organization - 54 53 Rotation , 567 , 47, 148 , 163n51 - 198 , 195 ability in deafness andgoodness of fit , 514 for shape , 131 recognition - 19 Route , 18 , 479 , 482 - 503 Routines , 501 , in spatialapprehension - 442 Rules of inference , 439 , , in mentalmodels - 448 446 Ruletheories , anddiagramuse ,~ - 73, 351 Russian , 40I , 72 language -framedlanguages Satellite , 404 - 287 Scales , 286 , in vectornotation - 268 . See also2-D, Schematization , 318 , 323 , 266

2-1 /2 D, 3-D sketch ) (Marr - 344 of reference , 334 object Scotoma , in fictivity - , 249

-motion . See alsoCaused Self , 373 , 377 , 371 motion ; Motion ; FictiveMotion Theory Semantic Features , 424 Hypothesis Semantics in, 381n4 breakdown of, 32 components - 420 cross , 403 linguisticcategorization - 39 formsin, 36 - 541 functionafter stroke , 538 mapfor, 278 - 548 of, 533 pathologies - 425 andprimitives , 422 - 532 , 531 processing - 380 for, 101 , 365 representations 538 541 andsensory quality, - 391 - 393 , 387 , 389 , 391 spatial structure in, 402 Semitic , 173 languages - 367 - 388 Sensorimotor , 387 , 390 , 365 period - 257 , 6, 12 , 252 Sensory encoding - 226 , in fictivity, 224 Sensory paths - 541 , 546 , 538 Sensory quality mode , 270 perspectival Sequential Serialvisualroutines , , in computation theory - 524 520 Shadow , 224 , 226 paths

. - 239 - 271 Stationariness , 270 , 211 , 238 . See alsoBinocular , 10 disparity Stereopsis Stickfiguredrawings , 267 , in structure sensing ~ , in fictivity, 248 Stimulus dependen , in fictivity, 246 Strength functionafter Stroke , 537 , semantic ~ , 502 Stroop - like interferen

596
, in fictivity, 250 Structurality - 268 Structure , 386 , 264 , 40 frameof reference , 130 Subjective motion , 215 Subjective - 224 , 233 , 221 Sunlight colliculus ), 556 (mammalian Superior in emanation , theory , depiction Superman 233 , 563 , 368 Superordinates , 392 , 386 , 388 concept Support - 398 - 378 cross , 393 , 377 linguisticfactor -schema for, 374 neonatal image in neonates , 371 andspatialtemplates , 526 Surface , 95- 103 ellipsis , 482 , 479 Survey -grounding ), 372 (Hamad Symbol mode , 270 perspectival Synoptic , 26n7 Syntax in American , 174 SignLanguage structure andconceptual ,7 in I -language , 64 in perspective , 185 in phonology ,4 - 32 of, 31 semantic component bias andshape , 345 - 380 for, 365 base spatialrepresentation - 314 in vectorgrammar , 313 in vectornotation , 282 Tabassarian , 61 language Ta in Tzeltallanguage , 321 , 227 , in fictivity, 220 Targeting paths of, 540 Taste , impairment ,6 , andCS Taxonomy . See alsoRepresentation Templates in computational , 499 theory - 526 - 497 , 505 , 498 , 496 spatial for spatialcognition , 500 cortex , 354 Temporal , 14 , 173 Temporality - 546 in error patterns , 544 277 of , mapping - 375 in neonates , 374 - 455 , 444 , 438 reasoning and rulesof inference , 448 in scene identification , 356 andspatialrepresentation , 44- 45 lobe Temporal function , 277 memory in naming task , 537 deficit andobjectnaming , 358 - 558 andobjectproperties , 557 - 305 . See alsoentries , 301 prepositions Temporal preposition for each . See a/soTzeltal , 110 , l04n2 language Tenejapan language

Subject Index frameworks , 156 - 125 , 124 gesture inference transitive , 120 Tense , 264 , 301 , 258 of, 44 Texture , integration Thalamus (rat), 557 , 437 Theoryof reasoning activation Theta , 557 . See Cognition Thinking , 427n8 , 104 , 77- 78 Thinkingfor speaking - 478 Third-person , 473 , in narratives -waysystem Three , III - 324 Through . 301 , 323 , 324 of, 266 properties - 380 , 379 Tight-fit concept . See alsoTemporality Time , 52 andaxialsystems , 23 of, 48, 438 conceptualization - 269 , 268 mapping linguistic - 375 for, 374 neonate concept , 214 spatialconnection andspatialrepresentation , 44- 45 - 305 andtemporal , 301 prepositions To. in semantic , 305 mapping -typedistinctions Token ,6 , andCS , 352 Top relators , 134 Topological - 320 , 317 representation Topological . See a/soDescriptions Tour, 469 ; ; Descriptors Gazetour Transformation , 314 , 286 Inference . See inference Transitive verbs Transitive , 377 - 83 , 82 Transitivity alsoModalities , 45. See Transmodality ' deicticcenter (B\ihlers), 147 Transposed in 479 Travelbooks , perspective , Turkishlanguage , 405 , 143 , 68- 69 Typology - 113 . 134 . 176 . Tzeltallanguage , 110 , 62, 105n8 321 , 325 , 395 - 570 directions andabsolute , 569 -part system , 352 body of side , 140 concept in, 570 gestures in, 70 constructions locative andmixedperspective , 80 system in, 73 adjectives positional - 116 relative , 115 coding - 333 in, 331 spatialpredicates in, 324 spatialpreposition characteristics , 350 unique -boundperspective andvision , 88 Tzotzil Mayanlanguage , 418 , 428n18 - 391 - 311 - 293 Under , 389 , 390 , 134 , 308 , 289 - 313 relationuse in causal , 311

Subject Index to express socialstatus , 306 in Gennan , 32 - 526 for, 505 spatialtemplate - 391 . See Underextensions alsoExten, 377 , 390 sions ; Overextensions - 35 Universal structure , 33 , 73 , 125 Until, 304 , 305 - 289 , 390 , 287 Up, III , 142 in perspective , 470 taking - 314 - 314 Vectorgrammar , 277 , 280 , 281 - 548 Verbalrepresentation , 532 Verbs , 405 - 377 of, 375 , 376 acquisition , 225 agentive in American , 176 SignLanguage in Atsugewi , 324 andCVA recall , 543 , 62 positional - 213 in, 211 , discrepancy Veridicality Verification tasks , in spatialapprehension , 503 , 88 , 544 Verticality - 294 - 296 . See Verticalprepositions also , 282 , 295 entries for each preposition Vestibular cues , 92 Via. 301 -based Viewer frame . See Viewer - centered frame of reference - 131 Viewer - centered frameof reference , 130 , 132 , 134 . See alsoEgocentric directioncoding , 254 Virtual motion , 215 Virtual structures , 265 - 257 Visualsystem , 256 - 262 , 261 ceptiontheory fictivity patternin, 213 in, 130 imagery - 540 andneuropsychological , 533 impairments - 558 , 557 processing - 124 recallin, 123 - 548 from, 2, 101 , 532 representation - 533 semantics of, 532 andspatialdevelopment , 387 relative to auditorystream , 173 specialization theoryof, 151
. SeeLearners Young learners

597

Z -dimension , in v~ tor notation , 282 Zinacantan , I S8nS language

Whorfianhypothesis , 114 , 125 , 377 , 379 , 571 definition , 404 of spatialrepresentation , 102 Wit Munganlanguage , 147 Word count , 336 Word learning , 563 X-ray vision , in emanation , 233 theory in maprecall , 479

What , 10 , 268 , 354 , 355 , 557 , 561 , 562 , 563 system Where system , 10 , 268 , 354 , 355 , 356 , 399 , 561 , 562

Anda mungkin juga menyukai