DOI 10.1007/s10936-012-9235-1
Abstract Dickinson and Szeligo (Can J Exp Psychol 62(4):211222, 2008) found that
processing time for simple visual stimuli was affected by the visual action participants had
been instructed to perform on these stimuli (e.g., see, distinguish). It was concluded that
these effects reflected the differences in the durations of these various visual actions, and
the results were compared to participants subjective ratings of word meaning but it was
also possible that word characteristics like length might have influenced response times. The
present study takes advantage of word length differences between French and English visual
action words in order to address this issue. The goals of the present study were to provide
evidence that (1) the processing time differences previously found were due to differences
in the cognitive actions represented by these words (and not due to characteristics to the
words themselves), and (2) that individuals subjectively differentiate visual action words in
such a way that allows for predictable differences in behaviour. Participants differentiated
14 French visual action words along two dimensions. Four of these words were then used in
the instructions for a size-discrimination task. Processing time depended on the visual action
word in the instruction to the task and differed in a predictable manner according to word
meaning but not word length.
Keywords
Introduction
In all languages, a great selection of words exist to describe visual actions. For example,
if someone wished to describe devoting mental resources towards visually processing a
presented stimulus they could choose a number of the following words/phrases to describe
J. Dickinson (B) L. Cirelli
Department of Psychology, Laurentian University, Sudbury P3E 2C6, ON, Canada
e-mail: jdickinson@laurentian.ca
Frank Szeligo
Department of Psychology (Retired), University of New Brunswick, Fredericton, Canada
123
that action: see, notice, observe, attend to, be aware of (to name a few). Cacciari and Levorato
(2000) suggested that the choice to describe a visual action with one of these words instead
of another reflects that these actions differ along some defining characteristic of perception.
Therefore, by understanding how the words used to represent these actions are differentiated
from one another, we can better understand the perceptual properties of the actions themselves
(Miller 1991; Cacciari and Levorato 2000).
A number of studies have now explored how individuals differentiate visual action words
from one another (e.g. Cacciari and Levorato 2000; Dickinson and Szeligo 2008; Pasanen
1978; Schwanenfluegel et al. 1994; Shabanova 2000). Although these studies provide us
with a greater understanding of how we categorize these words, they do not provide insight
into how these differentiations relate to behaviour. The unique contribution of Dickinson and
Szeligo (2008) is in their development of a behavioural measure for these words. Specifically,
by instructing participants to perform the visual actions represented by these words and
measuring the time that it takes for them to do so, the relationship between participants
understanding of the words and the impact of these words on processing time and accuracy
in simple visual tasks could be assessed.
With this measure, Dickinson and Szeligo (2008) showed that the time required to visually
process stimuli (both words and images) significantly differed in terms of the visual action
that they were asked to perform on these stimuli. For example, when participants were asked
to respond with a mouse click immediately after they see that an image was presented,
they responded faster (mean RT = 325 ms) than if they were asked to perceive it (mean
RT = 369 ms) or become conscious of it (mean RT = 382 ms). Such results were found in
various task types (responding to a word, responding to an image, and a go-nogo task)
suggesting that the effect of these words on cognitive performance was consistent (Dickinson
and Szeligo 2008).
Dickinson and Szeligo (2008) compared the behavioural results to subjective ratings of
word meaning, and so participants were asked to rate the similarity in meaning of 14 visual
action words in the The Mental Operations: Rating of Sameness Scale (Dickinson and Szeligo
2008, adapted from Cacciari and Levorato 2000). Using multidimensional scaling (MDS)
analysis, it was found that 78 % of the variance between the subjectively rated word meanings
could be explained by a one dimension solution. The word ratings along this one dimension
followed the same ordinal pattern as the processing times found in the above mentioned
experiments and so it was concluded that individuals had some meta-cognitive knowledge
of how these words differed in terms of the resulting perceptual processes. That is, participants appear to have differentiated the words along a characteristic that has a salient affect
on perceptual processing. There were however issues with Dickinson and Szeligo (2008)
conclusions, which are listed below and will be addressed in the current paper.
The Possible Effect of Word Characteristics on Processing Time
Although the processing time effects found were consistent with differences in perceived
meaning (as rated by participants), there were also other word characteristics (both objective
and subjectively rated) that correlated with response time. Specifically, words eliciting longer
response times were longer in length, led to greater certainty that they had been performed
correctly, were more difficult to perform, and were understood, by the participants, to take
longer to perform (see Dickinson and Szeligo 2008 for a more detailed description of these
relationships).
This raised the question of whether the results of the behavioural measure truly reflected a
measurable difference in the cognitive process being performed or if results were a byproduct
123
of the mental action words characteristics. Dickinson and Szeligo (2008) argued that since
participants are responding to stimuli that they are asked to perform certain visual actions
on, and not to the words representing these visual actions themselves, word characteristics
should not affect processing speed. However, this requires experimental verification.
Previous research on words and their impact on behaviour tends to evaluate participants
responses to the actual word itself. For example, Pulvermller et al. (2001) found that, when
asked to respond only to actual words but not to pronounceable pseudowords (lexical decision
task), participants responded much faster to face-related verbs (e.g. smile) than arm- (e.g.
throw) or leg-related (e.g. run) verbs. Pulvermller et al. (2001) concluded that these response
time differences reflect a difference in word meaning retrieval speed. Such retrieval speeds
can be greatly affected by characteristics of the word itself like frequency, familiarity and
length. Dickinson and Szeligo (2008) claim that their methodology is not measuring the rate
of word meaning retrieval but the actual processing time of the visual action represented
by the word. Demonstrating that word characteristics like frequency of use and word length
(which affect the speed of word meaning retrieval) are not associated with the processing
time differences would support this claim.
The Ability of MDS Ratings to Predict Behavioural Differences
Dickinson and Szeligo (2008) concluded that because the multidimensional rating results
followed the same ordinal pattern as the response time differences found, the aspect of word
meaning that participants considered when differentiating words along this dimension might
account for the time differences. If this were true, then multidimensional scaling results
(which inform us on how participants are differentiating these multiple words) could be used
to accurately predict the direction at which the visual actions represented by these words vary
in terms of processing time. On the other hand, the processing time results found by Dickinson
and Szeligo (2008) might simply be an artifact of the four words chosen for their specific
task. If this relationship exists within and generalizes across languages, then Dickinson and
Szeligo (2008) interpretation can be supported.
The Present Study
The present study uses Dickinson and Szeligo (2008) multidimensional scaling and behavioural measure with French visual action words in order to confront two main goals: (1)
to support the idea that Dickinson and Szeligo (2008) methodology is measuring the performance of the cognitive actions represented by the visual action words and not merely
participants responses to the words themselves and (2) to demonstrate that the way participants discriminate between these words in terms of meaning can be used to predict measurable
components (specifically processing time) of these actions.
Because of the nature of the English visual action words used by Dickinson and Szeligo
(2008), a direct investigation on the impact of word length on processing time was impossible.
However, since the French counterparts of some of these words varied in word length from
their English counterparts, the processing time of French vision words that differed in word
length but not multidimensional rating could be taken advantage of. If it were found that the
actions represented by French visual action words differing in rating but not word length did
differ in processing time, then the first goal of the present study could be supported.
Also, if such processing time differences were found in French, this would demonstrate
that previous results were not isolated behavioural artifacts to English visual action words.
More importantly, if multidimensional scaling of the French words can lead to directional
123
predictions in how processing time is affected, the second goal of the present study could be
supported.
The first experiment of the present study gathered subjective ratings of French visual action
word meaning and assessed them using Multidimensional scaling. This was a necessary step
before conducting Dickinson and Szeligo (2008) behavioural measure of these words in
French since these ratings could be used to help select words for the behavioural measure
that differ in either meaning (subjective rating) or word length. By being able to select these
words based on meaning and word length, it could be determined which of these factors acts
as a predictor for processing time differences. This was the goal of the second experiment.
If word meaning and not length predicted processing time differences, then the validity of
Dickinson and Szeligo (2008) measure would be supported.
123
Table 1 The 14 words/phrases from the Mental Operations: Ratings of Sameness Scale (French and English)
(Dickinson and Szeligo 2008) with MDS values for English and French 2 dimensional solutions, with words
and values sorted by FMDS1 values
FRENCH
ENGLISH
French MDS 1
French MDS 2
Distinguer
Distinguish
1.66
0.29
0.64
Regarder
View
1.51
1.21
1.23
Voir
See
1.32
0.99
0.88
Identifier
Identify
0.64
0.76
0.36
Discerner
Discern
0.57
1.59
1.50
Percevoir
Perceive
0.23
0.13
Reconnatre
Recognize
0.03
Prter attention
Attend to
0.02
Remarquer
Notice
0.1
0.52
0.03
Dtecter
Detect
0.36
0.39
0.27
Se rendre compte de
Be aware of
0.54
0.12
0.20
tre conscient de
Be conscious of
1.21
0.17
0.39
tre au courant de
Be cognizant of
1.27
0.86
0.05
Ressentir
Sense
2.45
1.54
1.40
0.6
1.16
English MDS
0.19
0.15
2.58
solution to a three dimension solution was not significant, therefore a two dimensional solution was selected. The two dimensional solution can be seen in Fig. 1 (the specific coordinates
of the French words along each dimension can be seen in Table 1).
To evaluate if these words were being differentiated in a way similar to the English
differentiations, correlations between the coordinates of the French visual action words on
each dimension of the solution and the coordinates of the corresponding English words
from the single dimension solution found by Dickinson and Szeligo (2008) were calculated.
For the first French dimension 1, the value was a non-significant pearson r of .30. For the
second French dimension, the correlation was r = .85, p < .05. The word ratings for the
first and second French dimensions were not related to word length or to either frequency
measure.
Discussion
Comparison to English Dimension
The fact that Dickinson and Szeligo (2008) found a one-dimensional solution and the
present experiment found a two-dimensional solution for visual action word meaning
is not terribly surprising. In both cases a significance test was used to evaluate if the
next dimension accounted for a significant amount of additional variability. As with
stepwise multiple regressions, this procedure has been known to be somewhat unstable
(e.g Thompson 1995).
Also, despite the fact that only one dimension of word meaning was found in the English
solution, the English and French vision words still seem to be differentiated in a similar manner (the second French dimension had a correlation of r = .85 with the English dimension).
Such similar word meaning differentiations suggest that when these words are differentiated
in terms of meaning, it is truly the abstract action represented by the words (which should be
123
universal) that is being considered. The English MDS should therefore be repeated in future
studies to determine if increasing statistical power might lead to results more similar to those
found in the present study (specifically, finding two significant dimensions).
123
123
Table 2 Response time per instruction from current study and (Dickinson and Szeligo 2008)
French
Response time
Instruction
Mean
Voir
1141
English
Response time
SD
Instruction
Mean
SD
396
See
1410
639
Regarder
1149
387
Recognize
1453
611
Discerner
1166
394
Are conscious
1468
680
Ressentir
1239
452
Distinguish
1554
731
Results
Processing Time
For all variables, repeated measures ANOVA were conducted with an alpha of .05. Processing
time data was averaged across trials prior to analysis. An instructional main effect was
found for processing time F(3,117) = 4.85, p < .05, 2 = .11. Post-hoc tests indicate voir
ressentir: F(1,39) = 10.28, p < .05, 2 = .21, regarder ressentir: F(1,39) = 6.89, p < .05,
2 = .15, and discerner ressentir: F(1,39) = 5.24, p < .05, 2 = .19 to be significantly
different. The instructional means are displayed in Table 2 along with the response times
from Dickinson and Szeligo (2008).
123
Accuracy
No Instruction main effect was found for proportion of hits F(3,117) = 1.89, p > .05 or correct
rejections F(3,117) = 1.06, p > .05. Mean proportion of hits across all instructions was .75,
while mean proportion of correct rejections across all instructions was .74.
Same Versus Different Trials
No significant differences for response time or accuracy were found between same and different trials. Signal detection analysis was also performed, however no significant differences
were found on either sensitivity or criterion measures.
Discussion
The results of this experiment suggest that word characteristics such as length and frequency
do not contribute to the processing time differences found. The words voir and regarder
(which differ in word length) were rated almost identically on both dimensions and did
not differ in terms of the processing times elicited. As well, the processing times for the
regarder and discerner conditions significantly differed from the processing times in the
ressentir condition despite the fact that all three words are highly similar in word length.
Together, these results suggest that word length and frequency do not relate or contribute to
the processing time differences found.
Instead, the ratings of word meaning along the first dimension appear to account for the
processing time differences. Ressentir, which led to significantly longer processing times
than the other three instructions differed from these other three instructions along the first
dimension but not from discerner along the second. As well, discerner differed greatly
in dimensional rating from regarder and voir on the second dimension but not the first
and these three instructional conditions did not lead to significantly different processing
times. Together, these findings suggest that it is the factor(s) that lead individuals to differentiate these words along the first dimension of word meaning and not the second that also
account for differences is processing time. So, in simpler terms, the results suggest that the
cognitive characteristic of the action ressentir which differs from the other three words is
associated with the increased processing time when the action represented by this word is
performed.
General Discussion
Since word length and frequency are not related to the ratings of word meaning, the relationship found between dimensional rating and processing speed can be attributed to the
meanings of the words and not to the characteristics of the words (which commonly affect
word meaning retrieval time). This supports (Dickinson and Szeligo 2008) claim that their
processing time measure allows for quantitative measurement of the cognitive actions represented by visual action words and not merely the effects of word meaning retrieval. These
findings support the first goal of the present study, and further validate Dickinson and Szeligo
(2008) behavioural measure.
The second goal of the study (to demonstrate dimensional ratings of word meaning can be
used to predict measurable components of these visual actions) was also supported. Significant processing time differences were found, and are associated with the dimensional ratings
123
10
along the first dimension of word meaning. The fact that these processing time differences
were found in a different language using different words suggest that the processing time
effects found by Dickinson and Szeligo (2008) were not simply an artifact of the particular
English words that had been chosen for their task.
Theoretical Implications
Based on associative models of verb meaning, results of this study support the hypothesis that
the meanings of the visual action words is directly reflective of the visual action represented
by these words. This associative model of verb meaning (Pulvermller 2005) suggests that
increasingly strengthened connections form between a verb and the corresponding action
performed or observed directly before, during or after hearing that verb. Human neuroimaging
evidence has supported this model by demonstrating that processing an action word relating
to the face, arms or legs activated the area of the motor strip containing the topographical
representation of the related body part (Pulvermller et al. 2001; Hauk and Pulvermuller
2004).
This model corresponds to Barsalou (1999) perceptual symbols theory which suggests that all perceptual input relating to a concept contribute to its meaning. For example, the action of kicking (which commonly occurs before, during or after hearing the
word kick) contributes to our understanding of the concept represented by the word
kick. If we extend the theories of Pulvermller (2005) and Barsalou (1999) to abstract
concepts like the vision verb see, our findings suggest that it is possible that the
introspective perceptual experience that we come to associate to the word see may
come to define our understanding of the concept of seeing. So when participants are
asked to differentiate between the meaning of these visual action words, they may be
drawing on the introspective information that represents their understanding of these
actions and so distinctions between words are based on distinctions between perceptual characteristics that can differ among these words. This is supported by the finding that visual action words are differentiated in a similar way in both English and
French.
Future Directions
The results of this study suggest that these vision verbs are learned in a similar way within
and across language, supporting the associative models of verb learning (Pulvermller 2005;
Barsalou 1999). Therefore, the next logical step would be to evaluate cortical activity using
ERP technology to compare activity when reading the different vision words within the
instructions (therefore targeting differences in word meaning retrieval for action words with
no verifiable product) and when this action is performed on stimuli following the instructions
(therefore targeting differences in resultant processing when performing these actions).
The previous (Dickinson and Szeligo 2008) MDS showed only a weak relationship with
the present MDS dimension that was relevant to response time. Future studies might also
investigate the characteristics of these visual actions that might vary from one word to the
next and how these differences contribute to the ratings of word meaning found. Specifically, Dickinson and Szeligo (2008) hypothesized that one of these factors may be level of
processing; the cognitive actions represented by each visual action word may differ in terms
of how deeply information is processed when these actions are performed. Future research
might seek to investigate this hypothesis further.
123
11
Conclusion
The goals of the present study were both met. We have provide evidence that (1) the processing
time differences that occurred in Dickinson and Szeligo (2008) are due to differences in the
cognitive actions represented by these words (and not due to characteristics to the words
themselves), and (2) individuals subjectively differentiate visual action words in such a way
that allows for predictable differences in behaviour. The consistent findings of how people
differentiate words both within and between languages suggests that we may be learning
visual action words through associative learning in a similar fashion to more concrete action
verbs (e.g. Pulvermller 2005; Barsalou 1999).
References
Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(4), 577660.
Baudot, J. (1992). Frquences dutilisation des mots en franais crit contemporain. Montreal: Les presses
de lUniversit de Montral.
Cacciari, C., & Levorato, M. C. (2000). The semantic structure of vision verbs: A psycholinguistic investigation
of Italian. European Journal of Cognitive Psychology, 12(1), 87106.
Davison, M. L. (1983). Multidimensional scaling. New York: Wiley.
Dickinson, J., & Szeligo, F. (2008). Impact of mental operation instructions. Canadian Journal of Experimental
Psychology, 62(4), 211222.
Hauk, O., & Pulvermuller, F. (2004). Effects of word length and frequency on the human event-related potential.
Clinical Neurophysiology, 115, 10901103.
Miller, G. A. (1991). Lexical echoes of perceptual structure. In R. Lockhead & J. R. Pomerantz (Eds.), (pp.
249261). Washington, DC, US: American Psychological Association.
New, B., Pallier, C., Brysbaert, M., & Ferrand, L. (2004). Lexique 2: A new French lexical database. Behavior
Research Methods, Instruments & Computers, 36, 516524.
Pasanen, M. L. (1978). Finnish and English verbs of vision. Further contrastive papers. Jyvaskyla Contrastive
Studies, (6), 128.
Pulvermller, F. (2005). Brain mechanisms linking language and action. Nature Reviews Neuroscience. Special
Issue: Focus on Pain, 6(7), 576582.
Pulvermller, F., Hrle, M., & Hummel, F. (2001). Walking or talking?: Behavioral and neurophysiological
correlates of action verb processing. Brain and Language, 78(2), 143168.
Shabanova, T. (2000). Semantic and pragmatic interface in English verbs of vision. http://fccl.ksu.ru/winter.
2000/paper5.pdf.
Schneider, W., Eschman, A., & Zuccolotto, A. (2002). E-Prime (Version 1.0). [Computer software and manual].
Pittsburgh, P.A.: Psychology Software Tools Inc.
Schwanenfluegel, P., Fabricius, W. V., Noyes, C. R., & Bigler, K. D. (1994). The organization of mental verbs
and folk theories of knowing. Journal of Memory and Language, 33, 376395.
Thompson, B. (1995). Stepwise regression and stepwise discriminant analysis need not apply here: A guidelines
editorial. Educational and Psychological Measurement, 55(4), 525534.
123