See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/281069602

When image and text meet - teaching with visual and multimodal texts

Article · January 2011

CITATIONS READS

0 1,887

1 author:

Jon Callow
The University of Sydney
40 PUBLICATIONS   193 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Jon Callow on 26 May 2016.

The user has requested enhancement of the downloaded file.

 PETAA
Paper 181
When image
and text
meet:
teaching with visual
and multimodal texts
Jon Callow
http://www.advertolog.com/wwf/print-outdoor/lungs-11711955/

Being immersed in a visual culture, we can sometimes not even This means that we need to make sure
notice how we interact with images and words. This photo of we don’t just read a text like a picture
the logged forest catches our eye, summoning the image of a book, talk about one or two literary
pair of lungs. The poster comes together with the conservation features, discuss which picture we like
group’s logo and the tag line ‘Let the planet breathe’, making the best then move on to the next activity.
a powerful point. Multimodal texts make meaning because
all the elements work together to create
Engaging with various texts can sometimes seem a simple,
a whole text. This is the case for picture
everyday act, belying the complexity involved. Until something
books, using only image and text, or for
catches our eye, jarring us, or delighting us with its simplicity,
video and multimedia, where image,
its beauty or its drama, we may not stop and reflect more
gesture, movement, words and sound all
deeply about what we have seen or experienced. It is when
work together to create the final piece.
we do stop that we wonder how the author, artist or designer
communicated their message so effectively.

The concept of multimodal texts is integrated in the Australian

This e-update will:
Curriculum: English, across the language, literature and literacy
strands. From understanding social contexts, expressing and ●● explore multimodality, with a
developing ideas through language and literature to creating focus on image and text
new texts, the multimodal nature of English is deeply embedded ●● review key visual metalanguage
in the rationale and strands of the curriculum. Teamed with
●● demonstrate how to plan
a knowledge of written language is the inclusion of a visual
effective modelled viewing
language thread (ACARA, 2011:66), as well as assessment lessons using picture books
linking to multimodal texts, expressed in the achievement
standards from Foundation to year 10. ●● present examples of annotated
picture book images to guide
A key concept to understand when working with multimodal classroom practice
texts is that the whole is greater than the sum of the parts.

© 2011 PETAA – Primary English Teaching Association Australia. PO Box 3106 Marrickville Metro NSW 2204 Tel 61 (0)2 8020 3900 www.petaa.edu.au ISSN 1442603X 1 ......

Consider this advertisement. Initially we
are drawn to the fin on the left, perhaps
reacting with a sense of fear. The caption
reinforces this, yet when we look to the
right, the fin is gone, but the caption
suggests this is ‘more horrifying’. Isn’t
this a contradiction? Is it because the
shark is under the water? What could the
advertisement mean? Reading the text on
the bottom, the purpose becomes clearer
– the loss of wildlife has an impact on the
ecosystem. This is the same ecosystem
on which humans also depend. The
poster cleverly plays with fear of sharks
and the juxtaposition of image and word,
in order to convey its message.
We can learn through multimodal text. Our
emotion and intellect have been engaged.
However, we can also learn about this
advertisement, by understanding how
both the written and visual elements work,
as well as reflecting on what world-view
or ideology is being presented (Callow,
1999; Kress & Van Leeuwen, 2006).
Visually the fin is the most salient
element in the poster, with the adjective
‘horrifying’ shaping our interpretation
and response, as well as augmenting the
image. This juxtaposition of two images
...... 2 © 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet
http://www.advertolog.com/wwf/print-outdoor/shark-13804905/
is powerful because it sets up a question in the viewer’s mind,
a hook to get attention. Reading left to right is also important
here. We move from the shark being present – the ‘given’,
to its apparently sudden disappearance – the ‘new’. This new
information is possibly more unsettling. The dark blue colours
further add to a fearful atmosphere, as well as the fact that the
fin in the first photo (and by association the shark) is framed
to appear quite close to us as the viewer.
The placement of the large photos at the top, with the more
detailed text and logo down the bottom is also common. Many
advertisements put their main ideas at the top, considered the
‘ideal’ in western culture, while the more everyday aspects of
the product, such as the details, ingredients or specific features,
are often at the bottom, symbolically grounded in what we can
term the ‘real’.
The written text here is also powerful – ‘Exploiting the ecosystem
also threatens human lives’. Strong action verbs like exploiting,
and threatens create an immediate emotional reaction, especially
when linked to the noun group human lives.
From a critical literacy aspect, we can also ask about their
world-view as advocates for wildlife. Is the information true –
would the loss of sharks really impact us as humans? Should we
think about supporting this organisation? Reflecting critically
allows us the space to step back and consider the points of
view presented. (See Callow, 2008.)
Exploring multimodal texts can range from advertisements,
posters and videos to one of the mainstays of any English
classroom – the picture book.
Modelled viewing and step and explore how they work together
(Walsh, 2010). Pedagogically this will
picture books include reading the whole text, revisiting
key pages and events in the story,
Teaching viewing using picture books provides both exposure
explicitly teaching an aspect of the visual
to quality literature as well as a stimulating variety of visual
and written text and how they work
images and techniques. The following provides a guide for
together, then consolidating the concepts
choosing and using pictures books with a focus on the visual
and skills in a concluding activity.
and written text interplay.
In a modelled viewing lesson (whole class
Read and enjoy or small group), a teacher demonstrates
key visual and written features as well
Having chosen a piece of quality literature, make sure you
as involves their learners in a variety
read it and enjoy it, as your own knowledge and enthusiasm is
of discussions and hands-on activities.
important and will always positively influence your teaching.
Within one lesson, a teacher may move
across a scaffolding continuum, from
Literary themes and features more teacher-regulated practices (reading
As you are teaching a whole text, consideration of the broader the text, making statements about key
literary features should inform your lesson planning (Saxby, features, asking focused questions about
1997). These elements are also key aspects of the Australian features) to more scaffolded activities,
Curriculum: English. Features include: where students take some responsibility
for practising the demonstrated skills or
● Theme – ideas that hold the story together, such as
knowledge (Hammond, 2001). Ideas for
friendship, survival, family, courage
activities can include:
● Plot – traditional narrative structures or variations on this
● sorting a selection of images into
● Characters – major and minor characters, heroes and shot distances or types of angles
villains, animals as characters
● listing key features such as salient
● Point of view – first person or third person points, characters that demand
viewer attention or effective use
● Place or setting – realistic (urban, city, bush), fantasy
of colour
or imaginary, historical
● re-drawing a character so that they
Choose a key page from the story either look at the viewer (demand)
or away from them (offer)
There are often key moments in a story, which are represented
in word and image. When planning, choose a key page, note ● creating a frozen moment scene
down the main ideas and the visual and verbal features that are from the story and using a digital
important in creating the ideas and feel of the page (Refer to camera to experiment with
the samples on the following pages). Table 1 gives a summary angles, framing or shot distance.
of features that may be present. Only choose the features that
are relevant or strongly represented on the page.
In the classroom
Scaffolding multimodal meanings Consider the example of Fox on the
The middle column in the table brings together the visual following page, which shows an overview
and written aspects of the texts to ensure that the multimodal of a chosen double page spread. The
aspects are explored. Rather then just listing specific visual following ideas are suggestions for
features and grammatical aspects, we need to take that extra planning a modelled viewing lesson.

© 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet 3 ......
 TABLE 1 ● MULTIMODAL MEANINGS – IMAGE AND TEXT
Visual Multimodal meanings Written
(Concepts of print and screen; Visual Guiding principles to consider (Structures of different types of
Language*) when image and text meet texts; Sentence and clause level
grammar; Word level grammar;
Vocabulary*)

Happenings In the context of the unfolding Events and Ideas

Actions and settings – how line and shape story, how does the page Sentence and clause structures
represent people, objects and events, develop the characters, story – building happenings and ideas
including speech and thought events and build themes and through different verb types
conceptual and symbolic –images showing concepts? How do the visual and adverbs/ adverbials
ideas and written work together? – description of characters and
setting through noun groups
and adjectives

Reacting and relating How do the visual and written Interacting

shot distance – long, mid or close text shape our feelings and statements, questions and
angle – high, low or eye level reactions to the characters? commands
gaze of characters – offers and demands Do we react differently to the attitudes and opinions
colours – moods and symbolism visuals than to the written text? modality – degrees of certainty
shapes and textures and their impact Do the literary features literary features – alliteration ,
realism of images – photo, artwork, drawings, complement or contradict the repetition, rhyme, personification,
diagrams visual elements? metaphor; interesting vocabulary
choices

Design and layout How does the layout of Text organisation

balance – symmetry and contrast of elements the visual and written text genre/ text type –literary
top/bottom, left/right, centre/margin – influence the reader/viewer? (narrative, ballad, recount etc.) or
placement of elements Does the layout privilege factual
salience – what attracts attention particular visual or written paragraphs
reading paths – vectors directing our gaze elements? Why might the sentence order and structure
framing – how text and image may frame author/illustrator have made
each other these choices?

* - denotes thread from Australian Curriculum: English (ACARA, 2011)

For more details on visual grammar, see Callow, 1999, and for more details on written grammar, see Derewianka, 2011

Before Reading

Building prior knowledge During Reading

Discuss the cover, finding out what
preconceptions learners have about foxes.
List words associated with foxes
Explain the concept of a fable: a short story
with a moral
Prediction
Could this Fox be a hero or villain?
Look at the endpapers - what does the
harsh red paint suggest?
Why is there a dog with a magpie in its
mouth before the title page?
Have a balanced number of questions and think-aloud statements that
allow the story to unfold, as well as prompt comprehension for events
and themes.
Think aloud statements and questions
Dog and Magpie build a strong friendship at the start, as Dog is faithful
and kind.
Why is fox always watching? How does he tempt Magpie? Should
Magpie go with Fox?
The descriptions of Fox as having ‘haunted eyes’ and a ‘rich red coat’
are very effective.
Why does Fox leave her in the desert?
Is Fox’s final cry a scream of triumph or despair?
After viewing the final endpapers, discuss whether Magpie gets home
safely or not.

...... 4 © 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet
Fox by Margaret Wild and Ron Brooks

Themes: friendship, loyalty, betrayal

Synopsis: A fable of friendship and treachery where Dog and Magpie’s

relationship is challenged by Fox’s arrival in the bush.

Key features and context of image

As the orientation moves into the complication of this narrative, Fox
arrives after Dog and Magpie have become close friends. A feral animal,
he doesn’t belong either in the bush or in Dog and Magpie’s friendship.
This page foreshadows the betrayal by Magpie, with Fox coming between the two friends, visually and
in the written text. Fox’s flattery sees Magpie initially recoil in fear but her feelings change as his words
influence her in the unfolding story.

TABLE 2 ● MULTIMODAL MEANINGS – AN ANALYSIS OF FOX

Visual Multimodal meanings Written

Happenings – Magpie is pulling away,
laying on back, showing her fear. Dog is
looking off the page, appearing trusting
or possibly naive? Fox stares at Magpie –
are his intentions friendly or evil?
Fox says one thing but does he
mean another? Does his image
match his words or contradict
them? What do the words
and image of Dog and Magpie
suggest about their future
friendship?
Events and Ideas – “You looked
extraordinary” – a flattering statement
but is Fox telling the truth?
Dog beams but Magpie shrinks away –
use of ‘but’ joining the clauses shows
contrast between the two characters.

Reacting and relating – The colours
give an earthy feel, with Fox’s red
standing out as he dominates the page.
Red is symbolic of danger, influencing
our emotional response, as does the
messy, handwritten lettering.
How does the use of colour
build up our feelings about
each character? Are the strong
verb choices reflected in the
image? Or do the images tell us
something more?
Interactions – Strong verb choices
build tension in the story.
Magpie shrinks away – shows her fear
Dog beams – shows his innocence
Fox staring at her burnt wing –
suggests evil

Design and layout
Fox and Dog on the left, opposite Magpie
on the right, gives an unbalanced feel.
Magpie is now separated, small and
vulnerable at the bottom of the page.
Fox is at the top, in the ideal position,
appearing dominant.
Fox’s speech box is vertical. Both Fox
and his speech box come between Dog
and Magpie. His words are the centre of
the page and the centre of attention too.
Why did Ron Brooks put the Text organisation – No clear
speech box vertically? Because paragraphing or text layout adds to
it is askew, does it make the confusion and the conflict that is
words in it mean something growing. Fox’s words and actions
different? (look for other create fear in Magpie.
examples in the story where the
text isn’t horizontal. What does
it signal about the characters’
relationships?)

Teaching strategies Colour

Show the double page spread, either using the book or Discuss possible symbolic meanings of
projected on a screen or an interactive whiteboard. the colour red – danger, power, strength,
romance, passion. Do any of these apply
Vectors and gaze to Fox?

Ask a student to trace along the gaze of Fox to Magpie (or draw
an arrow on the IWB). Re-read Fox’s statement. Pose some of
the questions in the multimodal meanings column (see Table 2).

© 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet 5 ......
 Layout
Discuss Fox’s flattery. Point out how
his words and body are curving around
Magpie. How does she react? How do we
feel about Magpie?
Reactions
Circle or label the eyes. Do they look at
us (demand) or elsewhere (offer)? What
does this tell us about the character’s
feelings or intent?
- Circle each character’s eyes - who is
looking at who? Do they look at us?
Draw a line down the middle of the page.
Dog and Fox are on the left, separated from Magpie.
Why might Ron Brooks have done this?
Visual literacy and Interactive White Boards
The interactive tools and techniques available on Interactive White Boards extend the creative
ways of teaching about multimodal and visual texts (see Goodwin, K. e-update 017, 2011)
Pens and highlighters
Ask students to annotate an image, circle key
features or draw vectors and reading paths
Drag and drop
Match labels to parts of the page such as
description of angles or shot distances, noun
groups or verb types.
...... 6 © 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet
Written grammar
Box some of the strong verbs and discuss their impact using
the multimodal meanings questions.
Key narrative point
How do the text and pictures mark a change in the story? How
does this scene change the friendship between Magpie and
Dog? How does it make us feel about each character?
Fox and his words are in the centre & at the top of the page.
Both are framing Magpie, trapping her in the corner. How is
Magpie reacting? How do we feel about Magpie?
Strong vector from Fox’s gaze - who has the power?
- Box the powerful verbs e.g beams, shrinks,
staring. Do they complement or contrast
what we see?
Manipulate elements
Using elements from a page (such as
characters or objects), re-design the page to
tell a different story, re-size them to make
one element salient, or create text boxes to
manipulate the font, size and the position of
text on the screen.
Gleeson, L., & Blackwood, F. (2009). Clancy & Millie and the
very fine house
Themes: friendship, play, imagination

Synopsis: Clancy moves to a new house, which is much too big and very
lonely. However, Clancy meets Millie and together they find friendship,
building towers, trains and houses for the three little pigs.

Key features and context of image: This second double page spread
of the book contrasts strongly with the first more colourful one, where
the three little pigs cavort across the sky. As orientation blends into
complication, this page plays with angles, proportions and colour, placing Clancy in a home that is
overwhelming in its size and loneliness. Visually this pattern is repeated over the next four pages.

Table 3 presents the multimodal meanings that can be read into this text.

TABLE 3 ● MULTIMODAL MEANINGS – AN ANALYSIS OF CLANCY & MILLIE AND

THE VERY FINE HOUSE
Visual Multimodal meanings Written

Happenings – Clancy and his parents,
appearing very small, stand outside and
gaze upwards at the imposing new house.
The cloudy shape of a pig leaps back to the
previous page, with the faint outline of a
wolf chasing it.
The written text provides the
dialogue here, while the visual
gives more information about the
characters. What do the pictures
tell us about what they are doing
and where they are looking?
Events and Ideas – The second
part of the sentence from the
previous page (Clancy has moved
from this house ) is completed
here, to reinforce the contrast
between homes (.. .to this house).

Reacting and relating – Like the
characters, we are at a low angle, with the
buildings purposely drawn with tall, angled
walls leaning ominously forward. Using a
long shot, Clancy and his parents appear
small and powerless. The colours are drab
with dirty browns, further affecting our
feelings towards the house.
How do the colours make us
react? Why do you think Freya
Blackwood drew the buildings
on an angle? We are low down,
looking up at them – how does it
make us feel?
Interacting – Each character
makes a statement about the
house. Clancy’s parents describe
the house as new, best, and a very
fine dwelling. Tellingly, Clancy
whispers rather than says that
it is ‘too big’. Clancy’s mother’s
positive attitude and comments
continue over the next four pages,
while Clancy yearns sadly for his
old home.

Design and Layout – The very small
characters are in the lower left-hand side,
looking powerless, while the buildings,
made salient by their size, are on the
right. Visually, the right-hand side can
be understood as what is ‘new’ in the
unfolding narrative.
The words describing the house
are on the left. The picture of the
house is on the right – do they
complement or contrast each
other? Why are the houses made
to look so big? How do you think
it makes Clancy feel?
Text organisation – The written
text is also in the lower left-hand
side, where the description sits
in opposition to the house being
described.

Teaching strategies Reactions - Clancy’s parents sound

confident when they describe the house.
After having read the book, come back to the first two pages.
Do they look confident?
Use the annotated image to explore angle, size and descriptions.
Key narrative point - How do the
Vectors and gaze - Circle Clancy and his parents. Trace
story and pictures tell us about Clancy’s
the gaze of their eyes. What are they looking at? Why?
feelings? Are the next pages similar? Keep
reading to see if there is a pattern.

© 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet 7 ......
 Circle Clancy and his parents - how big or small are they
Can you see a wolf in the clouds? What feel does he give?
compared to the house? Why might they be drawn this way?

Box the noun groups describing the house - compare Trace or draw along the vertical lines of the houses - how do
them to the pictures. Why might they be different? you think the angle of the houses makes the family feel?
A low angle shot can often make the viewer feel powerless

Conclusion
Working with multimodal texts provides literacy skills as part of engaging with and enjoying the rich
opportunities to develop our learners’ visual culture we live and learn in.
love and interest in literature, enhance
For further information and resources for teaching about visual
their knowledge of visual and written
and multimodal texts, visit the website at www.petaa.edu.au
language, as well as develop their visual
Refere nc es
ACARA. (2011), The Australian Curriculum: English Available from www.australiancurriculum.edu.au/English/Curriculum.
Callow, J. (1999), Image Matters – Visual Texts in the Classroom, PETAA, Newtown, Sydney.
Callow, J. (2008), ‘Show Me: Principles for Assessing students’ visual literacy’, The Reading Teacher, 61(8), 616-626.
Derewianka, B., (2011), A new grammar companion for teachers (2nd ed.), PETAA, Newtown, Sydney
Goodwin, K. (2011), ‘Engaging students in literacy learning with interactive whiteboards’, e:update, 017, PETAA, Newtown, Sydney
Hammond, J. (2001), Scaffolding : teaching and learning in language and literacy education, PETAA, Newtown, Sydney
Kress, G. R. & Van Leeuwen, T. (2006), Reading images: the grammar of visual design (2nd ed.), Routledge, London, New York.
Saxby, H. M. (1997), Books in the life of a child : bridges to literature and learning, Macmillan Education Australia, South Melbourne.
Walsh, M. (2010), Multimodal literacy: researching classroom practices, PETAA, Newtown, Sydney
While every effort has been made to trace and acknowledge copyright and ownership of all included works, should any infringement have
occurred, the publishers offer their apologies and invite copyright owners to contact them.

About the author

Dr Jon Callow is an experienced educator and teacher, having worked in primary schools, universities and in
professional development for teachers. He currently teaches at the University of Sydney, in the areas of English and
multiliteracies and he has worked alongside teachers in their classrooms, both in Australia and the United States.
His research interests include visual literacy, children’s literature, social justice and pedagogy and digital media.

PE TAA Papers
Originally published as PENs (Primary English Notes 1 to 162), and more recently known as e:updates (1 to 18), these informative monographs
or articles, are now known as PETAA Papers. Following the 2011 change to PETAA (Primary English Teaching Association Australia) and to
recognise the rich history of the 180 previously published PENs and e:updates, it was decided to continue the number progressively from PEN 1
and thus the first PETAA Paper is numbered 181.

...... 8 © 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet
View publication stats