Multimodal Transcription and Text Analysis


In the last eight years, the authors have worked with many audiences in many different countries in teacher training seminars, university courses and conferences concerned with research into multimodality to develop a sophisticated but easily-understandable analytical framework providing at least some answers to these questions. This book presents a scalar model of multimodality with a focus on the relations between the discourse stratum and multimodal systems and structures (words, images, movements, sounds, ...
See more details below
This Hardcover is Not Available through
Sending request ...


In the last eight years, the authors have worked with many audiences in many different countries in teacher training seminars, university courses and conferences concerned with research into multimodality to develop a sophisticated but easily-understandable analytical framework providing at least some answers to these questions. This book presents a scalar model of multimodality with a focus on the relations between the discourse stratum and multimodal systems and structures (words, images, movements, sounds, etc.) in both static and dynamic texts.

About the Author:
Anthony Baldry is Associate Professor in English Linguistics, Faculty of Medicine and Surgery, University of Pavia

About the Author:
Paul J. Thibault is Professor of Linguistics and Media Communication, Agder University College, Kristiansand, Norway

Read More Show Less

Product Details

Table of Contents

Foreword     xi
Acknowledgements     xiii
Preface     xv
Introduction: multimodal texts and genres     1
Introduction     1
Multimodal texts and the resource integration principle     4
Resource integration and the transcription of printed cartoons     7
Multimodal transcription of cartoon narratives and the question of the metafunctions     16
Sources of meaning in multimodal texts     17
Cluster analysis and the transcription of static multimodal texts     21
Multimodal transcription and questions of genre     30
Textual properties of short printed cartoons     34
Printed advertisements and their exemplification of the metafunctions     38
Metafunctions in relation to genre     38
Web pages and their transcription     44
Film texts and their transcription     46
The soundtrack     51
Conclusion     54
The printed page     57
Introduction     57
The printed page and its evolution     57
The resource integration principle in the scientific page     61
How can we study tables systematically?     64
How does the page communicate?     68
Science textbooks and multimodal meaning making     70
Visual, verbal and actional semiotic resources in a table     71
Visual and verbal resources     71
Thematic development of the page: hierarchies of textual periodicity     74
Actional semiotic resources     78
Blood under the microscope: multimodality in a photographic display     78
Integration of scientific photographs and verbal text     80
The textual metafunction     80
The ideational (experiential and logical) metafunctions     82
The interpersonal metafunction     89
The Italian texts: differences with respect to the Australian texts     91
Reading paths     91
The use of colour     92
Expertise and authority vs. comprehensibility and accessibility     93
Linguistic resources     96
Visual resources     99
Conclusion     102
The web page     103
Introduction     103
Page or screen?     105
Decoupling of material support and information on the computer screen     109
The relationship between web page, website, web users and web genres     113
The home page     118
The Nasa Kids home page      120
Creating a hypertext pathway     126
The British Museum Children's COMPASS website     130
Children's COMPASS home page: description of multimodal objects     130
A multimodal hypertextual thematic formation: daily life in Asia     136
Thematic system analysis: preliminary observations and an example     136
Multimodal thematic system development along a hypertext pathway     140
The action potential of hypertext objects     146
Experiential meaning     147
Interpersonal meaning     148
Textual meaning     153
The virtual world of hypertext     156
Community or social network of users and practices?     161
The WWW as technological infrastructure and meaning-making resource     162
Conclusion     164
Film texts and genres     165
Introduction     165
The Eskimo text: a macro-analytical approach to transcription     167
The Westpac text: an integrated approach to transcription     174
Etic and emic criteria in multimodal transcription     181
Phases, subphases and transitions     184
Column 1: Row number and time specification     186
Column 2: The visual frame      187
Visual frames and shots     187
Information structure: Given and New     189
Sequencing and relations of interdependency between shots     190
Column 3: The visual image     191
Specifying visual information     191
Perspective     195
Distance     195
Visual collocation     198
Visual salience     199
Colour     199
Coding orientation     200
Visual focus or gaze of participants     200
Column 4: Kinesic action     202
The meaning of movement     202
Interpersonal modification of movement     206
General observations on the notation of movement     209
Column 5: The soundtrack     209
Integrating auditory phenomena     209
Sound acts and sound events     210
Dialogic relations among sound events     211
A brief comment on the notation of the soundtrack     214
The rhythm of sound events     214
Accented rhythmic units     216
Rhythm groups     216
Degree of loudness     217
Duration of syllable, musical note, sound event     218
Tempo      219
Continuity and pausing     220
Dyadic relations among auditory voices: sequentiality, overlap, turntaking     220
Vocal register     221
Column 6: Metafunctional interpretation     222
Metafunctional notation in relation to Column 6     222
Display and depiction: two sides of the same semiotic coin in visual texts     223
Multimodal discourse analysis: the Mitsubishi Carisma advertisement revisited     223
From delimited optic array to visual text: the stratification of the visual sign     223
Transformations in the optic array: some examples from the Mitsubishi Carisma text     228
Visual transitivity frames and experiential meaning     230
Identity chains in visual semiosis     232
Dependency relations in the Mitsubishi Carisma text: implications for visual texts     234
Some sources of coherence in the Mitsubishi Carisma advertisement: Phase 1     239
Counter-expectancy and hypertext in the Mitsubishi Carisma advertisement     242
Conclusion: the shape of things to come     248
References     251
Appendix I     I
Appendix II     261
Index     265
List of Insets relating to Keypoints
Context of situation and Context of culture      2
Text     3
The resource integration and meaning-compression principles     18
Metafunctions     22
Clusters and cluster analysis     31
Bakhtin's distinction between primary and secondary genres     43
Phases and their transcription     47
Intertextuality     55
Projection     101
The trajectory     116
Visual transitivity frames     122
Scalar levels     144
System and instance     172
Material object text and semiotic action text: two sides of the same textual coin     175
Gibson's optic array     192
Perspective in sound: Van Leeuwen on Figure, Ground and Field     212
Recontextualising social practices     213
Stratification     236
Negotiation     245
List of Figures
Codeployment of space and hand-arm movements in a car advertisement     5
The two Marmaduke cartoons     8
Cluster transcription of narrative event structure in a cartoon     9
A London Transport leaflet (top part: cover side; bottom part: reverse side)     25
A macro-transcription of part of the LT leaflet     28
A micro-transcription of the cover page of the LT leaflet      29
Parts of a leaflet for the Chesapeake Bay tunnel-bridge complex     32
Clusters making up the reverse cover of the leaflet     33
Lupo Alberto and attitudinal stance     36
Read More Show Less

Customer Reviews

Be the first to write a review
( 0 )
Rating Distribution

5 Star


4 Star


3 Star


2 Star


1 Star


Your Rating:

Your Name: Create a Pen Name or

Barnes & Review Rules

Our reader reviews allow you to share your comments on titles you liked, or didn't, with others. By submitting an online review, you are representing to Barnes & that all information contained in your review is original and accurate in all respects, and that the submission of such content by you and the posting of such content by Barnes & does not and will not violate the rights of any third party. Please follow the rules below to help ensure that your review can be posted.

Reviews by Our Customers Under the Age of 13

We highly value and respect everyone's opinion concerning the titles we offer. However, we cannot allow persons under the age of 13 to have accounts at or to post customer reviews. Please see our Terms of Use for more details.

What to exclude from your review:

Please do not write about reviews, commentary, or information posted on the product page. If you see any errors in the information on the product page, please send us an email.

Reviews should not contain any of the following:

  • - HTML tags, profanity, obscenities, vulgarities, or comments that defame anyone
  • - Time-sensitive information such as tour dates, signings, lectures, etc.
  • - Single-word reviews. Other people will read your review to discover why you liked or didn't like the title. Be descriptive.
  • - Comments focusing on the author or that may ruin the ending for others
  • - Phone numbers, addresses, URLs
  • - Pricing and availability information or alternative ordering information
  • - Advertisements or commercial solicitation


  • - By submitting a review, you grant to Barnes & and its sublicensees the royalty-free, perpetual, irrevocable right and license to use the review in accordance with the Barnes & Terms of Use.
  • - Barnes & reserves the right not to post any review -- particularly those that do not follow the terms and conditions of these Rules. Barnes & also reserves the right to remove any review at any time without notice.
  • - See Terms of Use for other conditions and disclaimers.
Search for Products You'd Like to Recommend

Recommend other products that relate to your review. Just search for them below and share!

Create a Pen Name

Your Pen Name is your unique identity on It will appear on the reviews you write and other website activities. Your Pen Name cannot be edited, changed or deleted once submitted.

Your Pen Name can be any combination of alphanumeric characters (plus - and _), and must be at least two characters long.

Continue Anonymously

    If you find inappropriate content, please report it to Barnes & Noble
    Why is this product inappropriate?
    Comments (optional)