Multimodal Composing

Using Structure and Form as a Rhetorical Frame for Multimodal Composing

By Paul Dan Martin

Introduction

There is a lack of research on pedagogical approaches for teaching students how to rhetorically design multimodal messages. Writing instructors teaching multimodal composition tend to concentrate on analysis more than composing and need more examples, frames, models, and heuristics for teaching multimodal composing that are grounded in design and rhetorical theory, and students need more practice using rhetorical frames and design theory to generate multimodal messages. Rhetorical frames can extend the application of design theory in to multimodal composition. A rhetorical heuristic for multimodal writing that keeps objective, context, and audience at the forefront of the message design may better prepare students for the various rhetorical situations they may encounter in and out of the classroom that are both digital and non-digital, enhancing how they negotiate audiences, objectives, and constraints.

This webtext argues that using the language of rhetorical criticism to teach students how to design multimodal messages may widen their purview of the multimodal composing process. The terms "structure" and "form" from Roderick Hart and Suzanne Daughton's (2005) chapter on "Analyzing Form" in Modern Rhetorical Criticism provide a rhetorical framework and metalanguage for contemplating how to construct multimodal forms. I argue that using structure and form as a rhetorical framework to design multimodal messages with semiotic materials may improve how students construct those forms. These terms can challenge the privileging of any one type of mode, form, or medium and reposition a rhetorical design process as a composing process that effectively encompasses both print and digital forms. "Message design, message emphasis, message density, and message pacing" (p. 107) are four elements of structure instructors can use to teach students how to develop composing heuristics for multimodal message design. Instructors can also adapt Hart and Daughton's chart for "Common Structural Techniques in Persuasion" to develop pedagogies for teaching multimodal composing. Students can practice employing a structural type for a rhetorical function, examine the advantages and disadvantages of that structural type, and then generate a critical probe for implementing it.

Using rhetorical theory to enhance multimodal composing processes can potentially lead to increased multi and rhetorical literacies. Structure and form can represent the semiotic activity and rhetorical decision-making processes needed to design effective multimodal messages. Tarez Samra Graban et al. (2013) insist that "we cannot say students are 'creating'" multimodal forms unless "they have figured out and deliberately applied a methodology that guides the why and how of their choices" (p. 254). Rhetorical methods for composing multimodal forms ask rhetors to account for the ways in which language, modes, and mediums spin around in a semiotic and dialectical kaleidoscope that constantly resituates how modes (re)connect and how audiences make meaning from those reconnections. Rhetorical design strategies also emphasize the synergistic connections that exist between semiotic and rhetorical activity that is so common to multimodal writing. Foss et al. (2014) contend that rhetoric is the act of humans constructing reality through symbols, so using rhetorical theories to generate heuristics for multimodal writing capitalizes on the symbiotic relationship between rhetoric and semiotics. Cordova (2013) reiterates how this relationship between rhetoric and semiotics functions when he professes that "a reengagement with rhetoric can help us extend our understanding of the multimodal nature of meaning making and strengthen our development of critical pedagogy and multimodal literacy" (p. 146). The next section explores some of the existing scholarly engagement with multimodality and rhetoric.

Literature Review

Writing and rhetoric scholarship on multimodality has convincingly argued that implementing multimodal assignments into curriculums is paramount for teaching students the multiliteracies necessary to communicate in the digital age (Ball, 2004; Shipka, 2005; Selfe, 2009; Yancey, 2004), but we need more methodologies and heuristics for designing and making multimodal messages. Students are familiar with developing multimodal forms that are both digital and non-digital (NCTE, 2005) but not adept, and many instructors from across disciplines are interjecting multimodal assignments in to their courses, asking students to assemble a wide variety of semiotic materials without giving them clear composing methods. Several multimodal and design theorists have experimented with pedagogical approaches for teaching multimodal composing and made a call to implement more design theory into the pedagogies for multimodal writing (Sirc, 2002; Bezemer & Kress, 2005; NCTE, 2005), but there is very little research on how to use rhetorical theories, criticism, and devices to create heuristics for multimodal composing. Most of the existing research on rhetoric and multimodality tends to focus on improving how students interpret or analyze multimodal forms as opposed to making them.

The New London Group (NLG) affirms that all semiotic activity is broken down into three types of designing: "Available Designs, Designing, [and] Redesigned" (p. 61). They outline and define important metalanguage for multimodal writing, but they do not provide rhetorical heuristics for multimodal writing that draw out design theory or pedagogies for teaching composing strategies. Taking this a step further, Jeff Bezemer and Gunther Kress (2005) argue for the use of design theory as a methodological frame for multimodal composing because "Design is the practice where modes, media, frames, and sites of display, on the one hand, and rhetorical purposes, the designer's interests, and the characteristics of the audience on the other are brought into coherence with each other" (p. 240). They contend that multimodal writing combines rhetorical approaches with design theory and call for more instructors to teach multimodal composing through design theory, but they do not present any specific methods or heuristics for multimodal composing.

George Sirc (2002) also supports design theory for multimodal writing but as a frame for interpreting these forms and not making them. He is more interested in defining what multimodal composition can and cannot be, arguing against the use of heuristics because heuristics treat audiences as constructed instead of lived; however, multimodal design heuristics built on design theory and rhetorical concepts can teach students to see audiences as fluctuating and multiple and that structure and form are dependent on the characteristics of an audiences' lived experiences. Rhetorical frames can account for the ways rhetors negotiate structural decisions for their audience. One major rhetorical adjustment Ellis (2012) made to her pedagogy for teaching multimodal writing was "paying greater attention to multimodal rhetoric" (p. 65). She invited guest speakers to discuss the rhetoric of film and sound production, alloted more time to student drafting and feedback on multimodal work, and provided more access to the media labs and tools students required to complete multimodal projects--although she admits access to these media tools was problematic and out of her control. The only drawback is that Ellis does not provide any rhetorical guidelines students can use to design multimedia.

Brian Ray (2013) uses "uptake" as a pedagogical lens for teaching how to design new media forms (p. 183). He maintains that uptake is a "rhetorical tool" that can enhance student and teacher "awareness of genre and multimodality" (p. 183). He argues that employing terms like uptake as a multimodal metalanguage is highly beneficial for composing. "Adding uptake to the repertoire of multimodal terminology pushes teachers and students to examine what such remixes say about the larger rules governing the relations between genres" (p. 186). Ray presents uptake as a rhetorical frame for interpreting multimodal messages and genres, but he does not carve out space to discuss using uptake to compose multimedia or discuss how rhetors can employ uptake to structre modal content. He concentrates on teaching uptake to increase genre awareness and to improve student interpretation of multimodal environments.

Christopher Basiger (2011) relies on Foucault's author-function and Bawarshi's genre-function as a method for analyzing multimodal forms. He declares that using genre theory for interpreting multimodal composing allows for a more rhetorical understanding of multimedia, but he does not offer a method for composing multimodal messages. Carpenter (2009) points out several structural features of electronic texts that a rhetorical heuristic for multimodal writing could implement like "brevity, compression, and abbreviation; interactivity; graphical elements; a potentially global audience; intertextuality; multigenerical elements; structural linking; and multivoicedness" (p. 144). But he does not explain how rhetors can utilize these structural features when designing multimedia. Jody Shipka (2012) calls for an activity-based framework for multimodal composing to help students move beyond simply analyzing multimodal forms. She provides a list of questions for multimodal writing that are built on a rhetorical term from Hart and Burks' work on "rhetorical sensitivity." Her heuristic guides students through a rhetorical analysis and interpretation of multimedia. Rhetor's could refashion Shipka's questions to facilitate multimodal composing, but these questions are more suited for the interpretation and analysis of multimodal forms than composing them.

The Pedagogical Value of Structure and Form

Hart and Daughton (2005) note that structure is "the apportionment and sequencing of message elements. Structural decisions are decisions about which ideas should be given what amount of attention and how ideas should be arranged for maximum impact" (p. 103). Form focuses on "the patterns of meaning audiences generate when they take in a message. Form refers to the 'shape' of meaning, how ideas are linked together by audiences" (p. 104). Rhetors structure semiotic materials, or message elements, in ways that audiences can form meaning with those materials. Structure and form are the mechanisms that shift content and meaning. Each turn, change in size or movement, or (re)combination of the same semiotic materials generates a completely different structure that alters how audiences form meaning. Structure and Form are meta terms for contemplating the (re)arrangement and layering of meaning in messages. These terms emphasize how rhetors and audiences haggle over the meaning of multimodal forms.

Multimodal rhetors can add these two terms to a growing lexis, a metalanguage, for thinking about how to organize semiotic materials in digital or non-digital mediums for specific objectives and audiences. A comprehensive metalanguage for multimodal composing can improve a student's ability to reflect on his or her composing process. Richard Marback (2009) reminds us that print vocabularies cannot adequately capture the intertextual complexity of multimodal messages. He declares that "Not only do vocabularies of print fail to describe the distinguishing features of multiple, nonlanguage media, they also fail to adequately describe interrelations among different modalities of expression" (p. 265). Traci Fordham and Hillary Oakes (2013) echo this sentiment of "rhetoric as the transmodal frame, the metalangauge, for our approach to multiliteracies" (p. 318). Adopting and applying rhetorical terminology for multimodal writing promotes multiliterate activity. These terms do not prioritize print literacies and conventions or cater to a particular mode or medium.

Thinking about multimodal writing as the structuring of semiotic material can more accurately expose students to what rhetors actually do when they design messages. Students can use these terms to consider how to cater to specific audiences. Hart and Daughton insist that "Since the message structure relates so closely to how people think, it can tell much about a rhetor's mental habits or an audience's operating hierarchy of beliefs" (p. 107). Structural frames deepen the purview of the rhetorical situation and expose how rhetors make design decisions within and for each situation. The structure of a multimodal message represents a set of epistemological habits, ontological preferences, and "mixed logics" (Lauer, 2009, p. 24) that rhetors must account for when making multimodal texts. Learning how to structure negotiate these mixed logics takes practice well beyond analysis. Altering the structure of the content requires audiences to use an alternative logic to form meaning, and when the content is multimodal, the logics can drastically shift from linear to the complete absence of linearity.

Audiences examine the structure of a given set of semiotic materials to form meaning. When the structure changes, so does the form, even when the new structure uses the exact same semiotic materials as the previous structure. For example, a marriage proposal is a form that a proposer can structure in many unique ways. If someone were to pull out a ring box and fall to one knee, the likely form for that structure is a conventional marriage proposal. If the proposer were to pull out a ring box and fall on one knee at a basketball game and chose to make the marriage proposal on a jumbotron during halftime of the game, that structure creates a new form that may not be received as well as the first example. Both attempts at structuring a marriage proposal generate different experiences, forms, and meanings. Both proposals involve dropping to one knee and a ring box, but they were contextually restructured. Adjusting the structure of any given set of semiotic materials transforms how audiences make meaning from those materials.

To further explain this visually, I adopted Hart and Daughton's examples for structuring and recontexualizing content to demonstrate how structure and form work as a frame for thinking about multimodal composition. The images below represent the various modes (sound, image, text) that may exist within a given medium. The border represents the medium, and if we change the medium we alter what modes can be structured and how. If we alter, move, or adjust any modal element in Figure 1, the form and meaning changes too. Figure 1 demonstrates how various modes could be structured within a medium to create a form. The size and positioning of each mode within the medium creates a specific structure and form. Figure 2 takes the same modes from Figure 1 and adjusts their size and positioning to create a new form with the same modes, within the same medium.


Figure 2 is an image of the same modes and medium from Figure 1 except the text has increased and come together with sound and image to create a completely new form. Connecting modes together creates new multimodal forms. Adding, shrinking, trimming, or removing any of these modes generates another form of interpretation. Figure 3 demonstrates how moving the same structure to a new medium also creates a new form.

Using the same structure within a new medium changes that structure even though the structure looks the exact same as it did in Figure 2. Moving this modal structure from a TV to a computer screen alters its structure in more nuanced ways as represented by the color changes of the modes.

Form draws attention to the ways in which audiences are increasingly called upon to generate meaning from semiotic materials. Audiences actively represent and negotiate meaning in multimodal messages. Cheryl Ball (2004) reminds us that "Meaning is made through the reader's choice and arrangement of multimodal fragments" (p. 420). That is why teaching students how to think about structure and form as a rhetorical design process may improve their ability to envision the various forms and arrangements and audience could make as part of the structuring process. Considering how audiences can and may arrange multimodal fragments to form meaning is a characteristic of multimodal composing. Form is also a frame for rhetors to use to consider how audiences generate various meanings from the implicit and explicit structuring of semiotic materials within varied contexts and rhetorical situations, and how audiences form very different meanings from those same semiotic materials when they are (re)structured in new ways for new contexts and audiences.

Structural Elements and the Multimodal Message

"Message design, message emphasis, message density, and message pacing" (p. 107) are four elements of structure I argue students and instructors can use to develop pedagogies and heuristics for multimodal composing. Thinking about multimodal composing as the designing of a message provides students with another way to think about genres. Genre as a term tends to confuse students. If students are assigned to produce a specific genre without a clear rhetorical purpose, they often seek to replicate that genre and lose track of the audience and objective of the message, or they fail to see all of their rhetorical options because of a hyper-focus on replicating the form. Students should select genres based on their audience and objective so that their rhetorical purpose is at the forefront of their composing process.

Sheridan et. al (2005) call on students to consider "What modes and media are best suited to the kinds of change [they] are trying to effect and to [their] intended audience and purpose" (p. 818) when doing social justice work. They call for rhetors to make design decisions based on their rhetorical intentions and then utilize the genres, modes, and mediums that best allow them to achieve those goals. However, this could be difficult to adhere to for many of our students. Rhetors must recognize the cultural and social limitations they face when deciding how to access and compose with multiple modes for multiple audiences. Students working from and within strained socio-economic backgrounds or under-privileged communities will need to consider the unique demographics and constraints that shape their rhetorical moves. Some of our students may not have access to certain mediums or modes, or they may not have access to the mutiliteracies required to use them effectively. They may also face backlash or violence if they publicly call out certain groups of people or individuals with modes and mediums on platforms that can reach deep into communities, placing rhetors in more danger. In the next section, I explain how to use message design, emphasis, density, and pacing as frames for designing multimodal structural types.

Design

Rhetorical theory can bond design theory and multimodal composing together more tightly. Rhetorical theory extends our purview of design theory because rhetorical frames expose how rhetors design messages and make decisions for a specific purpose and audience. Marback (2009) notes that "Designers" find solutions to problems, using questions and concerns that arise from rhetorical situations (p. 261). He contends that "Design is rhetoric because rhetoric is a study of our most wicked of all problems: making responsible use of the persuasive power inherent in all artifacts" (Marback, p. 262). Design theory emphasizes solving problems with messages through composing. George (2002) asserts that "thinking of composition as design shifts attention, if only momentarily, from the product to the act of production" (p. 17). When designing a message, rhetors must consider all available and possible semiotic actions to solve a problem. Pedagogy for multimodal composition must not limit students' understanding of what multimodal messages can be and how far they can push the boundaries of message design. Giving students a broader purview of the decision-making processes involved in multimodal composing may enhance how they create multimedia. Developing theoretical multimodal composing methods that coherently synthesize rhetorical awareness and design theory can also provide students with a comprehensive understanding for how audiences and rhetors rhetorically negotiate purpose in multimodal writing. The New London Group (1996) reaffirms this need to connect design theory with a rhetorical purpose when they contend that professional and academic communities are moving towards a design theory for multimedia. Using structure and form to push the boundaries of design theory may give students a more robust rhetorical design process.

Perhaps one of the most important values for using structure and form to think about designing multimodal messages is that "Structure argues" (Hart and Daughton, p. 113). Learning how multimodal forms make arguments through structure is difficult for students. They are conditioned to form meaning of all the forms they encounter with print literacy conventions. Students have internalized and normalized the structure and form of print literacy as a stabilizing feature for content and knowledge-making. They do not initially see how structure in multimodal writing instantiates an argument and that the structuring of semiotic materials is the argument. Structure argues. Learning how to see structure as the argument of a multimodal form is vital for developing rhetorically-minded multimodal writers. Ball (2004) expounds upon this idea when she conveys how a multimodal text "explicitly performs its meaning through the audience's understanding of its multimodal elements and interface design" (p. 410). The structure of the modal elements initiates a performance of meaning that enacts the argument. Designing structure as argument is an important convention of multimodal literacy.

Instead of revising a paragraph of words and sentences, a multimodal rhetor might adjust the volume of a sound clip, and then add an additional sound clip to increase the efficiency of the message design. These adjustments and additions are in themselves arguments of structure and conventions for multimodal composing. Moving modes and (re)connecting them to other modes and mediums alters how audiences shape meaning from the message; however, the movement and (re)connection of semiotic material is itself an argument and rhetorical representation. Remely (2017) argues that "the message is not just the content but its form relative to how it is presented and the communicators' relationship to each other and their experiences" (p. 21). Structure and content interact with each other in ways that are not easy to separate. The message's structural design encompasses how content is sequences and delivered to audiences and how audiences generate meaning.

Emphasis

To consider how the structure of a multimodal message emphasizes certain ideas over others, it is important to identify the modal characteristics and affordances that serve as catalysts for that emphasis. Rhetors use semiotic materials to emphasize particular ideas and to solve specific problems for specific communities. Students can examine how rhetors structure modes to create emphasis and how sound, image, and/or text combinations create levels and layers of emphasis. Then they can practice formulating levels and layers of emphasis through the structuring of content in their work. This can improve their understanding of how sounds, images, and texts emphasize, and how structural combinations instantiate new levels and layers of emphasis. Rhetors weave modes and mediums together to form structural emphasis in multimodal messages, designing emphasis directly in to each message.

Increasing or decreasing the modal representation in a form can generate emphasis. If rhetors adjust, for example, the sound in a multimodal message, the form also changes. Rhetors can use additional structural elements to further consider how to structure sound. Cynthia Selfe (2009) maintains that "Speech conveys a great deal of meaning through pace, volume, rhythm, emphasis, and tone of voice" (p. 633). These structural conventions for sound and aurality can serve as units of analysis for identifying how a multimodal message emphasizes specific ideas with the positioning, movement, and aurality of sound. Students can practice structuring volume, pace, tone, and rhythm to generate emphasis in multimodal forms--to examine how volume, rhythm, and tone shape the emphasis of the message. For example, to make a point about how sound can emphasize a tone or mood for a film, I have students watch the opening scene of The Shining without sound and then with sound.

An example of the opening scene is provided here:

The opening scene without sound feels and reads like a family vacation or a car ride through the mountains. Once the scene is played with the music, viewers and listeners quickly come to a much more grim conclusion about form of the movie. Sound creates emphasis, and the type of music chosen for this opening scene has a particular pace, tone, and rhythm that formulates emphasis.

The spacing, inclusion, and exclusion of semiotic materials also establish emphasis through structure. Altering the types of sounds, images, texts, and mediums, along with the size of these modes, reshapes the emphasis of the content in the message, and students should spend time practicing these alterations. This will help them determine how the inclusion or exclusion of semiotic materials impacts their ability to emphasize. Rhetors alter the density of a message in much the same way as they emphasize.

Density

Multimodal messages accrue thickness from the layering of semiotic materials. This layering process creates a certain type of bond, a compactness, between modes, but the strength of the bond is dependent on the layering and, therefore, always different. Modal layering creates specific and unique bonds between modes that determine the strength of a message. These bonds are never the same because rhetors never layer modes the same way. Even if they try to layer a sound over an image in the exact same way they have done before, there will still differences in the size, shape, and accuracy of those layers. Modal layers never reline or connect exactly the same way. The strength of a certain message can severely weaken or strengthen over time as the bonds between certain modal layers erode or solidify. As messages increase or decrease in density, their structure and form change.

A message's density can expand or wither depending on how certain modes of content grow or lessen in popularity, regularity, and value. For example, using a popular image for a meme or a GIF may increase the density of a message because the mainstream image produces a set of recognizable representations for audiences, but as that image loses traction and popularity it becomes less known and therefore less dense. One example of message density is the addition of soundtracks in to movies. The Guardians of the Galaxy franchise increased its message density when it included a soundtrack of very specific classic rock tunes from the 1970's that older audiences would recognize and (re)bond with while watching the movie with their friends and/or children. Parents that may not be interested in watching an animated film made for younger audiences may sit through the film because of the music. The soundtrack for the film caters to older audiences, and the expansion of audiences for the film from the addition of music increases the movie's density. The more dense a multimodal text is the more places audiences can connect to it and personalize its meaning. Students can practice structuring various modal forms to increase or lessen the density of a message. Instructors can create assignments that task students with structuring music or sound into a story or text to increase and/or decrease the density of the story. Rhetors can lessen density as a rhetorical move, too.

Consider how a song is played over a series of images in a short video that gains favor with an audience. The audience strengthens that bond between the song and the video with likes, shares, reposts and comments, but over time the images in the video become controversial and fall out of favor, loosening the bond and density between the song and image that once existed to make that video popular. Perhaps the images were of a famous actor now accused of inappropriate behavior. Seeing those images in a video now decreases the density of the message because the cultural forces altered the power of the image. Rhetors of multimodal messages establish message density from the strategic layering of modes, but audiences ultimately solidify the bond between modal content and increase the density of the message with their interpretations and sharing habits. Using too many modal combinations to deliver a message can obfuscate density and decrease cognitive retention, distracting audiences from the intended message.

Lisa Bickmore and Ron Christiansen (2010) note how multimedia can be flashy and "shift our attention away from rhetorical knowledge" (p. 153) that is embedded in the design of the message. The density of the message can move our attention away or toward the objective of the message, so it is important to consider where and when those shifts occur and how the density of a message impacts those shifts. Hart and Daughton echo this sentiment when they assert that "The centrality of structure to content is best seen when structure is missing" (p. 106). Shifts in message density can expose where structure is devoid and lacking in a message and how that lack of structure modifies meaning. When messages become too dense or have no density, rhetors must restructure the content to create a balance between structure and form. Without structure the content has no boundary for audiences to form meaning. Students can practice using modes to increase and decrease density in a multimodal form. Rhetors structure pacing in a similar way to both density and emphasis.

Pacing

When considering how multimodal messages are paced, it is important to consider which modes impact pacing the most and why. Rhetors use modes to slow or speed up a message, and speeding up and slowing down the message changes the form of the message. Speeding up a multimodal message with sound is different than slowing it down with an image. Instructors can create assignments that ask students to practice speeding up and slowing down messages with different modes to achieve a variety of rhetorical effects. Certain images will slow or increase the pace of the message more than others. For example, pharmaceutical commercials often change the speed of the narrator's voice from a normal speed to a very fast speed when describing the side effects of the advertised drug. Students can learn how to see pacing as a structural convention for multimodal composing. Altering and applying sound, image, or text to control the pace of a multimodal message in a specific rhetorical situation is difficult, and rhetors need to practice pacing when designing multimodal forms for rhetorical situations to improve how they negotiate multimodal composing decisions in relation to various constraints, audiences, and exigences.

Rhetorical frames like structure and form provide rhetors a way to consider how to pace the movement of ideas in a message with various modes, and how pacing generates a structure that is itself an argument. Pacing structures content in ways that impact meaning. The pace of a multimodal message is impacted by each mode but also by and through the combination of modes. Certain ideas are presented in an order to control when audiences will know something. Another simple example of this is, again, pharmaceutical medication ads. All of these medication ads begin and end the same way. Audiences are first presented with a list of the benefits of the medication and then provided with examples, scenarios, and situations that further demonstrate how the medication will improve their lives. If the medication is for skin rashes, then audiences are presented with ads that have characters feeling comfortable about their skin in public and private situations as a result of the said medication. At the end of the ad, audiences are very quickly provided with the side effects of the medication in small font at the bottom of the page. Consider how audiences would react to an alternative pacing of the same content. What if medication ads began with the side effects in a slow, deliberate voice? Students can practice altering the pacing of their messages with a variety of modes to locate the right rhetorical pace for each message they make. In the next section I use Hart and Daughton's chart for "Common Structural Techniques in Persuasion" to examine how to use structural elements to develop rhetorical heuristics for multimodal composing.

Developing a Rhetorical Heuristic for Multimodal Composing

Hart and Daughton's chart for "Common Structural Techniques in Persuasion" calls for identifying a specific structural type, its rhetorical function, an example of the structural type, its main advantages and disadvantages, and a critical probe (p. 108). I have provided a screenshot of these techniques from Hart and Daughton"s text below:

Teachers developing pedagogies for multimodal composing can adopt, apply, and recontextualize this chart for designing and analyzing multimodal structural types. In this section I explain how to use these structural types as a method to generate critical probes and questions about multimodal composing that can develop a rhetor's design strategies and rhetorical decision-making for multimodal messages. Below is an example of an outline of the chart I adopted from Hart and Daughton for investigating structural types in multimodal messages.

The example below examines layering as a "structural type" for multimodal composition and then briefly identifies the rhetorical functionality of that structural type. After deciding on a structural type to investigate, students can practice implementing that type in their work so they can examine its advantages and disadvantages. Finally they can draw some conclusions, or "critical probes," about that structural type in relation to its effect on audiences and its ability to meet rhetorcial objectives. Students can then continue to practice layering modes in to a message according to a rhetorical functionality that is dependent on the advantages and disadvantages of layering.

Multimodal Composing Heuristic

PDF Version: Download

  1. Structural Type: Layering.
  2. Rhetorical Function: Provides depth, texture, and ethos.
  3. Example: Provide an example of layering or have students provide one. Teachers can choose examples to teach structural type identification and interpretation in class, but students can locate examples on their own for an assignment. Have students use their work for an example.
  4. Main Advantage: Allows rhetors to overlay modes for rhetorical impact and density. This can expose important and invisible tensions in the message for readers, viewers, and listeners.
  5. Main Disadvantage: Multiple ways to read modal layering. Multiple modal layers can create confusion and obfuscation or hinder the clarity of the message.
  6. Critical Probe: Why did the rhetor use layers in this example? What did it help him or her achieve? Students can use the critical probe to reflect on the effects of layering in the message. These critical probes are opportunities for students to be metacognitive about their multimodal composing process. These probes can become knowledge sets for multimodal composing students can apply to multiple writing situations.

Students can use this as a design heuristic for multimodal writing, and to consider the advantages and disadvantages of layering for designing multimodal messages. The critical probe gives students space to reflect on their use of layering to ensure its rhetorical value. They can practice implementing other structural elements like density, pacing, and emphasis in to multimodal messages with this chart outline, and instructors can encourage students to locate multimodal structural types on their own and apply them to this chart to increase their understanding of structure and form as a model for composing with multiple modes.

For example, Selfe's structural types for sound (volume, rhythm, and tone of voice) referenced earlier represent common structural techniques for incorporating sound in to multimodal forms. In addition, terms like transduction, "the moving of semiotic material from one mode to another" (Bezemer and Kress, 2005, p. 240) are also structural techniques for persuasion common in multimodal forms students can plug into their heuristic charts to improve how they design multimodal messages. Transduction is a means of structuring semiotic materials through movement, absence, and presence and a meta term for thinking about multimodal composing.

Students can also use the four specific rhetorical activities from Bezemer and Kress to represent structural techniques for persuasion in multimodal composing: "selection, arrangement, foregrounding, and social repositioning" (Bezemer and Kress, p. 248). These structural meta terms further expand the critical discussion of multimodal structural types. Below is an additional example for using Hart and Daughton's structural chart to examine foregrounding in multimodal composing.

Example:

  1. Structural Type: Foregrounding
  2. Rhetorical Function: Provides depth, texture, audience awareness, alters content interpretation.
  3. Example: Provide an example or have a student provide one. Have students bring in their work and practice foregrounding in class.
  4. Main Advantage: Can improve articulation and distinctions for readers/viewers. Can bring ideas in and out of focus or to the forefront. Allows for rhetors to make several connections between ideas.
  5. Main Disadvantage: Alters contexts. Could create confusion about what ideas are most important. All ideas may not connect clearly.
  6. Critical Probe: Why did the rhetor use foregrounding? What did it help him or her achieve in the message?

PDF Version: Download

Conclusion

Structure and form cannot account for every design move rhetors need to consider when composing multimodal messages; however, using these terms to build a metalanguage for linking design theory and rhetorical theory together can improve how rhetors create pedagogies for multimodal composing. Structure and form can help students generate rhetorical heuristics for designing multimodal forms and provide writing instructors an opportunity to teach students multimodal writing strategies grounded in rhetorical theories and design principles. This can potentially enhance how rhetors design messages with semiotic materials (Bezemer and Kress, 2008; George, 2002). These terms further demonstrate the symbiotic relationship between rhetors and audiences and how those relationships are negotiated through and with different modes. Formulating a rhetorical composing heuristic that can account for both print and electronic message design gives rhetors a more stable method for working with semiotic material.

References

Ball, Cheryl E. (2004). Show, not tell: The value of new media scholarship. Computers and Composition, 21 (3), 403-425.

Basgier, Christoper. (2011). The author-function, the genre function, and the rhetoric of scholarly webtexts. Computers and Composition, 28, 2, 145-159.

Bezemer, Jeff and Gunther Kress. (2008). Writing in multimodal texts: a social semiotic account of designs for learning. Written Communication, 25, 166-95.

Bickmore, Lisa and Ron Christiansen. (2010). 'Who will be the inventors? Why not us?' Multimodal compositions in the two-year college classroom. TEYYC , 230-42.

Carpenter, Rick. (2009). Boundary negotiations: electronic environments as interface. Computers and Composition 26. 134-148. Notes:

Cordova, Nathaniel. (2013). Invention, ethos, and new media in the rhetoric classroom. In Tracey Bowen and Carl Whithaus (Eds.), Multimodal Literacies and Emerging Genres 143-63. Pittsburg, PA: University of Pittsburg Press.

Edwards, Dustin. (2016). Framing remix rhetorically: toward a typology of transformative work. Computers and Composition,39, 41-54.

Ellis, E. (2012). Back to the future? The pedagogical promise of the (multimedia) essay. In Whithaus, C., & Rowen, T (eds), Multimodal Litercies and Emerging Gneres in Student Compositions. Pittsburgh, PA. University of Pittsburgh Press.

Fordham, Traci and Hillory Oakes. (2013). Rhetoric across modes, rhetoric across campus: faculty and students building a multimodal curriculum. Multimodal Literacies and Emerging Genres in Bowen, Tracey and Carl Whithaus Eds. Pittsburg, PA: University of Pittsburg Press, pp. 248-281.

Foss, Sonja K. and Karen Foss and Robert Trapp. (2014). Contemporary Perspectives on Rhetoric Waveland Press.

George, Diana. (2002). From analysis to design: visual communication in the teaching of writing. College Composition and Communication, 54, 11-39.

Hart, Roderick and Suzanne Daughton. (2005). Modern Rhetorical Criticism. Third Edition. New York, NY: Routledge.

Kress, Gunther. (2005). Gains and losses: new forms of texts, knowledge, and learning. Computers and Composition, 22, 5-22.

Lauer, Claire. (2009). Contending with terms: 'multimodal' and 'multimedia' in academic and public spheres. Computers and Composition, 26, 225-39.

Marback, Richard. (2009). Embracing wicked problems: the turn to design in composition studies. College Composition and Communication, 61.2, W397-W419.

New London Group. (1996). A pedagogy of multiliteracies: designing social futures. Harvard Review, 66, 60-92.

Samra Graban, Tarez et al. (2013). Multivalent composition and the reinvention of expertise. Multimodal Literacies and Emerging Genres in Bowen, Tracey and Carl Whithaus Eds. Pittsburg, PA: University of Pittsburg Press, pp. 313-335.

Selfe, Cynthia. (2009). The movement of air, the breath of meaning: aurality and multimodal composing. College Composition and Communication, 60, 616-63.

Sirc, Geoffrey. (2002). English Composition as a Happening. Logan: Utah State University Press.

Shipka, Jodi. (2011) Toward a Composition Made Whole. University of Pittsburgh Press.

Shipka, Jodi. (2005). A multimodal task based framework for writing instruction. College Composition and Communication 57, 2, 277-306.

Sheridan, David M., et al. (2005). The available means of persuasion: mapping a theory and pedagogy of multimodal public rhetoric. JAC, 25(4), 803-844.

Remely, Dirk. (2017). The Nueroscience of Multimodal Persuasive Messages: Persuading and the Brain. New York, NY: Routledge.

Rice, Jeff. (2007). The Rhetoric of Cool. Carbondale: Southern Illinois UP.

Ray, B. (2013). More than Just Remixing: Uptake and New Media Composition. Computers And Composition, 30 183-196. doi:10.1016/j.compcom.2013.07.003.

Yancey, Kathleen Blake. (2014). Made not only in words: composition in a new key. In Multimodal Composition: A Critical Sourcebook. Claire Lutkewitte Ed. Bedfords/St. Martins: Boston, 62-88.