Coronavirus Updates: USC  |  SCA
Regeneration: Multimedia Genres and Emerging Scholarship

June 29th, by Steve Anderson
» Download PDF

This white paper presents an initial taxonomy of generic strategies and conventions that have emerged from the past eight years of practice-based research in multimedia pedagogy at USC's Institute for Multimedia Literacy. These genres emerged organically across a wide variety of courses and disciplines at IML and have subsequently been incorporated into the curricula of the Honors in Multimedia Scholarship and Multimedia in the Core programs. The taxonomy begins with a more detailed description of the five genres that have been deployed most frequently in these programs, followed by brief outlines of additional genres.

The concept of genres has long been used to categorize and describe literary, cinematic and other creative endeavors. When thinking about the hybrid (creative, scholarly, technological) activities that define the emerging practices of multimedia scholarship, generic conventions offer a useful way of understanding both the processes of creation and the significance of this work within higher education. Both students and instructors may benefit from thinking in terms of genres in order to create self-awareness about the strengths and weaknesses of particular media and technological affordances. More importantly, genres offer the beginnings of a critical vocabulary which may help facilitate the assessment and legitimation of media- and technology-enhanced forms of scholarship. This investigation is thus both an observational taxonomy, drawing on nearly a decade of classroom experience with multimedia and education at the University of Southern California's Institute for Multimedia Literacy, and also a discussion of strategies for using genres in the design of multimedia curricula, classes and projects.

My goal is to address a broad range of multimedia genres as they function within specific contexts of scholarly research, publication and teaching. Although many of the genres discussed below exhibit familiar resonances with the conventions of other cultural forms (including those of commercial entertainment industries), I regard these genres as having been customized and transformed - or “regenerated” - to reflect the priorities of scholarly multimedia. In practice, few scholarly multimedia projects are constrained to a single generic mode. Hence, the effective selection and combination of generic strategies is an important aspect of scholarly multimedia authoring. Although this taxonomy remains far from exhaustive and includes numerous overlaps, idiosyncrasies and arbitrary divisions, it offers a point of departure for thinking about the usefulness of genres for designing multimedia-based scholarship.

Before proceeding, I will briefly distinguish my use of the term “genre” from its literary and cinematic counterparts. Part of what makes genres an interesting lens by which to consider the static and dynamic characteristics of cinematic genres is the ability to track changes over time and in response to cultural circumstances. Film theory tells us that shifts in genres occur in response to socio-historical events and cultural needs. Genres - and variations on the conventions of a given genre - gain currency because they offer temporary solutions to social problems that cannot be resolved in the real world, such as contradictions based on race, economic and gender inequalities. Regrettably, due to the relatively short time span under consideration here, we are unable to observe longitudinal shifts in multimedia genres. Nonetheless, this taxonomy represents a first step toward better understanding the value and limitations of multimedia in higher education and my hope is that it will help students become more articulate advocates of their work by being able to situate it within a recognized field of academic practice. Of equal importance with this strategic motivation is the goal of providing students with a critical vocabulary to reflect on their own work.

A crucial aspect of the multimedia pedagogy that has been in practice at the Institute for Multimedia Literacy for the past nine years is a cycle of:

Ideation - Production - Reflection

This circuit invites students to reflect on both the design strategies and effectiveness of their own work at regular intervals. Providing students with a critical vocabulary has proven to be a particularly effective tool for improving students' critical thinking and self-assessment. In addition to observing the emergence of the generic categories outlined below, I will take this opportunity to offer some additional insights on how the project-based curriculum supported at the Institute for Multimedia Literacy is implemented. This is relevant because the IML's teaching and evaluation methodology has emerged in parallel with - and is co-constitutive of - the development of these genres.

A core element of IML methodology is the structuring of multiple collaborative relationships as a cornerstone of project conception, design, development and critique. This collaboration takes place on several tiers, beginning with faculty-to-faculty collaboration. Beginning in 2004, the IML has been home not just to program administrators and postdoctoral researchers, but to University faculty on regular appointments. This development was crucial to the evolution of IML to its current status as a fully-functioning partner in two university-wide programs: Honors in Multimedia Scholarship and Multimedia in the Core. Under the direction of Professor Anne Balsamo, the IML has transformed its relationship with University faculty from the status of service-provider to that of equal partners who bring genuine expertise and confidence to the pursuit of multimedia scholarship. This shift has facilitated the development of genuinely collaborative relationships with faculty and enhanced the IML's reputation across campus.

A second vital tier of collaboration takes place at the level of instructional staff and Teaching Assistants, who continue to be responsible for the majority of in-lab instruction and support for student work. The professionalization of the lab and instructional staff has led to more consistent educational opportunities and positive learning outcomes for students. The final tier of collaboration takes place among the undergraduate students who enroll in IML classes and also serve as “Peer Mentors,” providing the first line of support for students during lab work. A basic ethos of IML production is to take advantage of the “collective intelligence” of its students, TAs and instructional staff. Whereas no individual working in today's media landscape can possess full expertise with all the available tools of multimedia, a surprising breadth of expertise can often be harvested from the experiences of any given group of individuals. This ethos of participation is effectively reinforced by the widespread use of social software as an integral part of classroom activities. Students who are accustomed to sharing their work, thoughts, ideas and resources via participation in wikis, blogs and collective tagging and bookmarking communities are predisposed to approach their work in terms of sharing knowledge and collaborative production.

One particularly effective strategy we have arrived at for the framing and execution of multimedia projects is based on what we call the “Five Obstructions” model, based on the 2004 documentary film by Lars von Trier and Jorgen Leth. In The Five Obstructions, Lars von Trier compels his former mentor Jorgen Leth to create a series of short films that comply with a series of increasingly arbitrary and sadistic constraints. As the film progresses, Leth grows increasingly recalcitrant in the face of von Trier's attempts to control his production process and develops a series of strategies for creatively overcoming the limitations placed on him. This model of creative responses to imposed limitations serves as a model for students undertaking multimedia work in IML classes, and each project assignment is accompanied by a set of “obstructions” carefully designed to spark creative solutions. As Leth's creative resistance demonstrates, these obstructions need not be followed precisely, provided students are able to articulate a convincing argument in defense of any transgressions they may commit. We have found this format to be particularly effective within the context of generic production, due to the fluid and permeable nature of genres and the benefit of combining generic conventions. Students are encouraged to selectively - but knowingly - depart from docile adherence to the parameters of a selected multimedia genre. Hence, both the “Five Obstructions” and genre-conscious approach to multimedia scholarship may be understood in terms of controlled and deliberate responses to various kinds of limitations (technical, creative, temporal, etc.) that are endemic to all media production.

As we move into a discussion of the genres of multimedia scholarship, I will note that the first five genres described below have demonstrated variable but consistent utility among the work to emerge from IML-affiliated classes. Within the Honors in Multimedia Scholarship program, I regard these as the “top five” genres with which students are encouraged to engage. However, variations in circumstance, discipline, topics of investigation and project concepts may drive students to work equally productively within any of the subsequent genres.

Documentary

The documentary genre involves work that makes a “truth claim” about some aspect of the world, using various forms of evidence, testimony and artifacts. Successful projects convey ideas and provoke engagement with an area of inquiry using a relatively straightforward mode of address. The Documentary Genre is fundamentally engaged in questions of epistemology. It invites us to ask how we know what we know about the world. Any form of documentary media invites us to question how it was created: What rules does it operate by? What promises does it make? How does it convince us that it is telling the truth? The documentary tradition is based on rational systems of thought and is rooted in familiar academic practices such as deductive reasoning, the scientific method, rational argumentation, etc. It also presupposes that meaningful information about the world may be obtained through the senses, especially vision.

The relationship between the creator and the viewer of a documentary is defined by an implicit agreement sometimes called the “documentary contract.” The contract is based on a mutual agreement that none of the materials (ideas, images, sounds, events, people, etc.) presented have been made up or are intentionally misleading. This “truth claim” is designed to help viewers negotiate the shift from representation to construction - we recognize that the act of documenting something can never be naïve or free from ideology, and viewers agree to provisionally accept the truth claim of the creator until they have reason for doubt. In his book Representing Reality (1991), documentary film theorist Bill Nichols created an influential taxonomy for thinking about the different “modes” by which documentary functions. Although these categories are not absolute or all-inclusive, they serve as a useful beginning to think about different approaches to documentary.

Expository Documentary

According to Nichols, the Expository mode is characterized by the use of an omniscient “Voice of God” narration. The expository mode purports to present factual information in a straightforward way, with support from images, sounds, music, performances, etc. The Expository mode appeals to our desire for a complete, satisfying aesthetic and epistephilic experience. A good example of the Expository mode may be found in the many historical documentary films made by Ken Burns, such as Brooklyn Bridge (1981).

Observational Documentary

The Observational mode is characterized by minimal (or no) narration, long takes, real-time, synch-sound, spatio-temporal continuity, present tense, indirect address, etc. In cinema, the observational mode is exemplified by the American “Direct cinema” movement and the French “Cinema verite” movement. In both cases, the presence of filmmakers is minimized as they ostensibly simply watch while actions unfold before the camera. Two rather different example of observational documentary may be found in Frederick Wiseman's Titicut Follies and the Maysles Brothers' Salesman, both made in 1969.

Interactive Documentary

The Interactive mode differs from the previous two modes in that the filmmaker is present on screen and often serves as the center of the action. In Interactive documentaries, the action is thought to be catalyzed by the presence of the filmmakers. Often the filmmaker will speak to the audience using direct or indirect address. The interactive mode represents a particularly useful set of conventions for multimedia work, because of the opportunity to engage in literal interactivity, with the user making choices and participating in the experience of a given project. Some typical examples of Interactive documentary cinema may be found in the work of Michael Moore, including The Big One (1998).

Reflexive Documentary

The Reflexive mode is both the most difficult to define and the least common of the four documentary modes outlined by Nichols. In essence, Reflexive documentaries undertake a structural critique of the documentary form itself, emphasizing the constructed nature of both film and reality. The goal of a Reflexive documentary is to get audiences to think about the ways that documentaries construct their vision of reality - it exposes the various “rhetorics of authenticity” by which the documentary mode asserts its truth claim. Two examples of the Reflexive mode may be found in the interactive installation project Terminal Time (2000) and the parodic documentary created during an interview with Ken Burns on the television talk show, The Colbert Report (2005).

Although most of the examples above are drawn from film and television, analogous examples of genre-driven documentary projects may be found in the realm of interactive or online multimedia work as well. Nichols' taxonomy and the relatively consistent examples offered above are primarily useful as a way of creating a more nuanced understanding of an otherwise undifferentiated field of “documentary” practice. Given the wide range of media included in this genre, it is important to note that, contrary to the expectations students often have, documentary should not be considered a form that is in any way objective or innocent in its construction. Indeed, it is a rare documentary that cannot be understood in terms of making an argument or supporting a thesis. We may thus consider Documentary to be a subset of - or integrally related to - what may be termed the “Argumentative” genre.

Argumentative

The Argumentative genre is characterized by work that makes an argument or attempts to persuade a user to adopt a particular viewpoint. Successful arguments creatively encourage users to think about something differently than they had previously. In other words, they do not simply recapitulate accepted ideas, but build toward a conclusion or course of action. It is important in crafting a multimedia argument to avoid heavy-handedness, moralizing or overly literal argumentative strategies. Given its close relationship to the conventions of the expository essay, we believe the Argumentative genre is of particular importance for achieving our goal of gaining acceptance for multimedia production within a broader academic community. The Argumentative genre proceeds from the assumption that something is at stake in the work being undertaken. This resonates with another overarching goal of the work of the IML to promote work that is of relevance both within and beyond the academy and to reinvigorate students' investment in the real-world significance of their work.

Our deployment of the category of argumentative multimedia draws on some of the most basic principles of literary rhetoric. As with the documentary genre, argumentative multimedia is a very small part of a large and venerable field of study that far exceeds the modest aspirations of this taxonomy. We have obtained positive results from introducing students to a small number of concepts and terms drawn from the field of rhetoric. Specifically, we introduce students to the following forms of rhetoric:

  1. Logos: the appeal to reason
  2. Pathos: the appeal to emotion
  3. Ethos: the appeal to character or ethics

It is useful to note that argumentative work may effectively make use of any or all of the forms of appeal described above and indeed the most powerful arguments may well be achieved by proficiently combining all three. Beyond this, we encourage students to think in terms of the following factors, also drawn from rhetoric, but of clear relevance to multimedia:

  1. Kairos: the occasion for speech
  2. Audience: the intended recipients of speech
  3. Decorum: the appropriate form of speech

For the purposes of demonstrating these rhetorical characteristics within media (as opposed to written) form, a particularly useful text may be found in the film by Davis Guggenheim, which documents and embellishes upon former Vice President Al Gore's campaign to raise public awareness about global warming, An Inconvenient Truth. This film, which won the Academy Award for Best Documentary in 2006, is exemplary in its use of virtually all of the rhetorical strategies outlined above. A close structural analysis of this film has proven to be extremely effective as a means of revealing the functioning of argumentative rhetoric.

Placed in juxtaposition with the Documentary genre, Argumentative media such as An Inconvenient Truth offers a stark formal contrast due to its use of information graphics, visualizations, charts, statistics, etc. Because it is goal-oriented - explicitly seeking to provoke active responses from viewers - the Argumentative genre may be more productively analyzed in terms of the factors above, rather than the formal properties that primarily motivated Nichols' documentary modes. Because of their shared characteristics with the expository essay, argumentative multimedia lends itself to familiar forms of evaluation. Questions to consider for assessment of students work might include: Was the thesis clearly articulated? Was evidence used effectively in support of the thesis? Was the intended audience appropriately addressed? A typical argumentative multimedia assignment might also include obstructions that require students to operate within precise form(s) of appeal and circumstances of speech.

Essayistic

Projects in the Essayistic genre are defined by their inclusion of a personal dimension (often autobiographical), which drives and focuses the project. Successful essayistic projects make viewer-users interested in the subjective viewpoint of the maker that often involves cultural, racial or gender identity. Although it is perhaps the most difficult to reconcile with conventional academic practices, I believe the essayistic genre is important to include among the privileged genres of multimedia scholarship we promote. A core commitment of the IML is to make use of the affordances of multimedia expression as a vehicle for work that may not be included in traditional academic canons. Effective work in the essayistic genre will avoid allowing personal or confessional dimensions to overwhelm the other goals of the project and the individual experiences and perspectives it presents will often have broader consequences and resonances beyond the immediate topic at hand. Like the Documentary and Argumentative genres outlined above, the essayistic tradition is rooted in previous expressive modes including the literary essay and essay film and may be productively examined in relation to the history of this work.

Within the traditions of literary or epistolary essay writing, which includes autobiography, counter-histories, oral histories, and work that interrogates conventions of truth, science and history, the essayistic genre is particularly effective. This genre also creates legitimate spaces for otherwise unheard voices to speak. Academia's traditional privileging of objectivity and rationality may be significantly challenged by work that takes account of and even values individual subjectivity and responses to dominant culture that negotiate their own relation to systems of power, knowledge and authority. The Essayistic genre must be deployed judiciously, due to the fact that it is easily dismissed by empirical traditions and may not serve the goals of rational argumentation in some disciplines. However, when managed effectively and used in conjunction with the affordances of multimedia, the Essayistic genre is arguably among the most powerful and potentially transformative of the genres profiled here.

Narrative

By convention, the Narrative genre may deploy any of a wide range of familiar elements of storytelling. These include the use of characters who have a particular point of view, a conflict-resolution structure, temporal progression, emotional arcs, etc. It is important to note that, from the perspective of the IML, multimedia scholarship is not fundamentally about storytelling. Due to the Institute's close relationship to the School of Cinematic Arts, the gravitational pull of the cinematic vernacular can sometimes seem overwhelming. Thus, we have consistently stipulated that narrative may productively serve as an element of a scholarly multimedia project but should not serve as an end in itself.

One particularly productive variation on the conventions of cinematic storytelling is the movement toward what may be termed “database narrative.” This narrative structure is productively enabled by the capabilities of interactive and computational media, in which emphasis is placed on the dynamic selection and combination of elements from a field of possibilities. Database narratives often deploy a non-linear temporal structure with the possibility of what Gerard Genette has termed “multiple narrative focalizations,” which deemphasize character and the use of a fixed perspective as the primary structuring principles of storytelling. Projects that effective deploy elements of narrative often do so through the use of what I would call “micro-narratives,” that is, narrative moments or transient, story-related elements that materialize as needed. The proliferation of database narratives within academic work has prompted stimulating debates over the precise nature of the relationship between databases and narrative, with one side of the debate holding that database structures are the enemy of storytelling; while the other argues that all stories - like language itself - derive from combinations of narrative elements and thus databases and narrative are mutually dependent and co-constitutive. For the purposes of scholarly multimedia, the database model of narrative construction has proven to be an effective tool for retaining the power of story and character without simply capitulating to the conventions of cinematic storytelling.

Game

Games are traditionally defined as having rules, scoring and outcomes; they invite players to compete or succeed. Game structures can range from simple to complex and some projects may deploy a game-like element within another genre. Successful games sustain interest by tapping into players' desires to have fun or to win, but as with the narrative genre, game projects must consciously avoid simple adoption of preexisting game vernaculars. Also, due to the technical complexity of many game design platforms, students must resist becoming overwhelmed by technical considerations and remain focused on the scholarly dimensions of their work. From our point of view, games represent one of the most exciting and promising environments for developing engaging and widely accessible educational experiences. As with the narrative genre, game elements are frequently embedded within a broader structural context, which includes characteristics of other genres as well.

The movement toward games that exhibit an explicitly activist or educational agenda - as in the “serious games” or “games for change” models - seems to support the value of game-based scholarly multimedia. The range of senses and degree of interactivity that is part of the game vernacular offer particularly compelling possibilities for multimedia expression. We are especially interested in recent research that seeks a more nuanced understanding of the ways that learning does and does not take place most effectively through game-based education. An especially promising factor to consider is the ability of games to enable a player to move between multiple planes of reality and to make connections between actions and sensibilities in the game world that correlate with experiences in the real world.

Additional Genres

The majority of the remainder of this paper is devoted to very brief descriptions of the genres that supplement this taxonomy. In practice, many of the generic conventions described below are combined with - or incorporated into - the genres outlined above. The remaining genres are deserving of more complete treatment than it is possible to offer here, but my hope is that these thumbnail descriptions will help to demonstrate the wide range of multimedia authoring strategies that has emerged in IML classes and suggest possibilities for a future expansion of this taxonomy.

Experiential

An experiential project creates a possibility space and invites users to explore a set of media elements according to their own interests and with great freedom of choice, usually with minimal forced temporal progression. Successful projects often activate multiple senses and move people to experience emotions or simulate a particular experience in a visceral way. To be most effective, experiential projects should sustain a clear focus on project goals and underlying research.

Encyclopedic

Encyclopedic projects bring together a collection of artifacts, objects or information, often in a historical context. Successful encyclopedic projects are clear about the scope of evidence they present and complete and detailed in their presentation. This genre is particularly inviting for projects that are enabled by the vast amount of information available through online networks and databases. However, we rarely encourage students to work in this mode, which sometimes results in disproportionate investment in the aggregation of resources at the expense of critical engagement.

Multi-perspectival

Arguably, one of multimedia's most powerful affordances is the ability to facilitate treatment of a subject from multiple perspectives. An interactive interface can easily allow a user to switch from one perspective to another, for example, or to present multiple viewpoints simultaneously. Multi-perspectival projects demonstrate the power of multimedia to present information from multiple viewpoints, perhaps allowing multiple interpretations or voices to be heard.

Multi-layered

Multi-layered projects demonstrate the power of multimedia to allow users to “dig down” through multiple layers of information to reveal the complexity of an issue or issues. This is particularly useful as a platform for encouraging students to think in terms of information architecture and user interface design. Which layers of information are revealed first? Which require further investigation and what sort of process is required to access subsequent layers?

Comparative

Comparative projects make use of multimedia's ability to perform comparisons or highlight contrasts between two or more media elements or artifacts.

Interactive

We view interactivity as a fundamental component of the most challenging work in scholarly multimedia. For some projects, particular focus is placed on creating a particularly challenging or innovative interface design, navigational structure.

Networked

One of the most dynamic fields of possibility in contemporary multimedia scholarship is enabled by the proliferation of high speed networks and the ease with which students may distribute and syndicate content online. Networked projects make particularly effective use of connections between computers, information systems and communities. This is especially important for projects that seek to mobilize the power of multimedia to reach beyond the confines of a single class or university.

Mobile

Although the potentials of using mobile devices and platforms for scholarly multimedia remain under-explored at the IML, we see great potential for integrating mobility into future coursework and thesis projects within the Honors in Multimedia Scholarship program.

Visualization

Within many of the disciplines supported at the IML (especially the sciences and social sciences), visualization has been long recognized as a valuable tool for translating and representing information. Effective Visualization projects use the power of multimedia to visualize data or to present otherwise abstract information in a readily understandable way.

Annotation

Annotation-based projects demonstrate the power of multimedia to add multiple or dynamic layers of annotation to a text or media object.

Open Architecture

Open Architecture projects are designed to be built upon by future generations or multiple users beyond their original contexts. Due to advances in network technologies and the proliferation of participatory communities, Open architecture projects offer one of the most promising genres to develop at IML.

Coda: Before Genres

Prior to the explicit introduction of genres into the curriculum at IML, student work tended to gravitate toward what I would call “multi-modal critical art projects.” Some of this work was technically, creatively or conceptually virtuosic and may very well have enriched the educational experience of the students involved. The inherent value of responding to creative challenges and working collaboratively in a stimulating media-rich environment should not be underestimated. However, the long-term goals of the IML and its programs call for a more systematic commitment to models of scholarly production that are pedagogically sound and which offer measurable learning outcomes.

The IML is committed to developing new scholarly vernaculars that contribute to an epistemic shift within higher education. A core tenet of IML programs has been the value of transforming the relationship between instructors and students; moving student work out into the world and supporting the contention that academic work can and should have relevance beyond the classroom. Increasingly, I believe that a key benefit of multimedia in higher education is the transformation of students' curricular production from stand-alone objects toward systems that are networked and extensible, promoting values of collective intelligence and civic engagement. My hope is that this modest, initial outlining of genres will support these goals and contribute productively to the IML's future curricular development in scholarly multimedia.