Introduction to Multimedia Scholarship: Student Handbook

June 29th, 2008

Download PDF 

“Introduction to Multimedia Scholarship” is the Companion Handbook for MDA 140, offered in conjunction with the Multimedia in the Core and Multimedia Across the College Programs

Introduction

This is a booklet about multimedia scholarship. It is intended to accompany the Practicum in Multimedia Authorship (MDA 140), the introductory multimedia practicum attached to courses in the Multimedia in the Core and Multimedia Across the College programs.

With this text, we attempt to articulate, organize and, to some extent, inspire how multimedia scholarship - specifically, undergraduate multimedia scholarship - is practiced here at the University of Southern California. We wish to emphasize, however, that your MDA 140 practicum will be only an introduction to what is an emerging, and therefore exciting and innovative, field. Because we are limited to a single semester, and because multimedia scholarship involves several divergent skill sets (e.g., design skills, technology skills, writing skills, production skills), we have had to focus MDA 140 so that it is a more limited experience than, say, courses within the IML Honors Program. If you are interested in learning more about multimedia scholarship as a result of MDA 140 (and we hope that you will be), we encourage you to investigate the IML Honors Program, or to ask your instructors about other multimedia opportunities at USC.

We begin here with a course rationale, from which we move into definitions of media and multimedia, and finally to the specifics of how multimedia scholarship is defined and practiced in MDA 140.

Course Rationale

The Multimedia in the Core program was formally established in April 2006 after a mandate from the University's Provost Max Nikias in which he announced his desire to offer every student at USC the opportunity to gain skills in multimedia production. The reason behind this mandate is the idea that to be literate in the 21st century requires not only effective skills in reading and writing, but also the ability to use and interpret media effectively. The Multimedia Across the College program expands the opportunities for USC students to gain exposure to multimedia, offering labs in conjunction of a full array of College courses.

The MMC and MAC programs in no way challenge the fundamental importance of reading, writing and traditional scholarly practices. Instead, they build on the idea that literacy is a constantly evolving notion linked to the skills needed for people to communicate clearly, and to participate effectively in a democracy. As Dougles Kellner writes, “Literary thus involves gaining the skills and knowledge to read and interpret the text of the world and to successfully navigate and negotiate its challenges, conflicts, and crises. Literacy is thus a necessary condition to equip people to participate in the local, national, and global economy, culture, and polity.” We believe that literacy in the 21st century, then, includes the ability not only to read and write, but to use and critique images, sound, typography, design, networks and other media-based tools. The “text of the world” is now a multimedia text, and effective participation requires knowledge of multimedia tools and practices.

The result of Provost Nikias' mandate was a collaboration between USC's College of Letters, Arts and Sciences and the Institute for Multimedia Literacy to form the Multimedia in the Core Program, which unites General Education courses with a two-unit multimedia practicum section. Students attend lectures for the GE course, as well as the lab practicum, where they are introduced to an array of media-based tools and skills, allowing them to create scholarly multimedia projects for their GE course. The program, the first of its kind in the United States, began in the fall of 2006. The Multimedia Across the College program will debut in the spring of 2008, with several courses enhanced with a multimedia practicum component.

Why do we think these programs are so important? In part, we are motivated by the realization that although many people lack easy access to computers and networks, a large number of people are nevertheless using media technologies at an unprecedented level, particularly those technologies that are centered on the use of the computer. Arguably, we tend to use these technologies primarily for casual communication (email or IMing are probably the best examples of this) or to communicate expressively, for the purposes of either art or political self-empowerment. In any case, one of the premises of the Multimedia in the Core program is that it is worth trying to understand not only what such prolific usage means, but further, how media might effectively be used in an academic setting.

The reader may already be aware that the ubiquity of media has its share of critics: consider, for instance, the argument that “the media” (a term usually used as a gloss for TV and video games) overwhelms its users with a treacherous pervasiveness, distracting us with shallow entertainment, and ultimately contributing to the “dumbing down” of culture. In this dystopian view (see for instance Neil Postman, Todd Gitlin, Jerry Mander, Marie Winn and Jane Healy), the role of educators should be to protect students from the nefarious effects of media. Further, in this view “the media” and “scholarship” are assumed to be antithetical. The dystopians take as a starting point the idea that media is a static phenomenon (e.g., “TV is always bad”); they espouse, in other words, a kind of media essentialism.

We, on the other hand, take the anti-essentialist position that media technologies are pliable, and that, particularly when they are new, they can be fashioned to suit a wide variety of meanings and purposes. We argue, in other words, that forms primarily associated with art or entertainment can be made to suit scholarship. For us, the very existence of a state of media saturation - which we agree with the dystopians is a significant development - means more than ever that students need to be encouraged to think independently, but from within this new context (which is, after all, unavoidable). We want students to understand how this “media saturated” context works, and how to use it to positive (i.e., in this case, scholarly) ends. We do not want to pretend it's not there. In short, we concur with Marshall McLuhan's assessment (in The Medium Is the Massage) that

it is a matter of the greatest urgency that our educational institutions realize that we now have civil war among these environments created by media other than the printed word. The classroom is now in a vital struggle for survival with the immensely persuasive ‘outside' world created by new informational media. Education must shift from instruction, from imposing of stencils, to discovery - to probing and exploration and to the recognition of the language of forms (100).

We are also interested in shifts within academia as colleges and universities across the country experiment with a broad array of technologies in the classroom. Some educators are using video games; others are investigating the potential of virtual environments such as Second Life. And many scholars are considering ways to use Web-based tools to enhance their work and its availability. Indeed, we are at a moment in academia during which ideas about scholarship are shifting dramatically, allowing not only for new forms of research, composition and publication, but a growing sense of community built around disciplines and based on networked interaction.

In its simplest form, this kind of “networked scholarship” refers to network-mediated scholarship. However, the term has come to designate shifts in writing (as in texts composed using collaborative software or books written “in public” with reader input), publishing (thanks to blogs and wikis), and distribution, as well as in events such as conferences, which are more and more frequently taking place online in virtual environments.

Perhaps one of the strongest examples of this shift is embodied by Vectors Journal of Culture and Technology in a Dynamic Vernacular, USC's own electronic journal, which is a model for new forms of scholarly publication. Edited by School of Cinematic Arts professor Tara McPherson, the journal has so far published five issues, each of which features half a dozen or so essays centered on a theme; the essays are crafted as multimedia projects in a collaborative process uniting the writer and a designer, and all are designed to engage reader interaction and to rethink the ways in which we conceptualize scholarly writing.

Yet another example is MediaCommons, a project undertaken in collaboration with The Institute for the Future of the Book. The endeavor, led by Kathleen Fitzpatrick and Avi Santos and inaugurated in July, 2006, hopes to find sustainable models of electronic scholarly publishing that has resulted in a draft proposal outlining several ideas. Explaining the need for new modes of publication, Fitzpatrick notes that “a reimagining of the scholarly press through the affordances of contemporary network technologies will enable us not simply to build a better publishing process but also to forge better relationships among colleagues, and between the academy and the public.”

Several scholars are experimenting with “designed writing” and writing in public. The New School's McKenzie Wark, for example, published his latest book GAM3R 7H3ORY, which explores games as allegories for contemporary life, online. Indeed, he composed the text as a series of “cards” that are stacked in nine color-coded chapters, each one allowing reader responses. Almost every single card in the book has attached to it a conversation between the writer and readers, who make suggestions, ask questions or push the writer to explain his points further.

Other writers are considering notions of “transmedia publication,” uniting book projects with significant media-based components, not as complements to the written text but as fully integrated material that acknowledges the affordances of each media included. Art Center College of Design's recent book The New Ecology of Things, for example, includes QR codes allowing cell phones passed over pages to access short videos; the project also includes an extensive Web site with further materials.

Other scholars are distributing their books in serial form through blogging software or on Web sites, while still others are making their books available for free download: Yochai Benkler's The Wealth of Networks: How Social Production Transforms Markets and Freedom can be downloaded in full as a PDF, or purchased as a regular book. It also features an ever-expanding wiki site with various discussions based on the book.

As more and more faculty investigate new approaches to scholarship, we can expect new forms of practice in the undergraduate classroom as well.

Media: A (Very) Brief History

What exactly is media, and how and when did it become so significant for our culture ? In this practicum we define media as those technologies and social practices that are used to convey ideas or information. Of course, from our early 21st century vantage point, informed in part by reader-response criticism, and in part by what we have just described as a state of media saturation, it is possible to argue that almost anything can be a medium: one's perception of a “message” in a book, a tree or a grilled cheese sandwich can make any of those things a form of communication. But until the 20th century at least, media was thought of in much more specific and limited terms. In fact, for a good deal of western history, “media” has been a fairly limited category signifying writing and printing moment, and related, to a lesser extent, to art (painters, for instance, might speak of working in a particular medium, such as watercolors).

Certain industrial and social developments in the 19th century led to the creation of mass media, and this became a dominant conceptualization of media for roughly 100 years. Mass media enabled broadcasting, or the dissemination of ideas and information to a wide audience; its forms included books, newspapers, phonography, telegraphy, telephones, photography, film and, eventually, television and radio. In the 20th century, mass media would come to be defined not only by the potential scope of its audience, but by the fact that it was typically controlled and administered by an economic and cultural elite.

More recently, of course, the idea of truly mass media has been complicated by the emergence of the personal computer and, perhaps more importantly, by the Internet. The personal computer makes sophisticated media production capabilities available at greatly reduced costs, and the Internet enables anyone to publish such productions for anyone else to access at any time, from anywhere else in the world (assuming the receiver also has a computer and an Internet connection). These technologies thus allow independent authors -i.e., individuals not affiliated with the aforementioned economic / cultural elite - to create high quality work that can then “be shared with an extraordinary number of people, practically instantaneously” (Lessig 41). As a result, there has been a blurring of the boundaries between “consumers” and “producers,” as well as the introduction of a new category of media: “independent” (as opposed to “corporate”) media.

Defining “Multimedia”

So how exactly is “multimedia” distinguished from “media”? We would like to propose defining multimedia in three possible ways.

• First, what might be the most popular definition, and one that overlaps somewhat with the term “new media”: multimedia is the environment of the computer. Or, as Fred Hofstetter puts it: “Multimedia is the use of a computer to present and combine text, graphics, audio and video with links and tools that let the user navigate, interact, create and communicate” (2). This definition makes a certain degree of sense: computers do allow for a “convergence” of media in that the microchip enabled the creation of “digital workstations” - self-contained production units that allow a single user to work in multiple media forms on the same platform. As a result, one can (for instance) write a script, edit a film sequence, mix an audio recording, create a Web page, manipulate a photograph and design a presentation, all within the context of a single laptop.
• Our second definition must acknowledge that although media technologies were physically disparate before the personalization of computers, this does not necessarily mean that multimedia (or at least “proto-multimedia”) practices did not exist before the so-called “digital age.” One could say that opera, for instance, which originated in 16th century Europe, is a multimedia art form combining literature, music, drama, stagecraft, and so on. McLuhan argues that every medium contains some other medium; if one accepts this formulation, then (paradoxically) media has always been multimedia. A definition of multimedia, in other words, may not depend on the computer per se, but may simply refer to the bringing together of any two (or more) previously dissociated media in a new environment or “platform.” In this sense it may only be the pervasiveness, obviousness and broad accessibility of multimedia that is new to our time.
• Third, an even more abstract definition of “multimedia” is that it is a type of awareness. That is, one can argue, given the media proliferation of the 20th century, that it is no longer possible (if indeed it ever was) to experience media objects “purely” or discretely (that is, as separate from other media objects). Again, McLuhan: “As we read, we provide a soundtrack for the printed word; as we listen to the radio, we provide a visual accompaniment” (234). While prehistoric humans may have been able to perceive a cave painting as only a cave painting, “postmodern humans” presumably experience cave paintings in terms of a number of other media forms. At the very least, since most of us have not actually traveled to view cave paintings for ourselves, we are forced to view them through the media of photography and printing. Further, our immersion in a world of film, television, fashion and other visually biased media forms means that we come to view cave paintings with a complex historical and experiential lens (a lens which, to complicate this even further, is clouded for every viewer by personal subjectivity). Perception is inherently a hall of mirrors (or echo chambers): one's experience of a given media object is at least partially determined by one's experience of other media objects (and other genres of media).

Still, these latter definitions are at this point more philosophical than pragmatic. Each is valid, but none is quite as useful for our purposes as what is more or less the most straightforward conception of the term: that is, the first definition, that a multimedia project is something that exists for and within the digital environment. To put it even more simply: a multimedia project for MDA 140 is a project created using the affordances (i.e., unique capabilities) of the computer.

Defining Multimedia Scholarship
Now that we have a basic sense of some qualities of media and multimedia, we need to introduce the basic criteria of multimedia scholarship as defined in MDA 140. As the program took shape in 2006, faculty, IML staff and teaching assistants gathered with the goal of defining the MDA 140 practicum. What are the fundamental multimedia skills or ideas that should be covered? After much debate, we arrived at the following list of five “foundational literacies” which every MDA 140 section should address, as well as a list of ten other literacies that are significant, and could be adopted depending on the needs and goals of a particular course.

Five Foundational Literacies

1. Digital literacy refers to the ability to understand the basic aspects of multimedia tools and software, and covers everything from the protocols for compression, back-up and file naming to definitions of terms (frame rate, dpi, etc.) and basic equipment usage.
2. Network literacy refers to the ability to use networked software for intelligent participation in online communities.
3. Design literacy refers to the ability to use appropriate design principles for multimedia authoring in a specific context, and the ability to control the relationship between form and content.
4. Argumentation focuses on the ability to develop, express and defend a persuasive thesis using media, as well as the ability to use evidence and complex thinking in constructing an argument.
5. Research literacy refers to the ability to perform effective, critical online research; knowledge of academically appropriate protocols for selection, citation and attribution of source materials; and knowledge of fair use and copyright issues.

Ten Supplemental Literacies

1. Presentation: The ability to understand and articulate basic strategies for effective presentation using multimedia, as well as how to disseminate these materials to a wide audience.
2. Visual literacy: The abilities to convey information visually and to understand and control systems of visual signification.
3. Sonic literacy: The ability to communicate effectively with sound.
4. Interpretation: The ability to use multimedia to enhance a critical interpretation, and the ability to identify and articulate the cultural, historical and ideological contexts of a media object.
5. Annotation: Understanding strategies for critical annotation of text, images and media.
6. Collaboration: The ability to work effectively in a group authoring environment, as well as the ability to lead a team project.
7. Narrative literacy: Knowledge of basic components and genres of narrative, and the ability to deploy elements of narrative in a critical context.
8. Pedagogical literacy: Understanding strategies for creating an effective tool for teaching.
9. Interactivity: The ability to communicate effectively in a non-linear, interactive context, and the ability to design an effective interface or navigational structure.
10. Code literacy: The ability to understand the basics of how code operates, and the ability to write or use basic code.

In addition to these core literacies, students will also write. It is important to recognize that the practicum does not reject traditional humanistic scholarship, by which we mean argument-driven writing that is clear, concise and correct in form. More specifically, multimedia scholarship in MDA 140 requires “articulated knowledge” about a subject - what David Buckingham calls “metalanguage” - and this articulation must often be in the form of writing. What's so important about writing in a multimedia practicum? Good writing skills allow you access to the broader academic discourse community, and they enable you to communicate your thoughts to other scholars and to find out what those other scholars are thinking. These skills remain vital, then, within any academic community.

So what can multimedia forms of communication do well? And why do we want to employ them in this class? Again, to clarify, we're using the term multimedia production as a way of describing communication through the technologies of multimedia (i.e., still images, moving images, sounds and so on). And it is our position that there is a good deal of value that this kind of communication adds to scholarship as we know it.

First there is the quality of scope. Typically, even a single MDA 140 project is “bigger” than a portfolio of 5- or 7-page papers produced for WRIT-140. This is partly a material issue: multimedia projects by definition bring together several different forms of communication, and as a result, file sizes are typically larger than those of a standard word document. Students need access to an increased number of research materials, raw data and production tools; they end up pulling more elements together, and learning many skills, in a single semester. This is true even though MDA 140 projects are essayistic and argument-driven (like traditional academic papers), and even though they generally do not involve databases. In other words, while MDA 140 projects do not necessarily maximize the capabilities of the digital environment, they're still “heavier” than traditional papers. This “weight” is surely a fitting addition to scholarship as it exists in a multivalent and complex world.

Second, multimedia writing prompts students to be aware of the design issues involved in multimedia scholarship. This is not to suggest that traditional writing doesn't have its own set of design issues (which come through, for instance, in the question of form, font choice and so on). But in MDA 140 projects, design is given more emphasis. Put another way: a given academic paper exists in a well-established tradition, and is thus more beholden to a narrow set of design conventions; multimedia production, on the other hand, freed from the limitations associated with a grammar of scale, is thus able to draw on (and subvert) multiple conventions - as well as aspire to new ones.

Many have argued that multimedia production lends itself more readily to the expression of affect, perhaps because it plays on the dramatic conventions that have grown up around the use of images or sounds, particularly in the film industry. Of course, at its best, even scholarly writing can be passionate, but in MDA 140 this quality can be more successfully and legitimately brought to the foreground. Academic papers, for instance, generally do not encourage the use of narrative forms, while narrative is fair game for multimedia production. And in multimedia production, “passion” can also be expressed in a wider variety of ways.

Another distinction is that multimedia productions may be easier to distribute, a condition that has particular implications for projects that deal with previously neglected histories, or the points of view of outsider populations. MDA 140 projects, on the other hand, may make their way into a social and even commercial network outside of the classroom.

Finally, given the freedom allowed by the undo function, the ability to save multiple versions of a project, and the flexibility and malleability made possible by an object's existence in a digital environment, multimedia production, as David Buckingham points out, encourages experimentation and improvisation (which are, incidentally, key elements of learning). Again, there is some overlap with traditional writing here: it is true that one has a considerable amount of experimental freedom when using, say, a word-processing program as opposed to a typewriter to create an essay. But there are simply more (and more far-reaching) possibilities with multimedia writing.

It is worth pointing out, incidentally, that each of these characteristics is not only a distinguishing quality of multimedia production but also a potential pitfall for neophytes in the field. The issue of scope can potentially lead to a project that is overbroad and lacking in depth, or else one that works against students' imagination (and memory) by submitting to the impulse to make everything visible. The issue of design can potentially lead to a project that is pretty but insubstantial - a multimedia version of what in WRIT-140 is called “the well-wrought void.” The issue of affect can lead to an excessive subjectivity that works against critical consciousness. The issue of far-reaching distribution can actually undercut any democratic goals a student might wish to express via a project (increased access to distribution puts more “stuff” into the world, arguably making it more difficult for producers to be heard; this issue also points toward the importance of an as-of-yet unattained standardization of platforms). And the notion of experimentation can make a student aware of so many design options that he or she feels paralyzed and is unable to make careful or informed choices (this is what Alvin Toffler called “overchoice”).

Still, there is a definite value to multimedia production, just as there is to traditional writing. And in MDA 140, multimedia scholarship is defined by a symbiotic relationship between the two. We do not seek a hierarchy in which one is more or less important. Multimedia projects are not intended to replace traditional papers, but to complement them. On the other hand, good traditional writing skills will not be enough to enable a student to be successful in this course. Multimedia scholarship in IML-140 is a highly complex skill set, in that it asks that students be great writers, media scholars and artists, while simultaneously displaying competence in the topics of a given field (i.e., the “content” of a project). Multimedia scholarship is a hybrid, inter- (or, if you like, post-) disciplinary, and comprehensive means of processing and conveying information and ideas. It is, in the end, intentionally multilayered, multimodal and multivocal, combining at least two distinct experiences.

As a student in the Multimedia in the Core or Multimedia Across the Curriculum programs, you are part of a paradigm shift and redefinition of scholarship within the university setting. Your professor and teaching assistants are working together to facilitate new learning experiences, and we congratulate your decision to participate in this endeavor, and wish you well in your work this semester.