One of the biggest draws for students considering the USC School of Cinematic Arts is the top-rate faculty. The Interactive Media Division is no exception to this. SCA Family Stories sat down with Chanel Summers, a pioneer in the field of audio design for games, to talk about her role as an educator, what students can do to prepare themselves for the real world and how in the world of videogames, and that innovation usually comes from non-traditional people who explore unorthodox paths.

Let’s start with your name and title. My name is Chanel Summers and I’m the co-founder of a company called Syndicate 17, which is an audio production house that creates music and audio for films, television, commercials and games. Starting in the Spring semester, I’m going to be lecturing here at USC in a brand-new class, which I created, called Audio Expression.

For people on the outside, what is Audio Expression? Audio Expression is about creating aesthetic and artistic audio for video games. Officially what it’s going to do is to introduce students to the key principles and technologies that will enable them to process, mix and control sound for aesthetic effect in order to craft the story elements of a game, control the pacing of gameplay, enforce the gameplay narrative, elicit and influence emotion, create mood, shape perception and reinforce the way that players experience game characters.

But really, we are going to teach them to become artists. We are going to look at things like the major attributes or components of sound, then we’re going to get into sound propagation and the physical manifestation of sound, the impact of sound on perception, the different “modes of listening” and the physiological responses to sound. Then, we’re going to get into the basic elements of audio aesthetics, including the development and documentation of an audio aesthetic. Finally, we are going to get into the implementation and execution of that audio aesthetic using game-specific audio tools, technologies and techniques. We will look at dynamic, real-time mixing and real-time DSP parameter control, physical modeling, procedural audio and things like that.

The inspiration for this class in many ways was Bruce Block’s amazing Visual Expression class. In a sense I hope that this will become in many ways the interactive audio version of Bruce Block’s class.

Who is this class for? Is it for advanced students only? Not at all. Ideally, students should have a basic understanding of recording, editing and processing sound, but frankly, I love it when students don’t have an extensive audio background because it opens them up to new ideas and draws out new forms of creativity. I’ve spoken to some of the students who have registered for the class already, and they have backgrounds as diverse as fine arts and English literature. I think that’s wonderful.

The folks that are going to push the boundaries are the ones that aren’t constrained by the rules. They are the ones that are going to do something cool. They’re going to push the industry. They are going to do things that people that have been in game audio for several years haven’t been doing. There’s a student here - she’s an MFA – and her thesis project is essentially a sonic adventure game. She had no formal background in audio production. Her game utilizes the principles of HRTF (head-related transfer function) and binaural audio, and it plays around with frequencies.

So, if you listen to the game, she uses these high frequencies where you feel like your brain is going to implode, and she uses these very low frequencies that you can feel deep inside your body. What’s great is that, although she doesn’t fully understand the concepts behind it (like resonance and entrainment), she’s doing it because she feels the need to create. That’s fantastic.

What should students do to prepare for the course? Get a little bit of understanding of recording, editing, and processing sounds. Familiarize yourself with an audio editor. Two good choices to start with are Adobe Audition and Audacity. Go out, record some sounds, put them into your editor and start manipulating them and tearing them apart. Apply filters. Reverb. Just experiment. Sound design is often about being a mad scientist. This class is going to be very much about experimentation and aesthetics. It’s about how we can do something tremendous. Just allow yourself to be free and to experiment. That’s the best preparation.

What are your thoughts on game audio as an art form? It’s funny that, as the technology has advanced, we’ve still only scratched the surface artistically. We’ve got all of these wonderful tools and technologies out there, such as procedural audio, real-time effects processing, and dynamic, real-time mixing. You can place sounds anywhere in 3D space. There are games supporting surround sound technologies. Composers are recording epic scores with live, large orchestras at sample rates comparable to film sound. But again, despite all our technical advancements and all this cool sounding stuff, I would argue that we haven’t even begun to scratch the surface, artistically.

So, game audio is as technically well-executed as a feature film, but with only a few exceptions, I assert that nobody is trying to create “art” with their audio. I should say, there are some game makers who are going above and beyond to create artistic experiences with the audio in their games, but they are the minority. These few individuals represent the future of game audio design.

You’re an advocate for promoting innovative and artistic audio for games, correct? Absolutely! When I was at Microsoft, I had the title, “Audio Technical Evangelist.” I was the company’s first “evangelist” for audio, and my job was to go out and educate folks on how to use audio technologies in order to encourage the adoption of those technologies into their applications. And as an evangelist you can only succeed through effective education about the benefits of those technologies and by making it as easy as possible for developers to take advantage of them.

What do you mean by adoption? Using the techniques in game development? It’s more than that. When I was in games, audio was just window dressing; a backdrop considered to be far less important than the game’s visuals. But I was involved in interactive audio technology development from very early on. At Microsoft, we developed a technology called DirectMusic. It was phenomenal. It’s no longer being actively supported by Microsoft, but it was way ahead of its time. Even now, years later, I continue to hear from developers that there’s nothing quite like DirectMusic out there today.

With technologies like DirectMusic and DirectSound, I was educating folks and getting them to think about interactive audio. Games are non-linear; audio should be non-linear, too! It should underscore the action. It should change and evolve based on players’ actions. It should parallel the narrative or play against the narrative.

That was a struggle at the time and an uphill battle. When I worked on Xbox audio, I was helping various game development teams to make sure that they took advantage of what could be done on this amazing new console. I created the game industry’s first support team to assist sound designers, musicians and graphic artists in taking full advantage of the Xbox’s capabilities.

So you were mainly talking to people that were already making games? Sure. It was like, “You want to make a game for the Xbox? We want to make sure it’s the best game possible.”

It’s often been a struggle to get producers or game designers to understand the importance of audio. What’s great about teaching this class is that I can talk to the next generation of game creators and help them to think about all the great things that interactive audio can do to enhance their players’ experiences. I can show students how important audio is and demonstrate what happens when they formulate a comprehensive approach to audio as early as possible, just as when they formulate a comprehensive approach to all aspects of their application. Really, the time for a game designer to start thinking about audio is during the initial design process. Frankly, audio can even drive a game design just as often as the other way around! Excellent content and game play means graphics and physics and immersion and sound and music and art and story. It’s all of it.

It doesn’t just have to be one piece that gets the focus. You need to focus on the whole package!

What are some primary examples of times when sound was perfect for a video game? I would say that one of the best games out there for audio right now is a game called Limbo by Playdead Games. It is unbelievable. The focus is on minimalism, with an emphasis on silence and subtlety. The audio designer is Martin Stig Andersen, and he basically creates this experience where the “music” emerges from the environment. Rather than relying on a traditional music soundtrack (which could be seen as overtly manipulative), the game instead uses sound effects for their “musical” qualities. He mixes it in a way that it is driven by the subjective experience of the character, emphasizing the sounds of upcoming obstacles and environments even before they are revealed visually. Conversely, as the player passes certain objects they may be silenced entirely, even though they may still be on-screen, if the on-screen objects are no longer “important” to the gameplay. It’s non-linear. It’s dynamic. The sounds are also created using electro-acoustic principles.

They use the concept of “ambiguity.” It’s a great aesthetic principle that games don’t normally utilize. Or perhaps I should say that games don’t utilize ambiguity in the way that, say, painters or sculptors do. Great artists utilize ambiguity extensively. Their works are open for interpretation.

Limbo purposely wanted to use that device. They were careful not to manipulate the player. They wanted to use sounds that didn’t have a strong identity or association to something.

Secondly, there’s a really fantastic movement that’s creating a small number of games where sound is the game. They are audio-centric games--games with audio-driven implementations, sonic adventures and augmented audio experiences that tell their story through sound. You “see through your ears.” Papa Sangre is a sonic adventure/horror game which utilizes binaural audio. You have your headphones in and you listen to the audio cues to help you navigate through the adventure. Enemies will growl, and snarl and chase you. And obviously you’ll want to avoid them. Movement is controlled by the touchscreen, where you take steps forward and then turn to face the sounds.

Similarly, another game called Dimensions from RjDj is an alternate reality soundscape game that uses augmented sound to turn the world around you into an adventure game. Dimensions is played with headphones and sounds from your actual world are picked up by the iPhone’s microphone and then enhanced and manipulated and integrated into the story of the game. It’s super trippy. I played it around my Mom’s dogs and thought my head was going to explode.

Then, there’s a game called Pugs Luv Beats, which is from an Edinburgh-based developer named Lucky Frame. This game marries elements of resource-gathering games with the grid-based music sequencing interface familiar to most musicians. This game turns arcade-style rhythm games on their heads! This game basically combines traditional gameplay with traditional music composition techniques to create something completely unique. It’s basically these pugs - the dogs - have destroyed their civilization because they have this love for beets... the vegetable. They get so overzealous that they blow their world up and they have to go to other planets and rebuild their civilization. In the game, the player needs to help the pugs (the dogs) rebuild their lost civilization by guiding them to grow beets (the vegetables) which in turn create beats (the music).

All of the game audio is generated by the player’s actions—you create music as you guide pugs around the game collecting beats, making increasingly complex melodies as you progress. Essentially, you are the composer and the pugs are the instruments. As the pugs run across the screen, they trigger sounds, which differ based on the type of terrain the pug lands on. And the sound is also affected by whether the pug has been guided to a tile by the player or if they are passing over that tile en route to a different destination. As the player progresses through the game, they travel to different levels (or “planets”), each with its own different composition template and sonic attributes. And giving the pugs costumes will help them to tackle various terrains and give them different synth voices. Pretty bizarre, but highly creative game!

These all sound amazing but a little bizarre. It’s not all that bizarre. Thatgamecompany, which came out of USC, has always done an amazing job with audio in each of the products they’ve done - flOw, Flower and Journey. And Journey’s soundtrack was just nominated for a Grammy. That’s really fantastic!

Is there something students can do to prepare for going into the Interactive Media Division?Let’s start with high school students. What advice do you have for them? You need to be well versed in a number of subjects. As I said, games are only going to get better and break boundaries when kids have a fresh, multi-disciplinary approach. This applies to both high school students and IMD students, by the way.

Again, when I was at Microsoft, I was asked to give a talk at USC about Xbox and careers in video games - this was before the formation of IMD - the School was like, “You’re from Microsoft so we’re only inviting engineers to your talk since that’s who we think would be an appropriate audience,” and I said, “Absolutely not.”

I wanted kids from the Cinema School there. I wanted kids from the Fine Arts School. I wanted kids from Letters, Arts and Sciences. I wanted Creative Writing. I wanted fresh minds that wanted to actually advance the industry versus keeping it stagnant.

When you get folks that say, “All I’ve done is study and analyze games in life,” you’re not going to get anything new out of them. A foundation in games is great, but you want to go beyond that.

When you’re not constrained by the rules -or even the past- that’s when you’re going to get new ideas.

Would you say that they should study the humanities or just a variety of stuff? They should study everything. I mean, sound design itself is a multi-disciplinary field. You want to understand acoustics. You want to understand psychology. You want to understand physiology. Sound manipulates the mind and it manipulates the body.

I want them to understand literary and filmic devices. I want them to understand painting, art, opera, etc. How to put that into sound design. Again, the concept of ambiguity, misdirection, counter-functional sounds, subtlety and silence. All of those principles of aesthetics. I can’t say enough that, when I’ve heard students that have said, “I’m going to take a psychology class. I’m going to take a class in ceramics.” That’s perfect. It will make you a better and broader person.

I always encourage both high school students and students here in IMD to not have a narrow focus. Being narrow isn’t good for you as a person and it isn’t good for the industry.

What about IMD students that are here? Do you have any advice for them? Aside from my advice above, I highly encourage our current students to network.

What does networking mean for an IMD student? Is it reaching out to the industry? Again, it’s more than that. Network with your fellow students. In your classes, you could meet your next programmer or artist or partner in your company. You’ve got to do that straight away.

The students should make teams and companies amongst themselves. It’s important to not just think, “I’m going to work for this game developer or this game publisher.” Think beyond that. Think about, “I’m going to make my own business. What do I need to do?”

Those folks who see their classmates as the people who they are going to work with for the rest of their career have a leg up. The Advance Game Projects class is great for this!

I went to Demo Day and it was incredible. I was talking to some of those teams. I was like, “I hope you’re thinking that you guys could be your own company.” Some of them want to do their own thing. I think that’s great.