The Projection of Sound on Image

Michel Chion

Persona and Monsieur Hulot’s Holiday

• Opening of Persona
• Nail through hand
• Without sound, three shots, not just one
• More abstract without sound, with sound it’s terrifying
• Mortuary
• Without sound, a series of stills, parts of isolated human bodies, out of space and time
• The boy’s hand no longer forms the face, but just wanders aimlessly
• Entire sequence has lost its rhythm and unity without sound
• “Could Berman be an overrated director? Did the sound merely conceal the images’ emptiness?”
• Monsieur Hulot’s Holiday (Tati, 1953) on the beach scene
• Subtle gags make us laugh - vacationers are uptight, they don’t have fun, they’re anxious
• Without image, it’s a whole new scene
• Shouts of children playing, voices that resonate in an outdoor space, a whole world of play and vitality
• It was all there, and yet it wasn’t...

Introduction

• “Is the notion of cinema as the art of the image just an illusion? Of course: how, ultimately, can it be anything else? This book is about precisely this phenomenon of audiovisual illusion, an illusion located first and foremost in the heart of the most important of relations between sound and image, as illustrated above with Bergman: what we shall call added value” (5).
• Added value: “expressive and informative value with which a sound enriches a given image so as to create the definite impression, in the immediate or remembered experience one has of it, that this information or expression ‘naturally’ comes from what is seen, and is already contained in the image itself” (5).
• “What gives the (eminently incorrect) impression that sound is unnecessary, that sound merely duplicates a meaning which in reality it brings about, either all on its own or by discrepancies between it and the image” (5).
• Sound/Image synchronism, via the principle of synchresis (the forging of an immediate and necessary relationship between something one sees and something one hears)

Vococentrism and Verbocentrism

• Added value is that of text, or language, on image
• Cinema is a vococentric or, more precisely, a verbocentric phenomenon
• Vococentric: the sound in cinema almost always privileges the voice, highlighting and setting the latter off from other sounds
• During filming it is the voice that is collected in sound recording
• It is the voice that is isolated in the sound mix like a solo instrument - for which the other sounds (music and noise) are merely accompaniment
• The voice as medium of verbal expression
• Not worried about fidelity to original timbre, worried about the effortless intelligibility of the words spoken
• “Thus what we mean by vococentrism is almost always verbocentrism
• “Sound in film is voco- and verbocentric, above all, because human beings in their habitual behavior are as well” (6).
• You always hear voices first, then once meaning is assured, you move on to ambient sounds and music...

Text Structures Vision

• “The anchor [in the French TV airshow broadcast] could have made fifty other ‘redundant’ comments; but their redundancy is illusory, since in each case these statements would have guided and structured our vision so that we would have seen them ‘naturally’ in the image” (7).
• “The added value that words bring to the image goes far beyond the simple situation of a political opinion slapped onto images; added value engages the very structuring of vision - by rigorously framing it” (7).
• “Thus if the film or TV image seems to ‘speak’ for itself, it is actually a ventriloquist’s speech. When the shot of the three small airplanes in a blue sky declares ‘three small airplanes,’ it is a puppet animated by the anchorman’s voice” (7).

Value Added by Music

• Empathetic and Anempathetic Effects
• Music can be empathetic: directly express its participation in the feeling of the scene, taking on the scene’s rhythm, tone, and phrasing - sadness, happiness, movement, etc)
• Music can be anempathetic: conspicuous indifference to the situation, by progressing in a steady, undaunted, and ineluctable manner - the scene takes place against this very backdrop of “indifference”
• Intensifies emotion through juxtaposition
• Reinforce the individual emotion of the character and of the spectator, even as the music pretends not to notice them
• “For, indeed, all films proceed in the form of an indifferent and automatic unwinding, that of the projection, which on the screen and through the loudspeakers produces simulacra of movement and life - and this unwinding must hide itself and be forgotten” (9). [Baudry…]
• Anempathetic music unveils this reality of cinema, its robotic face - conjures up the mechanical texture of this tapestry of the emotions and senses

Influences of Sound on the Perception of Movement and Perception of Speed

• Sound, contrary to sight, presupposes movement from the outset
• By its very nature, sound implies displacement or agitation, however minimal
• Some sounds represent stasis, but usually artificial like the telephone dial tone or the hum of a speaker
• Sound has its own temporal dynamic
• Difference in Speed of Perception
• The ear analyzes, processes, and synthesizes faster than the eye
• Hearing language is much faster than reading words
• The eye is more spatially adept and the ear more temporally adept
• Sound for “Spotting” Visual Movements and for Sleight-of-Hand
• Why don’t rapid movements in kung fu or special effects films create confusing impressions?
• They are “spotted” by rapid auditory punctuation, in the form of whistles, shouts, bangs, and tinkling that mark certain moments and leave a strong audiovisual memory
• Door-opening sound (psssht) in Star Wars makes us think the door slowly opens when in fact it’s a hard cut from closed to open

Influence of Sound on the Perception of Time in the Image

• The Ear’s Temporal Threshold
• The ear listens in brief slices, not continuous
• Paradox: we don’t hear sounds, in the sense of recognizing them, until shortly after we have perceived them
• Three Aspects of Temporalization
• 1. Temporal animation of the image - perception of the time in the image as exact, detailed, immediate, concrete or vague, fluctuating, broad, etc.
• 2. Temporal linearization - synchronous sound does impose a sense of succession
• 3. Sound vectorizes or dramatizes shots, orienting them toward a future, a goal, and creation of a feeling of imminence and expectation

Influence of Sound on the Perception of Time in the Image Cont...

• Conditions Necessary for Sound to Temporalize Images
• 1. The image has no temporal animation or vectorization in itself (static shots, or general fluctuating shots with no indication of resolution - rippling water)
• 2. The image itself has temporal animation (movement of characters or objects, movement of smoke or light, mobile framing)
• Sound’s temporality combines with temporality already present in the image
• Temporalization depends on type of sounds present (density, internal texture, tone quality, progression, etc.)
• 1. How sound is sustained - smooth and continuous sound is less “animating” than an uneven or fluttering one
• 2. How predictable the sound is as it progresses - steady vs. irregular rhythm
• 3. Tempo - not as influential as one might think
• 4. Sound definition - a sound rich in high frequencies will command perception more acutely

Influence of Sound on the Perception of Time in the Image Cont...

• Conditions Necessary for Sound to Temporalize Images Cont…
• 3. Model of sound-image linkage and on the distribution of synch points
• Microrhythms: rapid movements on the image’s surface caused by things such as curls of smoke, rain, snowflakes, ripples on a lake, dunes, photographic grain on the celluloid itself, etc. (Kurosawa)
• Sound Cinema is Chronography (written in time as well as in movement)
• Indebted to synchronous sound for having made cinema an art of time - standardization of projection speed
• Although of course there are accelerated and slow-motion sequences as well
• Sound temporalized the image through added value and normalizing projection speed
• Temporal Linearization
• Bridging two shots together - silent film crowd reaction scenes make more sense with sound bridges

Influence of Sound on the Perception of Time in the Image Cont...

• Vectorization of Real Time
• Specific sounds within the scene give it a certain linear progression, opposed to being able to play a calm beach scene forward or in reverse and having the same experience
• If that scene is played backwards with sound, it is immediately noticeable - sound gives a real and irreversible time on what we see, presents a trajectory
• Images are much more reversible (generally speaking) than sound is
• Stridulation and Tremolo: Naturally or Culturally Based Influence
• Cinematic and cultural codes, not purely physical and mechanical phenomena
• Universal and spontaneous effects in stringed instrument tremolo and ambient sounds that both build tension

Reciprocity of Added Value: The Example of Sounds of Horror

• “Sound shows us the image differently than what the image shows alone, and the image likewise makes us hear sound differently than if the sound were ringing out in the dark” (21).
• Screen remains the principal support of filmic perception
• Horrific images project meaning onto sounds they do not have at all by themselves
• Offscreen images are made horrific by the use of sound - woman being murdered offscreen sounds terrifying regardless
• Same sound of watermelon being smashed can be used in a horror film, in a war film, or in a comedy
• Sound of body falling into the crypt in Eyes Without a Face...