Cinema as proto-affordance language — spatial syntax
event/perception
Stories don’t exist.
Post hoc ergo propter hoc.
After this, therefore, because of this.
Aristotle: There are no contradictions, which
means that language
and stories are impossible.
The Copernican Revolutions
The Copernican Revolutions
Astrophysics
The Copernican Revolutions
Astrophysics
Biology
The Copernican Revolutions
Astrophysics
Biology
Neuroscience
The Copernican Revolutions
Astrophysics
Darwin
Neuroscience
The Copernican Revolutions
Astrophysics
Darwin
Kandel/Okeefe/Mosers
“Nothing illustrates the transition from mythic to theoretic culture better than this agonizing process of demythologization, which is still going on, thousands of years after it began. The switch from a predominantly narrative mode of thought to a predominantly analytic or theoretic mode apparently requires a wrenching cultural transformation.”
Merlin Donald Origins of the Modern Mind 1991
Words are not real.
Arbitrary/Specific
The arbitrary trap:
Symbols
Metaphors (words)
Narratives
Cause and effect.
“What do animal signals mean?” Rendall 2009
“Signs of all times: Entoptic phenomena.” Lewis-Williams
Early external specifics in media, direct neural contact
Certain languages are not as arbitrary.
Ideal languages are never arbitrary.
Path to the specific (or affordance) is morphemic/syntax.
(pluralization, deixis, aspect, concantenative morphology.)
Intelligence is spatial, fluid, active.
Words are stand-ins, hindrances to intelligence.
“a character, the basic reading unit in Chinese (see discussion in Li & McBride, 2014), represents a morpheme and maps onto the sound of an entire syllable rather than smaller phonological units such as phonemes”
Deixis — the movement and placement of space in Mayan is morphemic.
Mayan glyph/cartouche ideo/logo/syllabo/phono.
Mayan glyph/cartouche ideo/logo/syllabo/phono.
Mayan glyph/cartouche ideo/logo/syllabo/phono.
Mayan glyph/cartouche ideo/logo/syllabo/phono.
Navajo, the concatenating language.
Arbitrary/Specific
Concatenate
2024 double-whammy: words are not thoughts/grammar is not meaningful
The construction of the system of coordinates of horizontality and verticality is extremely complex . . . it is, in effect, not the point of departure of spatial knowledge, but the end point of the entire psychological construction of Euclidian space.”
Piaget quoted in “Bodies in Space: Film as Carnal Knowledge” Michelson 1968
The next path after words/narratives, the ecological:
J.J. Gibson
Julian Hochberg
James Cutting
Joseph Anderson
“...words are a terrible straitjacket. It's interesting how many prisoners of that straitjacket resent its being loosened or taken off.”
Stanley Kubrick
Muybridge’s trotting horse is the first moving photo, proving horses leave the ground.
His zoopraxiscope was the first screen projected motion picture, built in 1880 (a full decade prior to Lumiere/Edison) entrancing audiences in the dark.
We should perceive cinema as
analog software.
They’re a medium we’ve barely begun scratching the surface of.
To be specific, movies are
neural-retinal spacetime software.
“Analog computation is often contrasted with digital computation. . . Roughly, abstract analog computers are systems that manipulate continuous variables to solve certain systems of differential equations.”
Gualtiero Piccinini
There is no natural process
controlled by an algorithm.
Cave art → Alphabets → Movies
Organized chaos that flows episodically.
The ‘secret’ language of blockbusters:
Spatial blockbusters offer the next-format language.
The Brain’s process:
Where computers cannot touch us.
The Brain’s process:
Death Star Pac Man
“All these properties (motion-heading, speed, layout, ground, obstacles, other objects in relation) amazingly enough, are specified (not represented) by information in the optic flowfield. Remember the chase scene through the woods in George Lucas’s Return of the Jedi? That’s pure optic flow.”
J.A.S. Kelso
Our media is lacks mastery of the spatial.
�Movies are the only human language defining
s p a c e
Meaning is created in space and the relationship between things in space. Spatial syntax is the most important aspect of media and it’s only been used in a small percentage of films ever made.
Architecture and natural settings create emotion, affect our senses and craft deep memories without the use of words or narrative. Why have we not developed a format of the spatial to augment and even replace these primitive forms?
“I’m not that interested in narrative. The dialogue doesn’t have much meaning in any of my movies [which are] going for emotions over ideas.”
George Lucas
“... the plot is just a way of keeping people’s attention while you do everything else…”
Stanley Kubrick
“...no one has yet really found the way to utilize the greatest potential films have. I have a feeling that no one has begun to do what a movie really could do.”
Stanley Kubrick
“The designed world speaks a spatial language.
So much information emanates from the spatial forms and patterns that we create and surround us. They are informative in and of themselves. They tell us what they are and how to interact with them…”
Psychologist Barbara Tversky
"All living things must move and act in space to survive. The basic choice is to approach or avoid, an act replete with emotion. Our explorations in space and our actions with the things in it form the foundation of thought, not the entire edifice but the foundation. Evidence comes from the brain, from perceptual-motor couplings, from language, from gesture, and from graphics. This view has implications for thought, for action, for communication, for creativity, and for the design of the world."
Psychologist Barbara Tversky
“..the ability to think flexibly could be considered as a basic constituent of cognitive reappraisal in that it requires an individual to flexibly adopt and to generate
new perspectives, solutions or strategies
accompanied by the inhibition and overriding of the
typical and most obvious responses generated by situations.”
Creativity and Cognitive Control in the Cognitive and Affective [emotional] Domains
Fink, Perchtold & Rominger
��
“Let every eye negotiate for itself, �and trust no agent.”
Shakespeare
��
The eye and its optic nerve is the greatest compression of data in biology, which words inefficiently interrupt.
�Use this as the path for evolving media towards spatial syntax, infinite novelty, and much more.
��
examples of
spatial syntax visualization,
Erle Loran Cezanne’s Compositions
Studies of actual locations/views painted by Cezanne. By comparing them, Loran discovered discrepancies and diagrammed optical processes in play: multiple and overlapping POVs, refractions, spatial relations.
Here we get to peer into Cezanne’s abilities to combine optical perspectives (top image), or shift & fold space (bottom).
Top visualization: Interdependent local space (“seeing as many sides of problems as possible”).
• Multiple views composed into a single viewpoint. �• Problem-solving as by-product of disembodied, multiple-perspectives. Notice objects are bisected by perspectives unrelated to the frame’s plane; not contained by either. “Nothing is ever two-sided.”
Bottom visualization: Heliocentricity (how solar systems function)/angular momentum (how spinning alters gravity).
Embeds basic, unseen aspects of physics visually.
Erle Loran Cezanne’s Compositions
Studies of actual locations/views painted by Cezanne. By comparing them, Loran discovered discrepancies and diagrammed optical processes in play: multiple and overlapping POVs, refractions, spatial relations.
Here we get to peer into Cezanne’s abilities to combine optical perspectives (top image), or shift & fold space (bottom).
Top visualization: Interdependent local space (“seeing as many sides of problems as possible”).
• Multiple views composed into a single viewpoint. �• Problem-solving as by-product of disembodied, multiple-perspectives. Notice objects are bisected by perspectives unrelated to the frame’s plane; not contained by either. “Nothing is ever two-sided.”
Bottom visualization: Heliocentricity (how solar systems function)/angular momentum (how spinning alters gravity).
Embeds basic, unseen aspects of physics visually.
Erle Loran Cezanne’s Compositions
Studies of actual locations/views painted by Cezanne. By comparing them, Loran discovered discrepancies and diagrammed optical processes in play: multiple and overlapping POVs, refractions, spatial relations.
Here we get to peer into Cezanne’s abilities to combine optical perspectives (top image), or shift & fold space (bottom).
Top visualization: Interdependent local space (“seeing as many sides of problems as possible”).
• Multiple views composed into a single viewpoint. �• Problem-solving as by-product of disembodied, multiple-perspectives. Notice objects are bisected by perspectives unrelated to the frame’s plane; not contained by either. “Nothing is ever two-sided.”
Bottom visualization: Heliocentricity (how solar systems function)/angular momentum (how spinning alters gravity).
Embeds basic, unseen aspects of physics visually.
Field theory, physics folded intuitively into imagery.
Humans perceive asymmetry as signs of life and of motion, a continuum, showing depth even when organized in 2-D. In contrast symmetry denotes death and immobility, and cannot display 3-D images with their 2-D compression.
The concepts of motion, asymmetry and continuum meet headlong in Movie.
Michael Leyton’s Symmetry, Causality, Mind
Visualization: Hidden physics that surround us made spatial as doorways to perception.
A cognitive scientist. Leyton identifies aspects of time, space and perception lurking in works of Picasso and Raphael, showing how humans are attracted to complexity when it obeys certain rules of organization.
Leyton suggests our visual systems have adapted to develop fundamentally new ways of thinking. He’s saying humanity is empowered by the capacity for infinite novelty. This is the path to survival.
Movies transcend our arbitrary languages and hold the keys to a specific language that shifts us away from story and algorithm.
��
A topological map is a continuous function between two topological spaces.
��
Certain forward thinking painters, illustrators, filmmakers operate topological maps.
Escher’s ‘equilocal’ topological
Topological birth of the corridor
The sciences thought spoken word, written language and stories offered a path to unlocking how brains work, but words are unrelated to thoughts (slide 29) and the way stories arrange desires and beliefs isn’t wired in the brain (slide 5; which means storytelling is illusory but its deeper associations are not). The reality is brains assemble memories, focused at recording by emotion and senses, organized at different scales of space and time. (slide edited with G. Northoff)
The hexagons above right aren’t in the wiring of neurons, instead they’re a pattern in the brain’s spatiotemporal code. The code’s origin, endlessly reflecting our sensory input, is not merely internal, there are external aspects as well. Northoff (left) suggests a missing link to the spatiotemporal code is that brains are inherently Copernican: possessing irreducible relationships to celestial bodies and the invisible fields of physics. (slide edited with G. Northoff)
The spacetime code of brains operates in an earthly reality of knowledge framed in the sun and other heavenly bodies, something only just arriving to current studies of the brain. Stories, which rely on cause & effect, mindreading intention lead only to glass ceilings of earthly reality/knowledge (they’re simplistically ego-geocentric or Ptolemaic), which allows them to be computationally reduced. Forms beyond this scope - events - are beyond storytelling and are Copernican. (slide edited with G. Northoff)
Ecological psychology and cognitive mapping may lead to the organizing principles of consciousness - there exist doorways between the brain and uniquely reorganized, spatially complex movies.
This and the preceding pages are examples of
“motion-glyphs” which are externalized affordances/spatial syntax.
Motion-glyphs aren’t rational space or physics; illustrated above are four distinct space-times/optics impossibly overlapping yet combined seamlessly. A new form of language at its very beginning that we’ve yet to recognize.
Motion glyphs are homologous to ideo/logogrammatic writing like Chinese or Mayan.
Motion-glyphs offer a sense of timelessness
of “now travel” rather than the implied motion-picture idea of time-travel; imagined through set-design, continuity, editing, multiple-matte optical effects, integrating motion/space at different scales and speeds.
A radical form of language hiding in plain sight as entertainment. A basic rule of this reorganization:
It isn’t what it looks like, it’s what it’s made from.
Kelso’s integrated dynamics of optic flow …
Discard cause and effect,
retain the relationships of events.
Discard figure / ground,
retain the relationships of events.
Mass entertainment of specifics or
cinematic game of affordances.
“..our memory is influenced by experience that occurs long after the remembered events.
Remembering the past and imagining
the future may be regarded as
remembering the future and imagining the past.”
Imagining and Experiencing the Self on Cognitive Maps
Arzy & Dafni-Merom�
The problem: media conditions us through cause & effect and AI follows suit.
Yet the brain is a dynamic experimental, improvisational engine using discrepancies between actions and outcomes, action
and representation, planning and experiencing.
Even discrepancies between how adults problem-solve (we call this
phylogeny) and how children learn (ontogeny).
Future entertainment is down these paths, yet present-day criteria for media is ancient: narrative/cause and effect, a very low-dimensional form of very indirect perception.
��
Movies are shortcuts around the limitations of narrative and language that wind up replacing them.
��
Alternate reality games are replacements of narrative and language in a new format of play that reaches description/explanation.
��
Combine them.
��
Gesture/scene replaces words - they’re real
— event perception short-circuits language.
Effects are perceived while causes are conceived.
Cinema needs:
Episodes, unlike stories, are accounts of events unencumbered by causality, certainty, or intention:
Events are consensus experience.