1 of 10

proof of concept

maggi emi kai

2 of 10

core question/ objective

  • Spirits move around and record video for a specific duration.
  • Spirits work together to take on different filmmaking roles to decide the editing/narrative structure for the video.
    • Editing structure = score
      • which details shot length, camera, etc
    • Narrative structure
      • Narration is generated using neural storyteller
        • neural storyteller uses the first frame of every cut from the edited video to generate narration

​

​

3 of 10

workflow

Attain toys and cameras → in progress; ⅔ cameras and toys collected

Build environment → in progress

Put toy/camera in environment and have them film → tbd

Build Spirits (to edit/narrate) → in progress

Train a neural storyteller → in progress

Run image stills from video through neural storyteller → tbd

Combine text and video to synthesize spirit film perspective → tbd

4 of 10

building environment

5 of 10

6 of 10

7 of 10

roles

  • Directing Spirit → decides which camera
    • random (potentially influenced by sensors)
  • Screenwriting Spirit → narrates
    • Neural Storyteller
  • Production Spirit → decides cuts & shot lengths
    • random (potentially influenced by sensors) and Directing Spirit
  • Camera Operator (3) → film video
  • N toys: actors
  • Maggi → co-producer
  • Emi → co-producer
  • Kai → co-producer

8 of 10

current form

9 of 10

future exploration

  • how can we demonstrate an understanding for neural storyteller in the structure of this project?
  • what do (we want) the toys (to) see and why?
  • how is machine-generated storytelling meaningful to us humans circulating among machine-tools?

10 of 10

final form

documentary video

Cast of characters in the “black box” (enclosed environment)

generated storyboard

Score