1 of 16

Yiou + Alice

The Insomniac Double

Mobilized surgically by the naturally inhibited motor cortex, the dreaming cat was found to be more or less stalking preys. (Michel Jouvet) What would you do if you set yourself loose during your dream?

Neurogaming | Art | Trans-Consciousness Game | AR/VR

  • The Insomniac Double is a trans-consciousness game where the animated character's body movement is controlled by the signals from the dreamer’s motor cortex.
  • Uncover the unknown repertoire of oneiric behavior of the dreaming individual and use it to puppeteer their fictional double.
  • Through avatars, the dreaming player can interact with the waking player.

Day and dream: across different states of consciousness.

The dreamer puppeteers the animated character as their double.

Drawings & sculpture of his dreams, by Thomas Lyon Mills

If you have experience in:

- BCI Research/Brain Signal Collection

- EEG Data Analysis & Processing

- Unity or Unreal

2 of 16

Consciousness is Tied to Dynamic Stability

Stable system:

more instability

less instability

Unstable/chaotic system:

How similar is dreaming to waking consciousness?

3 of 16

Audiovisual cue integration

no sound continuous motion

with sound small discontinuity

mandatory cue integration occurs

combined percept is different from what we would perceive from only vision

4 of 16

Audiovisual cue integration

alternative explanation for sound can also break cue integration

sound is attributed to second cube

cue integration is sensitive to reasoning and physical constraints

Does this change during dreaming?

Is integration less constrained during sleeping?

temporal mismatch breaks cue integration

5 of 16

Stability

Loosened crossmodal perception

Instability

6 of 16

Stability

Loosened crossmodal perception

Instability

Constrained crossmodal perception

Contact:

dlatorre@mit.edu

eisenaj@mit.edu

7 of 16

Vision for the project

A human-consciousness-machine interaction system: conversation between machine and conscious. An art project envisioning machine mediated dreaming for augmented creativity. By layering up the idea from both human dream and machine generated text, this system helps make the initial ideas become divergent. For visualization part we will use CLIP diffusion to generate image based on text. Finally, we build up a VR dream world that is ‘dreamed’ by human and machine together.

Steps you will take to develop the experiments

Build the dream incubation system that

  • [HUMAN] takes user’s input (target ideation keywords) as the first step.
  • User takes a nap and dreams (audio prompt with input keywords)
  • [CONSCIOUS] user explain the dream
  • [MACHINE] machine come up with ideas based on dream explanation
  • the ideas becomes the next audio prompt for user’s dream
  • VR experience development in Unity to actualize users dream and ideation

News in the form of expected outputs/outcomes from the experiment

Collaborative dream/ideation between human and machine

Divergent & augmented ideation based on dream & ML

Contribution which contextualizes your work relative to the field

#Dream–ML-Generated Art #AugmentedCreativity #SleepOnset

Human-Consciousness-Machine Interaction for Augmented Creativity

Marc + Aria

8 of 16

Visual Impairment and Circadian Rhythm Disorders

Boyu + Yujie

yujiew@mit.edu

boyuz@media.mit.edu

Non-24-Hour Sleep–Wake Disorder

Many people who are blind or visually impaired have difficulty aligning their sleep-wake cycle with a 24-hour day. Currently, about 1 million Americans are legally blind (with corrected vision of 20/200 or worse) – a figure that may double by 2050. Among those, approximately 10% are totally blind, with no light perception at all. And as many as 70% report having trouble sleeping at night as well as trouble staying awake and functional during waking hours.

9 of 16

Boyu + Yujie

yujiew@mit.edu

boyuz@media.mit.edu

10 of 16

For the final project, we plan to work on a headset designed specifically to improve sleep quality during long flights, helping people cope with jet lag. We have attached the early design of the headset that utilizes light and other ambient factors to regulate sleep on flights, protecting the circadian cycle of passengers.

We have another potential project: how to design an office nap pillow/ sleep mask that maximizes sleeping quality during daytime nap? Some people often report tiredness after a nap, especially during working hours. This phenomena is called sleep inertia, and it is shown that sleep inertia after naps will significantly jeopardize working efficiency. Thus, understanding daytime nap and helping people avoid bad naps is a less studied yet crucial topic. Since most people take naps during the day, we figured the most pervasive and seamless intervention is the pillow (specifically, the office nap pillow: https://www.amazon.com/office-nap-pillow/s?k=office+nap+pillow).

Boyu + Yujie

yujiew@mit.edu

boyuz@media.mit.edu

11 of 16

Davide + Hye Jun + Alejandro

“Stream of dream consciousness”

How do we see the unseeable? How do we connect not only our waking selves, but also our dreaming states of consciousness? We are proposing an interactive installation that visualizes dreams in VR. Real-time EEG data of a sleeping participant is transmitted and visualized in real-time in the VR headset of a waking person. The waking person can in turn manipulate the visualization through gestural interactions, which feeds back to the sleeping person through a sensory stimulus. We are also interested in expanding this interactive stream of dream consciousness beyond the “self” and “the other” to incorporate “the collective” as well via projection or other spatial means.

Early visualization exploration

Diagram

Stages

12 of 16

Santiago + Maria

13 of 16

Santiago + Maria

14 of 16

Sleep Temperature

Literature

Current Technology

  • Chili Sleep

Our Extension

  • Real-time temperature changes based off of sleep stage
  • Monitor sleep quality with temperature changes based on changes in physiological state
  • Increase time for sleep onset for people with sleep deprivation

14

Kriti & Shreya

15 of 16

Sleep Posture

Literature

Current Technology

  • Ergonomic beds
  • Prototypes of sleep posture systems
  • Kinect, Cameras

Our Extension

  • Pressure sensors in mattress
  • RF based posture detection (Body Compass)
  • Early intervention of sleep diseases
  • Real-time posture adjustment
  • Monitoring posture vs sleep quality

15

Kriti & Shreya

16 of 16

Lucid Dream Catcher

Kevin Gao

Democratization of Lucid Dream experience

REM sleep and creativity

The studies from Harvard and Germany suggest that NREM napping may improve memory and learning.

Sleep-learning

(Also known as hypnopædia, or hypnopedia) is an attempt to convey information to a sleeping person. I.e. Typical experiments involves sound playing, smells etc.

Lucid Dreaming Treatment for Insomnia

A pilot study shows how lucid dreaming impacts insomnia and mood symptoms.

Challenges

Challenges that made it impractical for interface building. Later-stage REM lucid dreaming requires a full night of sleep, not just a nap like hypnagogia, it’s extremely hard to capture the dreams without fully exit the dreamers from their sleeping stages.

Measurement

Real-time Stage 1 Sleep Detection and Warning System Using a Low-Cost EEG Headset.

Try the MILD and WBTB methods.

Try the dream logs.

Lucid Dreaming can be a truly profound experience, but is often difficult to explain to someone who has never done it.