1 of 45

THE NEURAL CORRELATES OF BODILY SELF-CONSCIOUSNESS IN VIRTUAL WORLDS

Robert O. Duncan & Evan A. Owens

York College (CUNY) and The CUNY Graduate Center

rduncan@york.cuny.edu

2 of 45

“GBL works because students are in a state of flow, immersed in the game world…”

-Some academic

3 of 45

How do you know you exist?

4 of 45

How do you know this �body is yours?

5 of 45

How do you know you are located here, in this room, �in this chair?

6 of 45

Solipsism

There is no way to prove that you’re not in “the matrix”
All of this could be an illusion, and you might be a brain in a jar on a shelf.
We buy into the illusion of reality because sensory feedback is consistent with our experience of what the world is.
This happens via development.
What happens if sensory feedback is altered?

7 of 45

8 of 45

Virtual reality

9 of 45

https://www.statista.com/outlook/amo/ar-vr/worldwide#revenue

Grandviewresearch.com

Over the past decade, virtual and augmented reality (VR/AR) technologically have seen a tremendous rise in market value.
The global virtual reality market size was valued at roughly 22 billion dollars in 2021 and is expected to grow exponentially in the coming years (Grand View Research, 2021).
Although interest and investment into the VR market dates to the early 1990s, recent advances in HMD technology and the recent pandemic acted as catalysts for companies like Facebook, Microsoft, Samsung, Google, HTC, Sony, Oculus and others to invest large amounts of capital into the advancement and utilization of this technology (Grand View Research, 2021).
Despite the recent post-pandemic decline in the VR market, many believe that VR is still in what is known as the “trough of disillusionment,” and the consensus is that VR/AR will continue to grow and ultimately give rise to online spaces that may provide greater user immersion, engagement, and presence (Fenn & Raskino, 2008).

10 of 45

Balasubramaniam et al., 2021

Early virtual reality devices were used to show that presence could be altered via mechanisms associated with vision and sensorimotor perception that affect perception-action feedback loops
Its speculated that presence is associated with aspects of consciousness and embodied cognition
Perception-action feedback loops are a way of linking an organism to the environment.
In particular, cortical areas that are responsible for vision, audition, somatosensation, kinesthesis, and proprioception interact with networks that initiate eye movements, posture correction and balance, and action via motor commands.
And the prefrontal cortex facilitates this interaction by making predictions about the world using top-down knowledge from memory and bottom-up sensations from the outside world.
Because VR provides an arena where perception-action loops can be practiced and manipulated, VR is a useful tool for exploring embodied cognition and BSC.

Corbetta, D., DiMercurio, A., Wiener, R. F., Connell, J. P., & Clark, M. (2018). How perception and action fosters exploration and selection in infant skill acquisition. Advances in child development and behavior, 55, 1-29.

Early exploration into BSC only acknowledged one of its components, presence, and the nature of presence has been debated ever since its inception (Riva et al., 2015). Many have speculated that presence is associated with aspects of consciousness and embodied cognition. Embodied cognition proposes that body representation in space is an integral component of perception and action.

Early virtual reality devices were used to show that presence could be altered via mechanisms associated with vision and sensorimotor perception that affect perception-action feedback loops. Perception-action feedback loops are a way of linking an organism to the environment. They are made possible through several connections in the cortex. In particular, cortical areas that are responsible for vision, audition, somatosensation, kinesthesis, and proprioception interact with networks that initiate eye movements, posture correction and balance, and action via motor commands. And the prefrontal cortex facilitates this interaction by making predictions about the world using top-down knowledge from memory and bottom-up sensations from the outside world.

Perception Action feedback loops are automatic and heavily correlated with movement. Consequently, as a child’s understanding of the world progresses, so does their efficiency at achieving planned goals in the environment. As that child develops, predictions they may make about the world become more refined and accurate. Our experience with the world teaches us that objects have affordances. For example, if you see a basketball, you know exactly what to do with it, regardless of whether you have ever seen that particular one before. This implicit knowledge is the result of perception action feedback loops. When you perceive that new basketball for the first time, previous schemas about how basketball work enable you to carry out goal-oriented behavior in an effective way. Experience with motor movements affects brain regions that plan these movements. For example, an fMRI study showed differences in neural activation in the parietal lobe, premotor cortex, and intraparietal sulcus when dancers watched a ballet movement that they have practiced compared to ballet motions that were never practiced (Calvo-Merino et al., 2005). Because VR provides an arena where perception-action loops can be practiced and manipulated, VR is a useful tool for exploring embodied cognition and BSC.

Balasubramaniam, R., Haegens, S., Jazayeri, M., Merchant, H., Sternad, D., & Song, J. H. (2021). Neural encoding and representation of time for sensorimotor control and learning. Journal of Neuroscience, 41(5), 866-872.

11 of 45

Operational Definitions

Bodily Self-Consciousness (BSC) is the perception of bodily awareness that arises from the integration of neuronal signals in multiple sensory modalities

Embodiment is feeling ownership towards a real or virtual extremity

Presence is the feeling of being located in space

12 of 45

Rubber Hand Illusion

Early studies altered subjective experiences of embodiment using the classic rubber hand illusion (Botvinick & Cohen, 1998; Armel & Ramachandran, 2003)

Armel, K. C., & Ramachandran, V. S. (2003)

Rubber Hand Illusion

Embodiment is feeling ownership towards a real or virtual extremity
Before virtual reality technology was commonly used in scientific research, some early studies altered subjective experiences of embodiment using the classic rubber hand illusion (Botvinick & Cohen, 1998; Armel & Ramachandran, 2003).
For this illusion, one of the participants' hands is hidden from view. A rubber hand is then placed within a plausible location and orientation relative to the occluded hand. Synchronous tactile stimulation applied to the real and rubber hand will cause the participant to feel as if the rubber hand was their own. In some cases, this illusion is so strong that striking the rubber hand with a knife can cause participants to jump out of fear that their actual body is in danger (for a demo, see BBC Two., 2003).
This illusion of embodiment is attributed to conflicts between vision and perception. Although the participants can clearly see the rubber hand is fake, the synchronous tactile feedback provides the brain with seemingly reliable evidence that the rubber hand is their own

13 of 45

Behavioral Presence Studies

Early presence studies altered perceptions of presence by changing the perspective at which participants either viewed their actual body, a virtual avatar, non-corporeal space, or a rubber mannequin (Ehrsson, 2007; Lenggenhager et al., 2007; Lenggenhager et al., 2009).

Lenggenhager et al., 2007

Virtual reality HMDs can alter perception of presence unlike ever before.
Scientific studies have altered perceptions of presence by manipulating the perspective at which people conduct the experiment
Presence Literature

Presence is the feeling of being located in space
Non- Neuroimaging Studies

Blocking the majority of stimuli from the external world, virtual reality HMDs can flood the brain with sensory information, so the user feels as if they are located in a completely new physical space. Researchers have exploited this valuable attribute of VR to further understand presence in a novel and safe way (Aspell et al., 2009; Lenggenhager et al., 2007; Lenggenhager et al., 2009; Slater et al., 2010; Maselli, 2015). Some of the earliest presence studies caused alterations in perceptions of presence by changing the perspective at which participants either viewed their actual body, a virtual avatar, or a rubber mannequin (Ehrsson, 2007; Lenggenhager et al., 2007; Lenggenhager et al., 2009).
As participants were sitting or standing at a particular location, they viewed a 3-D image from a camera that was positioned at a different location in the room. When synchronous tactile stimulation was applied to their actual body and the virtual location, the participants reported feeling displaced to the virtual location, where visual and tactile stimulation were thought to occur (Lenggenhager et al., 2007; Lenggenhager et al., 2009).

As participants sit or stand at a particular location, they traditionally viewed a 3-D image from a camera that is positioned at a different location in the room.

When synchronous tactile stimulation is applied to their actual body and the virtual location, the participants reported feeling displaced to the virtual location, where visual and tactile stimulation were thought to occur (Lenggenhager et al., 2007; Lenggenhager et al., 2009).

14 of 45

Proposed Models of BSC

Coupled Network Models

Guterstam et al., 2015
Park and Blanke 2019
Blanke et al., 2015

Decoupled Network Models

Serino et al., 2013

15 of 45

16 of 45

17 of 45

Guterstam et al. (2015)

Guterstam et al., (2019)

In the fMRI protocol, participants lay in a supine position in the scanner while wearing an MR-compatible headset. The video feed presented an image of the body. The body image could be lying in the scanner (baseline) or at one of three positions (A, B, or C). A and B were first-person views looking back at the scanner bore, but from different corners of the MR room. For position C, the location was identical to B, but the position was perpendicular.

The goal was to measure differences in embodiment, self-location and head direction. Tactile stimulation was administered to the participants chest and visual stimulation was presented via the video feed, synchronously or asynchronously. Either a knife (threat condition) or a spoon (control condition) was drawn over the image of the chest to measure skin conductance responses during two threat conditions. Questionnaires pertaining to embodiment and presence, as well fMRI data were collected to explore the neural correlates underlying these experiences and to assess the interplay and effective connectivity between these two networks.

Guterstam et al. (2015) explored the neural correlates of both embodiment and presence in a fMRI experiment. Their fMRI analyses suggested that activity patterns in the hippocampus, the posterior cingulate cortex, retrosplenial cortex, and intraparietal cortex support the sense of presence. Whereas the sense of embodiment was associated with premotor-intraparietal activity. Subjects sat in a chair wearing an HMD. The participants observed a real-time video feed projected from cameras located two meters behind them. The experimenters then synchronously or asynchronously stroked the participants chest and an area just below the camera, which appeared to be at the location of the chest. Congruent stimulation between their actual body and the location visible in the HMD created the illusion that the participant was located behind their actual body. After acquiring preliminary data, this experiment was adapted for the MR environment. In the fMRI protocol, participants lay in a supine position in the scanner while wearing an MR-compatible headset. The video feed presented an image of the body. The body image could be lying in the scanner (baseline) or at one of three positions (A, B, or C). A and B were first-person views looking back at the scanner bore, but from different corners of the MR room. For position C, the location was identical to B, but the position was perpendicular. The goal was to measure differences in self-location and head direction. Tactile stimulation was administered to the participants chest and visual stimulation was presented via the video feed, synchronously or asynchronously. Either a knife (threat condition) or a spoon (control condition) was drawn over the image of the chest to measure skin conductance responses during two threat conditions. Questionnaires pertaining to embodiment and presence, as well fMRI data were collected to explore the neural correlates underlying these experiences and to assess the interplay and effective connectivity between these two networks.

Results of the post-scan self-location task revealed that the synchronous condition was associated with a strong sense of self-location in the out-of-body positions and a weak sense of self-location inside the scanner. The skin conductance responses evoked by a knife, but not a spoon, threatening the body was significantly higher in the synchronous compared to the asynchronous condition. fMRI analyses of BOLD signal changes in ROIs revealed a main effect of visuo-tactile synchrony across all conditions. The results suggest that the left premotor and bilateral intraparietal cortices were associated with the feeling of ownership during synchronous stimulation conditions. Guterstam et al.’s (2015) findings support the hypothesis that activity in the multisensory premotor and intraparietal cortices are associated with the feeling of ownership of a body seen from a first-person perspective in a real-world environment. A multivoxel pattern analysis was used to measure changes in self-location and head direction. To measure self-location, they compared data acquired for position A and B. Self-location results revealed significant decoding of self-location in the left hippocampus, left posterior cingulate cortex, and right IPS. Furthermore, Guterstam et al. (2015) found decoding of self-location to correlate with activity in the left supramarginal gyrus of the left inferior parietal cortex. Interestingly, the decoding accuracy was positively correlated with the behavioral self-location in the left PCC, right IPS, and left retrosplenial cortex. The perceived head direction was significantly decoded from BOLD activity patterns in the IPS, precuneus, RSC, and PCC.
Finally, Guterstam et al. (2015) investigated the interplay between the neural representations of body ownership and self-location by measuring the illusion-induced changes in effective connectivity. They searched for voxels that displayed increases in connectivity to the PCC for both ownership and self-location related conditions. A psychophysiological interaction (PPI) analysis was performed using the contrast from synchronous and asynchronous conditions across all positions. The results showed that participants with self-location measures (position A versus B) in the left PCC showed proportionally stronger effective connectivity to the left IPS, right RSC, and left hippocampus. These findings suggest that ownership of the stranger’s body viewed from the first-person perspective is associated with an effective connectivity increase between the PCC and the intraparietal-retrosplenial-hippocampal cortices that is intimately linked to the representation of self-location in the PCC.
In conclusion, Guterstam et al. (2015) assert that the premotor-intraparietal activations observed in this study reflect the dynamic integration of synchronous tactile and visual signals. In addition, premotor and intraparietal activation were believed to reflect tactile and proprioceptive signals, resulting in the assimilation of the stranger’s body as the self. Their decoding analyses revealed that patterns of BOLD activity in the posterior parietal cortex, PCC, and hippocampus reflect the perceived spatial location of the bodily self. Based on their findings, Guterstam et al. (2015) propose that the hippocampus supports an allocentric representation of the perceived self-location and is intimately linked to the egocentric representation of self-location in the IPS via intermediate processing in the PCC (FIGURE 2.1). These results suggest that an interconnected set of regions in the hippocampus and the posterior cingulate, retrosplenial, and intraparietal cortices plays an important role in the construction of this fundamental subjective feeling of self. It is important to note that this study uses augmented reality to present actual views of the human body, and this passive experiment does not require action or volition on the part of the subject.

18 of 45

19 of 45

20 of 45

“I can’t be somewhere �unless I am something”

21 of 45

We also assert that embodiment and presence interact hierarchically, meaning that the perception of presence depends on the perception of embodiment. Furthermore, we contend that changing embodiment conditions affects these proposed overlapping areas downstream, but changing presence does not necessarily affect all areas encoding embodiment, even if there is a degree of back propagation.
Consequently, rather than embodiment and presence having separate networks that both feed into the perception of BSC (FIGURE 2.6), or separate networks that feed into two separate manifestations of BSC (FIGURE 2.7), or networks pertaining to embodiment and presence sharing overlapping neural networks via integration areas that then feed into the perception of BSC (FIGURE 2.8), we propose an overlapping, hierarchical network to account for these experiences (FIGURE 2.9).

22 of 45

We predict that frontoparietal networks including the premotor and parietal cortices correspond to the perceptions of embodiment. Whereas temporoparietal networks including the TPJ, hippocampus, supramarginal gyrus, and precuneus correspond with the perception of presence.
Coinciding with coupled models discussed in this paper, we further propose that networks pertaining to embodiment and presence partially overlap. Therefore, regions including the PCC, IPS, insula, and LOC are believed to bind these two perceptions (FIGURE 2.5).
We also assert that embodiment and presence interact hierarchically, meaning that the perception of presence depends on the perception of embodiment. Furthermore, we contend that changing embodiment conditions affects these proposed overlapping areas downstream, but changing presence does not necessarily affect all areas encoding embodiment, even if there is a degree of back propagation.
Consequently, rather than embodiment and presence having separate networks that both feed into the perception of BSC (FIGURE 2.6), or separate networks that feed into two separate manifestations of BSC (FIGURE 2.7), or networks pertaining to embodiment and presence sharing overlapping neural networks via integration areas that then feed into the perception of BSC (FIGURE 2.8), we propose an overlapping, hierarchical network to account for these experiences (FIGURE 2.9).

23 of 45

Methods

Subjects

N = 11

Mean age = 33.8

Apparatus

This experiment was conducted using an HTC Vive Pro head mounted display

The HMD was accompanied by hand-held controllers and other accessories that were used to track the hands in virtual space.

Retrieved from https://www.pinterest.com/pin/195484440059331716/

24 of 45

Procedure

Participants lay in the bore of the fMRI scanner in a supine position with a desk over their lap. They held an inverted trackball at the center of the board before the experiment began.
There were eight runs total for each session, and each of the four conditions was run twice.
Participants placed their virtual hand over a centrally positioned fixation target to start each trial. Then, participants were presented with a red cue in one of four possible locations on the table. This red cue reliably predicted the spatial location of the target 80% of the time on valid- versus invalid-cue conditions. After two seconds, a green target would appear in one of the four possible locations at random.
The participant’s goal was to indicate the correct target on the table by gliding their virtual hand over the correct target. The entire MRI session took approximately one hour.

25 of 45

EMBODIMENT STEP PROCEDURE

Embodiment Step 0

Embodiment Step 9

There were two main conditions for this experiment: Embodiment and Presence. Each condition had 10 different steps (0-9).
Conditions (Images)
Embodiment Step

The tracker coordinates were rotated along the y-axis between 0 and 90 degrees in 10 deg. increments. At the position with the largest step size (i.e., “step 9” = 90 degrees), the vector of the avatar’s moving arm was at a right angle from the veridical direction of motion made by the subject. In other words, at step 9, when the image motion was 90 degrees away from the veridical motion of the arm, the display would move the virtual arm "up" when the subject moved their arm to the "right.”

Presence Step

The position of the camera was rotated between 0 and 90 degrees in 10-degree increments. At the position with the largest step size (i.e., “step 9” = 90 degrees of rotation), subjects viewed their avatar from the left side of the body at a viewing distance and angle from the horizon that was equivalent to that of the unrotated scene.

For both conditions, each step was presented pseudo-randomly using interleaved ascending and descending staircases until every step for that condition was run once. After every step for a condition was complete (e.g., embodiment), the participant would begin the alternate conditions (e.g., presence). The cycle was repeated until the designated number of trials were completed.
Procedure (One image of the HE condition)

Similar to Posner’s classic cueing paradigm (Posner, 1980)
Participants wore the HMD while sitting in a chair in front of a table. In the virtual scene, users were presented with a virtual avatar seated in a chair looking downward at a table from a first-person perspective.
The avatar’s right hand was attached to the in-game positional tracker and inverse kinematics were used so that the corresponding joints in the virtual arm followed the movement of the tracker in real and virtual space.
On the virtual table, participants were presented with a fixation cross placed at the center, and four disks were placed equidistant from the fixation target
Participants placed their virtual hand over the fixation target to begin each trial. When the trial began, a red, spatial cue would signal the eventual appearance of a target.
After two seconds, a green target disk would appear in one of the four possible locations at random. The participant’s goal was to indicate the correct target on the table by gliding their virtual hand over the correct target.
Accuracy and reaction time were recorded for each condition

26 of 45

PRESENCE STEP PROCEDURE

Presence Step 0

Presence Step 9

Procedure (One image of the HE condition)

Similar to Posner’s classic cueing paradigm (Posner, 1980)
Participants wore the HMD while sitting in a chair in front of a table. In the virtual scene, users were presented with a virtual avatar seated in a chair looking downward at a table from a first-person perspective.
The avatar’s right hand was attached to the in-game positional tracker and inverse kinematics were used so that the corresponding joints in the virtual arm followed the movement of the tracker in real and virtual space.
On the virtual table, participants were presented with a fixation cross placed at the center, and four disks were placed equidistant from the fixation target
Participants placed their virtual hand over the fixation target to begin each trial. When the trial began, a red, spatial cue would signal the eventual appearance of a target.
After two seconds, a green target disk would appear in one of the four possible locations at random. The participant’s goal was to indicate the correct target on the table by gliding their virtual hand over the correct target.
Accuracy and reaction time were recorded for each condition

27 of 45

REACTION TIME RESULTS

One -way ANOVA for embodiment reaction times (Image)

Test for multiple comparisons found that the mean reaction time for embodiment step 0 (M = .85 s, SD = .21 s) was significantly different than when the motion vector was rotated 70 degrees (M = 1.63 s, SD = .38 s) (q (9, 100) = -.78, p < .05), 80 degrees (M = 1.73 s, SD = .5 s) (q (9, 100) = -.88, p < .05), and 90 degrees (M = 1.77 s, SD = .39 s) (q (9, 100) = -.92, p < .05)

One -way ANOVA for presence reaction times (Image)

Tukey’s HSD Test for multiple comparisons found that the mean value of step 0 (M = .85, SD = .19 s) was significantly different from step 9 (M = 1.769, SD = 1.45 s), q (9, 100) = -.92, p < .05

One -way ANOVA for embodiment accuracy (Image)

Tukey’s HSD Test for multiple comparisons found that the mean accuracy for embodiment step 0 (M = 100, SD = 0 s) was significantly different than embodiment steps 8 (M = 69.7, SD = 22.38 s) (q (9, 100) = 30.3, p < .05) and 9 (M = 55.85, SD = 26.7 s) (q (9, 100) = 44.1, p < .05)

One -way ANOVA for presence accuracy (Image)

Tukey’s HSD Test for multiple comparisons found that the mean accuracy for step 0 (M = 100, SD = 0) was significantly different than steps 8 (M = 92.1, SD = 10.3) (q (9, 100) =7.93, p < .05) and 9 (M = 91.1, SD = 9.24) (q (9, 100) = 8.93, p < .05)

knife /spoon valid/invalid data? The knife/spoon and cue data collected was for every step, so these calculations are misleading.

28 of 45

ACCURACY RESULTS

One -way ANOVA for embodiment reaction times (Image)

Test for multiple comparisons found that the mean reaction time for embodiment step 0 (M = .85 s, SD = .21 s) was significantly different than when the motion vector was rotated 70 degrees (M = 1.63 s, SD = .38 s) (q (9, 100) = -.78, p < .05), 80 degrees (M = 1.73 s, SD = .5 s) (q (9, 100) = -.88, p < .05), and 90 degrees (M = 1.77 s, SD = .39 s) (q (9, 100) = -.92, p < .05)

One -way ANOVA for presence reaction times (Image)

Tukey’s HSD Test for multiple comparisons found that the mean value of step 0 (M = .85, SD = .19 s) was significantly different from step 9 (M = 1.769, SD = 1.45 s), q (9, 100) = -.92, p < .05

One -way ANOVA for embodiment accuracy (Image)

Tukey’s HSD Test for multiple comparisons found that the mean accuracy for embodiment step 0 (M = 100, SD = 0 s) was significantly different than embodiment steps 8 (M = 69.7, SD = 22.38 s) (q (9, 100) = 30.3, p < .05) and 9 (M = 55.85, SD = 26.7 s) (q (9, 100) = 44.1, p < .05)

One -way ANOVA for presence accuracy (Image)

Tukey’s HSD Test for multiple comparisons found that the mean accuracy for step 0 (M = 100, SD = 0) was significantly different than steps 8 (M = 92.1, SD = 10.3) (q (9, 100) =7.93, p < .05) and 9 (M = 91.1, SD = 9.24) (q (9, 100) = 8.93, p < .05)

knife /spoon valid/invalid data? The knife/spoon and cue data collected was for every step, so these calculations are misleading.

29 of 45

Methods

Subjects

Same as before

Apparatus

The stimulus was similar, but changes were made to optimize the paradigm for use in the MR environment.

Subjects

Participants were 11 right-handed men and women between the ages of 27 and 57 years old with normal or corrected-to-normal acuity. The mean age for participants was 34.6 years old.
Participants were screened using standard fMRI criteria for metal, medications, or health conditions that could harm them or the quality of the data.
Participants were not excluded for sex, race, or gender. Participants were compensated with $60 for completing the fMRI experiment.

Apparatus

Similar to Experiment 4, stimulus presentation and behavioral data acquisition were managed by Unity's game development software on a laptop computer (Alienware, Intel(R) Core(TM) i7-9750H 2.60GHz, 16 GB RAM).
The stimulus was similar to that of Chapter 3, but changes were made to optimize the paradigm for use in the MR environment.
The staircase procedure was no longer needed because this experiment compared conditions known to reliably support or alter perceptions of embodiment and presence (i.e., step 0 vs. step 9). The trial timing was also modified to provide a generous window for participants to respond, while also assuring a minimum number of trials could be complete during each block.
Custom desk was made that participants placed over their lap so that they could make their responses

30 of 45

31 of 45

Images collected using a Siemens 3-Tesla Prisma MRI Scanner
Visual stimuli not presented using a HMD
Responses indicated using an MR-compatible computer mouse

Cambridge Research Systems

The fMRI images were collected using a Siemens 3-Tesla Prisma MRI Scanner at the Advanced Science Research Center in Harlem, New York. Participants lay supine in the scanner bore, and their heads were secured with padding in the “birdcage” head coil. Participants viewed the visual stimuli using an angled mirror that was affixed to the head coil. Each session started with an anatomical scan using a standard T1-weighted gradient echo pulse sequence (MP-RAGE, 1 × 1 × 1 mm resolution). Anatomical scans were used as a reference volume for each participant and to align functional data across each session.
An MR-compatible widescreen display (BOLDscreen 32" LCD, Cambridge Research Systems, UK; 1920 x 1080 pixels, 120 Hz refresh rate, 1400:1 contrast ratio, 24-bit RGB) was used to present stimuli for this experiment.
Although this study was inspired by virtual reality, visual stimuli were not presented using an HMD. Because participants are not able to move their heads in fMRI, there was no need to update the visual image in response to head movement. Dichoptic presentation was also not used because immersion was presumed to be primarily driven by perception-action feedback loops rather than stereopsis. Immersion was further supported by removing all sensory distractions that could potentially compete with the stimulus. Specifically, all light was turned off in the scanner room, and curtains were placed over the monitor and at the entry to the scanner bore to prevent light from penetrating.
Mouse was flipped upside-down so that it could roll smoothly across a desk placed across the participant’s lap. The trackball’s coordinates were tracked with near-zero latency and used to position the avatar’s hand in virtual space.

32 of 45

8 scans total
High-Embodiment example:

LE (20s)

HE

(20s)

4.3 mins

HE (20s)

START

1 Epoch

1 Run

Procedure

Up to eight functional scans were acquired for each participant using a low-bandwidth EPI pulse sequence lasting 260 seconds (TR = 1 second, TE = 30 ms, flip angle = 52°, 39 oblique slices of 3-mm thickness and 3 × 3 mm in-plane resolution, 1638 x 1638 mm FOV).
The blood-oxygen level dependent (BOLD) signal was acquired for each condition using a standard block design. Complementary conditions (e.g., HE vs. LE) were contrasted during each run in alternating half-cycles of 20 s for 6 cycles (excluding the first half-cycle). The order of each condition was counterbalanced in a different run (e.g., LE vs. HE) and each run was repeated, yielding a total of eight runs with four repeats for each condition (e.g., HE).
Similar to Experiment 4 in Chapter 3, users were presented with a first-person perspective of a virtual avatar seated in a chair that was looking downward at a table. Stimulus presentation was designed to accommodate fMRI data acquisition in a block design, where each epoch lasted 40 seconds.
When the experiment began, conditions would alternate between high- versus low-presence or high- versus low-embodiment conditions every 20 seconds for the entirety of the run.
Each run lasted roughly four minutes and 20 seconds (or 6.5 epochs).

33 of 45

EMBODIMENT BEHAVIORAL RESULTS

34 of 45

PRESENCE BEHAVIORAL RESULTS

35 of 45

EMBODIMENT fMRI RESULTS

36 of 45

PRESENCE fMRI RESULTS

TPJ

37 of 45

Control ROI =>

38 of 45

FMRI EMBODIMENT RESULTS

Embodiment ROI	P Value (< .025)
L LOC	.00768*
R LOC	<.001*
L Intraparietal Sulcus	.00043*
R Intraparietal Sulcus	<.001*
L PMv	<.001*
L Anterior IPS	<.001*
L Mid. Frontal Gyrus	.0052*
R Mid. Frontal Gyrus	.0052*
L. Supra. Gyrus	<.001*

Embodiment ROI	P Value (< .025)
L Hippocampus	.045
L Inf. Parietal Lobe	<.001*
L Insula	.0019*
R Insula	.0045*
L Intraparietal Sulcus	.00043*
R Intraparietal Sulcus	<.001*
L Posterior Cingulate	.017*
R Posterior Cingulate	<.066
R PMd	.0207*

Embodiment ROI	P Value (< .025)
L. Area PFt	<.001*
L ACC	<.001*
R ACC	.012*
L Premotor Cortex	<.001*
L Postcentral Gyrus	<.001*
L Anterior Insula	<.001*
R Anterior Insula	.0052*
L Middle Insula	.0021*
R Middle Insula	<.001*

39 of 45

FMRI PRESENCE RESULTS

Presence ROI	P Value (< .025)
L Parietal Operculum	.79
L Postcentral Gyrus	.0064*
R Postcentral Gyrus	.002*
R Intraparietal Sulcus	<.001*
L Anterior Insula	<.001*
R Anterior Insula	<.001*
R LOC	<.001*
L Middle Insula	<.002*
R Middle Insula	<.001*

Presence ROI	P Value (< .025)
L Hippocampus	.363
R Hippocampus	.058
L Intraparietal Sulcus	.00212*
R Intraparietal Sulcus	<.001*
L Posterior Cingulate	.22
R Posterior Cingulate	***�
L Supra Gyrus	<.001*
L Precuneus	.14
R Precuneus	.01238*

Presence ROI	P Value (< .025)
L RSC	.305
R RSC	.62
L TPJ	.027
R TPJ	.00187*
L LOC	<.001*
R LOC	<.001*
R LOC	<.001*
R Mid. Frontal Gyrus	.063
R Sup Prec Sulcus	.354

40 of 45

Effect sizes

The effect size of the embodiment manipulation on presence ROIs (M = .0231; SD = .022) was larger than that for the presence manipulation on embodiment ROIs (M = .0067; SD = .0043), t(11.16) = 2.41, p = .034.
Cohen’s d was computed (d = .94) using the pooled standard deviation (SD = .0176), which is a large effect size for the difference in mean effect sizes for each group.
Thus, the effect of embodiment manipulations on ROIs associated with presence is larger than the effect of presence manipulations on ROIs associated with embodiment.
Manipulations to embodiment affect presence, but not necessarily vice versa.
This data supports a hierarchical model where the perception of presence is predicated upon the perception of embodiment.

41 of 45

How can I use this for the classroom?

We should always be designing for immersion
Design for flow
Scale difficulty to support immersion and flow
Provide short-term (sensory) feedback
Provide long-term (cognitive) feedback
Etc.