PROJ03: Cube Sort

Members: Annie Zheng, Stephanie Claudino Daffara, Lian Song, Soohyun Cho

Designing for Emerging Technologies, Fall 2019

Project Description

Our project explores communication as a method of generating mutual perception -- when presented with a common goal, two individuals are often unable to achieve that goal because they have different perspectives as to how the goal should be achieved. In order to achieve the goal, the individuals must communicate their personal perspectives to align their strategies. We try to simulate this situation in a simple multiplayer cube sorting activity in which two players spawn in the same space, but see different colored cubes. The players must realize that the score will only go up once the cube gets placed into the bin that is the resulting color mixing the two different colors they see.

Table of Contents

Project Description        1

Table of Contents        2

01 Introduction        3

02 Research        3

Improper distance        3

Nash equilibrium        4

Video game inspiration        5

03 Ideation        5

Multiplayer Experience        5

Initial Idea: Whack-a-Mole        6

Switching Gears: Towards More Abstraction        7

Design Goals        7

Why VR?        8

04 Game Design + Mechanics        9

Designing the Environment        9

The Game Mechanics        10

Interactions        11

05 The Code        12

06 Future Directions        12

APPENDIX        14

Getting Started        14

Links        14

01 Introduction

Cube Sort is a multiplayer VR game that encourages the shifting of perceptions through communication. Two players must work together to sort all the colored cubes into the corresponding bins. However, they are unable to accomplish this goal unless they communicate and cooperate because the players see the same cube as having different colors. The overarching concept behind the game elicits the frustration that two people experience when they have a common goal in mind but cannot achieve that goal unless they collaborate. We aim to change a player’s perspectives through communication; although they cannot visualize the colors that the other player sees, the players’ experience a change in perspective the moment they realize that the success of the game hinges on their ability to communicate their personal perspective.

VR is powerful in that it allows users to experience an entirely new perspective. In this case, we chose to limit the user’s perspective to the one they are able to see through the VR goggles, and come to achieve an understanding for the other player’s perspective through verbal communication. In other words, both users come to accept that they will never see what the other person sees but they can still work together to achieve the end goal. Furthermore, VR has strong interactive capabilities that allow for multiplayer experiences.

02 Research

In approaching this project, we wanted to ensure that what we ultimately created would engage productively with the game design and the potential of virtual reality. In this section, we’ll provide an overview of sources that influenced our work.

Improper distance

In Kate Nash’s  study of virtual reality as a tool for humanitarian journalism, she warns of “improper distance”, when “the other becomes ‘indistinguishable from ourselves’”.^[1] The immersive nature of virtual reality grants its great power in allowing ourselves to see from another’s perspective — but with this power comes a risk. For this reason, Nash argues that the “empathy” which is generated from virtual reality witnessing is not necessarily a proper moral response — that is, Nash states that VR experiences may ultimately “obscure the distance between the spectator and other and, in doing so, fail to provide sufficient resources to allow the user to judge the predicament of the other”.^[2]

When considering the goal of our project — namely, to shift someone’s perspective — we took Nash’s warning very seriously. Any VR experienced designed to shift perspective must be very intentionally designed, and we wanted to ensure that our project would not replicate any of the risks that Nash outlined. For this reason, we are committed to creating a multi-player experience that provokes  a shift in perspective as an act that emerged from cooperation, rather than from simple witnessing — in other words, an act that affirmed the difference of two people, while ultimately opening the possibility of a mutual, co-constituted perception.

Nash equilibrium

Another key idea that emerged from our research was the Nash equilibrium: a simple but fundamental idea which suggests that through cooperation, two players might be able to maximize the outcome at a system level — potentially at the loss of their own, individual outcomes. The Nash equilibrium helped to substantiate our idea of a “mutual, co-constituted perception,” by showing us how we might be able to create perception at a supra-individual level — in this case, through the design of a multiplayer game mechanism. Thus, the Nash equilibrium can be seen in the very core of our game — the only way to succeed is to meet in the middle, through communication and cooperation.

Video game inspiration

With this idea of teamwork in place, we wanted to better understand how other, existing VR games approached cooperation. One title in particular stood out: Keep Talking and No One Explodes. In this VR game experience, one person wears a VR headset while their teammates are equipped with instructions that will help them diffuse a virtual bomb. The only way to overcome this challenge is by working together, and this experience is a great way to get people communicating in order to achieve a common goal. This game was especially interesting to us in the way it forced a team to resolve information asymmetrically — and it would deliberately create moments of confusion and tension between teammates.

In contrast with this, we imagined an experience that would be a bit more calm

03 Ideation

Multiplayer Experience

Many VR games are single player games so people are unable to co-experience the same VR world simultaneously. For this reason early on in our ideation, we shifted our focus to multiplayer experiences.

Unlike multiplayer interactions on a computer or through a screen, VR allows this interaction to feel more real and immediate. VR puts users in the same virtual space and creates an illusion that they are sharing the same physical experience. We used the ability to create shared experiences to our advantage since our goal is to build connections and foster empathy between the players.

Initial Idea: Whack-a-Mole

We initially explored the idea of designing a whack-a-mole game that would cause the players to gain empathy for the mole by swapping roles throughout the game.

1. In the first round, one player would assume the role of the whacker while the other player assumes the role of the mole. The whacker would try to hit as many moles as possible. The player taking on the role of the mole must attain food for her hungry mole babies; the whacker does not know this. The mole succeeds if she reaches the food before she gets hit by the whacker.
2. In the second round, the players swap roles. The previous whacker is now the mole and is able to experience the mole’s frantic scramble for food. Furthermore, the new mole is able to see the cute mole babies that must be fed.
3. In the third round, the players swap roles one more time. Ideally, the whacker would no longer try to hit the moles after having experienced the mole’s perspective. If the whacker instead chooses to assist the mole and help her attain food, the game would give both players a higher score.

After much discussion, we ended up abandoning this idea for various reasons:

1. In the case that the whacker does end up assisting the mole, there is no easy way to distinguish whether players truly gained empathy for the moles by round three or if they simply discovered how to win more points in the game.
2. To create a compelling story, the visuals of the game must be of a really high quality. Given the time limitations, we were not confident that we could create such compelling visuals. For example, we would need to design adorable moles that would make the players feel an emotional connection.
3. Whack-a-mole is a classic game that almost everyone has seen before. It is a natural for players to want to whack the moles since that is the point of the original game. Trying to measure empathy with such a game is difficult; the players might have some level of empathy to the moles, but at the end of the day the game is still a game.

Switching Gears: Towards More Abstraction

With Whack-a-Mole, much is dependent on how the player perceives visual elements of the game so an actual change in perspective is difficult to achieve. Furthermore, the elements in Whack-a-mole are nuanced since they belong to a well-known game. We wanted to design a new game that would focus on the interactions between the players. We didn’t want nuances in the visual elements to distract from the interactions so we chose simple, abstract elements for our game. Geometric shapes and colors were a good starting point.

Design Goals

VR allows users to perceive something that they might not be able to perceive physically in reality. However, we wanted to explore the aspect of perception that results from communication. In this case, we explored the situations in which one person is unable to see what the other person is seeing.

Emphasis is placed on the discovery aspect of the game. The player discovers that they cannot achieve their intended goals without the help of the other player. The shift in perspective occurs here when the player realizes that what they see is not the entire “truth” and that even if they cannot see the rest of the “truth”, he or she must trust what the other player is communicating.

Often when individuals are presented with a common goal, they are unable to achieve that shared goal because each individual has a different perspective. To align their strategies, individuals must communicate their personal perspective, one that is not accessible to the other. This is the big idea that we want our users to resonate with after going through our designed VR experience.

In a society of growing divisiveness and “echo chambers”, individuals rarely acknowledge that what they know as the truth may not be the truth for everyone else. Although individuals might not fully understand another’s point of view, they can utilize communication as a mode to build a better understanding for the other.

Why VR?

During our ideation phase, we also discussed how we could design our VR experience to fully utilize the affordances offered by virtual reality. In VR the user can’t cheat by looking directly at the other player’s monitor, nor can they screenshot what they see to send each other. In our game the user’s only method of sharing what they see is by describing it vocally to their teammate. VR also provides more presence than a tiny avatar on a monitor screen. Sometimes someone’s body language, or expressions (which is very close to being a reality in  VR ^[3]) can lead to further understanding that you just can’t get through a computer screen and internal game messaging systems. This sense of presence facilitates collaboration, which in term promotes empathy ^[4].

04 Game Design + Mechanics

Designing the Environment

We wanted the environment to be simple yet impactful, so we decided to put in as little assets as possible, and kept the score UI integrated directly into the scene. The bins are incorporated into the floor as holes, and they glow with their respective colors. There is nothing else in the scene other than 4 cubes and the player’s avatar once the game starts.

The Game Mechanics

Each player is spawned in the same scene and see the same bins and the same scoreboard, and the same cubes. The only difference is that the cubes are colored differently between each player. For example, a cube will appear yellow to player 1, but teal to player 2. They may initially attempt to score by placing the cube into the colors they see, to no avail. In order to score a point, the example cube must be placed in the green bin, which is a mix of the two colors (yellow and blue). Below is a table of the possible color combinations of how a single cube will appear to the players, and the winning bin for the cube of that color combination.

The idea behind this grid is that we want each player to have a set of 2 colors, with at least 2 cubes of each color they see. This way, for example, player 1 cannot memorize that yellow cubes go in the green bin, because only one of the yellow cubes actually goes in the green bin, while the other goes in the orange. This way we minimized cheating the game and being able to achieve the task without communication amongst players. Getting a set of colors with the above constraints was very difficult since the primary colors are a set of 3, and we have 2 players. So I had to use secondary and ternary colors (Teal and Purple) as two of the colors the players see.

One consideration of these game mechanics is that we expect the player to know how to mix colors in an RGB space . For example, Purple and Teal might create Blue in RGB space, but this would be impossible in the color space of a painter trying to mix physical colors. Since Blue is a primary color, it cannot be created from mixing any other color. This consideration Could lead to future iterations of this project where we don’t depend on color anymore and maybe choose a different category for combinations such as numbers or shapes.

Interactions

Each player has access to all of the cubes and the scoreboard. However, as mentioned before, the same cubes display a different color for each player. This fact is something that is not known to the players at the start of the game; instead, it is intended for the players to assume that they both see the exact same colors.

Players will attempt to place their cubes into the corresponding color bin, but will eventually realize that they are not gaining any points on the scoreboard. Furthermore, since the players see different colors, one player might see that the other player attempts to place a cube into a mismatched bin. In other words, what appears to be correctly matched for one player will appear wrong to the other. Soon enough they should realize that their approach is flawed. This is when we intend for the players to verbally communicate with one another to figure out the tricks behind the game.

When the player gets a cube in the correct bin, the bin lights up white and has a “coin”^[5] type sound to indicate that the points have gone up. If the player puts a cube in an incorrect bin, then the bin’s light goes dark, and there is an “error”^[6] sound, indicating that the action was incorrect. The game ends when all 4 cubes have been placed in the correct 4 bins.

05 The Code

This project was built using Unity 2019.2.10f1 and used the oculus integration asset, and the Normcore^[7] asset. The oculus asset was only used to sync up the headset and controllers to our virtual avatar’s head and hands. Normcore was used to create a multiplayer experience. Normcore provides an API and server that syncs up models in real time.

Although we were using Normcore’s SDK to create a multiplayer game there were a couple of complications we had to work around. Normcore has very little documentation so a lot of time was spent reading through their SDK’s code and trying to understand what functions were calling what when. For example, getting the avatar to sync up with the oculus headset and use the oculus locomotion was very hard to get working.

06 Future Directions

This game was a great proof of concept and could be used as the skeleton for many collaboration games that depend on a certain amount of empathy amongst players in order for them to give up their individual benefits and work towards the collaborative goal.

As mentioned before, we would probably use something less biased than color as the differentiated appearance quality. A good example of this could be shapes or numbers. Another interesting enhancement to our game could be rewarding players with something beyond just points and higher scores. Rewards could grow, as the depth of collaboration grows. Maybe the task should be impossibly achieved on your own (which right now could still be achieved through trial and error).

Also this game made sorting a fun activity in general. So this type of game mechanism could be used to teach children about sorting shapes and categorizing objects. The benefit of this being in VR versus a physical board game is that we could display objects in real life size, and really immerse the child in the game play. Maybe they have to sort animals and the beetle, the fish, and elephants are life size! Another interesting application around this idea is a math game. Children could visualize quantity and through collaboration (with one child’s number 1, and another child’s number 3) succeed in making a number 4! This could be fun and teach children emotional skills beyond what a math class generally teaches.

APPENDIX

Getting Started

Welcome to Cube Sort! This is a game designed for two players, to explore how empathy can be generated through collaboration. Let’s get started!

Once both players enter the game, you’ll find yourself in a simple, shared room: with just a set of bins and a few cubes. You’ll have access to a few basic controls — each player is able to move around the game room by using the controller joysticks; and each player can pick up the cubes in the room  by using the grip buttons. The goal of the game is simple: place the cube in the correctly colored bin in order to earn points — but be careful, the colors might not match up easily as they seem! It is up to the two of you to work with each other, in order to understand how the colors you both see might come together.

Links