Tethered�Game Evaluation Report
Marjorie Ann Cuerdo
HCI 590: Games User Research
DePaul College of Computing and Digital Media
Initial Data Obtained: June 12, 2019
Overview�
(Click on links below to jump to respective sections)
Heuristic Review
We used the Heuristic Evaluation for Playability (HEP) framework by Desurvire, Caplan & Toth in their 2014 journal article “Using Heuristics to Evaluate the Playability of Games”.
These heuristics are categorized into gameplay, game story, mechanics, and usability.
Heuristic Review
We used the Heuristic Evaluation for Playability (HEP) framework by Desurvire, Caplan & Toth in their 2014 journal article “Using Heuristics to Evaluate the Playability of Games”.
These heuristics are categorized into gameplay, game story, mechanics, and usability.
Expert Heuristic Review: Prioritization
Positive issues indicate good practice, where the feature worked as intended.
Critical issues are those that occured on a core task, were not easy to overcome, and occured persistently for the same user. Fix urgently.
High issues met two of the criteria of occuring on a core task, being hard to overcome, or occuring persistently.
Medium issues either occurred on a core task, were hard to overcome, or occured persistently.
Low issues were usability issues that met none of the criteria, but too many of them will impact people’s perception of the experience.
High
Medium
Low
Positive
Critical
Highlights
Heuristic Review:
Visual and Auditory Indicators
Time limit in a body is reinforced by both visually and auditory.
(Through the bar winding down and heartbeat sensor).
Heuristic(s)
Usability 1 - Provide immediate feedback for user actions.
Usability 9 - The interface should be as non-intrusive to the Player as possible.
Positive
Heuristic Review:
User Interface
Minimal UI allows for unobstructed vision. The player can then focus on using the sonar mechanic to search for people in a dim environment.
Heuristic(s)
Usability 3 - The Player experiences the user interface as consistent (in control, color, typography, and dialog design) but the game play is varied.
Usability 9 - The interface should be as non-intrusive to the Player as possible.
Positive
Heuristic Review:
Sound Design
The sound effects of the game match its overall atmosphere.
Heuristic(s)
Usability 7 - Sounds from the game provide meaningful feedback or stir a particular emotion.
Positive
Heuristic Review:
Sound Design
The sound effects of the game match its overall atmosphere.
Heuristic(s)
Usability 7 - Sounds from the game provide meaningful feedback or stir a particular emotion.
Positive
Lowlights
Heuristic Review:
Objectives are not conveyed to the player, and are not listed, leading to potential confusion over where to go and how to accomplish goals in the game.
Violated Heuristic(s)
Mechanics 3 - A player should always be able to identify their score/status and goal in the game.
Recommendation(s)
Consider adding voice lines that reinforce what the player must be doing; additionally, consider creating a list that tracks objectives.
Critical
Heuristic Review:
Players aren’t visually shown what the exit and criteria for success looks like nor their failures. For instance, it isn’t clear to the player where they should be tethering towards nor what victory looks like. When a player reaches the blue capsule room, the game abruptly ends without feedback as to whether it was a success or failure.
Additionally, when a player stays in the sight of a camera too long
the screen visibly turns red but there is no reinforcement such as a
timer or a health bar ticking down to indicate how long the player
has.
Violated Heuristic(s)
Game Play 3 - Provide clear goals, present overriding goal early as well as short-term goals throughout play.
Usability 6 - Players should be given context sensitive help while playing so that they do not get stuck or have to rely on a manual.
Recommendation(s)
Consider adding voice lines that reinforce what the player must be doing; additionally, consider creating a list that tracks objectives.
High
Heuristic Review:
Recommendation(s)
Consider the following suggestions:
(1) Allow the player to see the end of the level by panning from the exit to where the player is.
(2) Ensure that players know how long they can remain under a camera, by accompanying the existing visual red signal with a sound/auditory cue such as an intensifying heartbeat that acts like a timer.
High
Heuristic Review:
There are two core mechanics in this game; (1) tethering and (2) using the sonar to detect living beings. During the tutorial, players learn sonar without indication they can do this multiple times. Additionally, because players can use only the crosshairs to navigate (they turn green), it is possible that players will neglect a core mechanic; i.e., using the sonar.
Violated Heuristic(s)
Usability 6 - Players should be given context sensitive help while playing so that they do not get stuck or have to rely on a manual.
Mechanics 7 - Player should be given controls that are basic enough to learn quickly yet expandable for advanced options.
Recommendation(s)
Consider making levels that require the use of sonar. For example, require players to look at people behind walls and find items that can only be seen with sonar.
Medium
Usability and Playtest
– Research Questions
(a) Can the player understand the consequences of not jumping to a living being after the initial jump? (e.g. there’s a finite time allotted)
Usability and Playtest: Participants
Four male participants who were DePaul University students in the range of 20-24 years old.
Participants were screened for:
Usability and Playtest: Summary
Highlights
● Finding 1: Overall UI is Understandable
● Finding 2: Controls Are Easy to Pick Up
Lowlights
● Finding: Learning How to Play
o Issue 1: Goals and Objectives Are Not Understood
o Issue 2: Tethering Mechanics Are Not Clear
o Issue 3: Navigation is Not Clear
o Issue 4: Failure & Danger Indicators Are Not Noticeable
o Issue 5: Sonar Mechanics Are Not Clear
● Finding: Tutorial Level Could Not Be Completed
● Finding: Reticle/Crosshair is Not Noticeable
Usability and Playtest: Summary
Top Action Items
● Recommendation 1:
Consider introducing an objective tracker for the player to refer back to. Consider additionally ensuring that players have visually seen the objective, for instance, have the target to tether to pass by through a short animation.
● Recommendation 2:
Consider having a functional camera in the tutorial level that turns itself, and the screen red, in order to introduce a mechanic to the player before the first level. Consider adding an “End Screen” that clearly states that a player failed a level, and why they failed a level.
● Recommendation 3:
Consider introducing distinct landmarks, or “living beings”, that players can refer to as they navigate a level. Alternatively, consider allowing players to leave behind temporary breadcrumbs or contrails for the player to refer to past locations.
Highlights
Usability and Playtest:
Controls Are Easy to Pick Up
All participants commended the simplicity of the game, and the ability to focus on gameplay rather than complex controls.
“Pretty simple which is good for me because its just right click and left click. Unless there were other
controls that weren’t explained or I didn’t figure it out.”
“I liked it. I thought it was super simple and intuitive. Once I went through the tutorial I thought of this is
it? It’s just these two buttons? This will be very easy to use.”
Positive
Usability and Playtest:
Overall UI is understandable
Participants found that UI elements that they were confronted with were understandable, such as the lifebar during tethering.
“Yeah my thought process was that there was this bar that was counting down time and I needed to find a person to tether to, with the crosshairs so I was trying to do that before time was going to run out.”
Positive
Lowlights
Usability Test:
Learning how to play
Participants had a difficult time understanding core gameplay mechanics and found it difficult to proceed to a specific goal.
Issue #1: Goals and Objectives Are Not Understood
Participants were not certain what they should be working towards, nor were they certain on what success looked like. Participants would often express frustration on not understanding why they were tethering in the first place.
“I don’t know that I did [what the goals were], I thought initially the dialogue was explaining what I need to do, left is to tether and right is the sonar thing. But I felt like I was in a loop, I didn’t know what I was working towards.”
“No, I had no idea. I was clicking around, using sonar a bunch of times, I also didn’t realize when the tutorial ended or when I was playing the first level. “
“I didn’t know how to fail, was there a number? Was there a number I had to reach? There no objective or goal. It put me in limbo. I felt confused I guess.”
Critical
Usability Test:
Issue #2: Tethering Mechanics Are Not Clear
Participants understood from the tutorial that they could tether to “living beings” but had little grasp on what that actually meant. In practice, this meant that participants would often try to click on everything in the level in an attempt to tether to it. At least half of the participants attempted to tether back to the rabbit. Eventually some participants understood that when the reticle is green, it is tetherable.
“Maybe not, I don’t know if I really tethered people or I was doing one step a bit. I don’t know there was nothing that told me I did anything successfully.”
“It took me a while to figure out what I’m tethering to. It was really quick saying “living things” but then I had to figure out, once the cursor turned green, that was something I could tether to… Yeah, I mean maybe it could’ve been a little more clear that green means a living thing that I could tether to.”
Critical
Usability Test:
Issue #3: Navigation is Not Clear
During the playtest, participants would aimlessly tether from person to person, often remaining in the same room. Participants audibly commented during gameplay that they were not certain what direction they were going towards, and where they came from.
“I didn’t know what to do at that point where I get stuck.” (After tethering).
“I just mess around with it and something happens… I just clicked on, honestly I just clicked on things that I thought needed to be clicked on. Like that silhouette was the only thing, there really wasn’t anything else.”
Critical
Usability Test:
Issue #4: Failure & Danger Indicators Are Not Noticeable
One of the only ways a participant is able to be “killed” or otherwise fail the the level is to be caught in the sights of a camera. Though all participants recognized the camera at some point only one realized the threat it posed to them. When they were “killed” by a camera, participants did not realize that the camera was the reason that the level reset. Additionally, when a level reset, it was subtle enough that participants did not recognize that they failed.
“When the camera saw me, I didn’t realize that was something I had to avoid, so I was confused why did it jump me right back to my original starting location.”
“No, I had no idea [that I died]. I didn’t know my screen was red.”
Critical
Usability Test:
Issue #5: Sonar Mechanics Are Not Clear
The sonar mechanic had within itself three issues: (1) most participants did not realize that they could use the sonar multiple times, and (2) one participant assumed that the sonar was a resource that would drain, and (3) participants are able to bypass the sonar and only use the green reticle.
“I didn’t know if there was going to be some sort of energy drain if used it or some adverse effects if you use it too much.”
“And then I don’t know if I tethered or did something in the tutorial, but all of a sudden, I guess something went wrong. And then I started to get out of the lab. I was like, oh yeah I can use sonar again. Then I forgot about it. And then I started using it as now like yeah I can see the people.”
Critical
Usability Test:
Actionable Items
Tethering:
(1) Considering having NPCs explicitly instruct the player that green targets are tetherable, alternatively NPCs can comment that the creature detects living creatures as green beings.
Navigation:
(1) Consider introducing distinct landmarks, or “living beings”, that players can refer to as they navigate a level. Alternatively, consider allowing players to leave behind temporary breadcrumbs or contrails for the player to refer to past locations.
(2) Consider having the camera pan and zoom away from the objective to the player, so the player has a glimpse of the overall level.
Usability Test:
Actionable Items
Failure and Danger:
(1) Consider having a functional camera in the tutorial level that turns itself, and the screen red, in order to introduce a mechanic to the player before the first level.
(2) Consider increasing the size and contrast of the red hue on the periphery of the player when in danger. Additionally, consider adding an auditory cue to the camera.
(3) Consider adding an “End Screen” that clearly states that a player failed a level, and why they failed a level.
Sonar:
(1) Consider having NPCs comment on the sonar, encouraging the player to use it more frequently, for instance explicitly informing the player they can use the sonar however much they would like.
Usability Test:
Reticle/Crosshair is Not Noticeable
Three out of four participants commented that the retticle was too small, leading to difficulties tethering.
“What’s that? The crosshair? Oh, in general? It’s kind of annoying. It’s like a focus, it’s a speck. I didn’t even know until you pointed it out. Now that I see it and why it’s there. They should make it bigger, make it catch people’s attention, if it was a target or an arrow.”
Recommendation(s)
Consider increasing the size of the reticle for tethering.
Critical
Usability Test:
Tutorial Level Could Not Be Completed
Three out of the four participants could not complete the level, and assumed the first level was the tutorial level. This left participants confused as they did not have an opportunity to practice tethering to the rabbit.
Recommendation(s)
Consider ensuring that the level cannot be completed without tethering to the rabbit.
High
Thank you for your attention!
https://drive.google.com/file/d/1t5IwreciE84ZeKNtKnah0T1cY-O81sp8/view