| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | AB | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Task | Task Criticality | Where | Description of Issue | Impact | P1 | P2 | P3 | P4 | P5 | Frequency | Severity | Interviewees | Impact | Task Criticality | |||||||||||||
2 | Onboarding | Onboarding: Create a brain and name it after yourself | 4 | Startup Screen | User is confused about whether the green color of the "connect" button indicates the robot is successfully connected. | 3 | 1 | 1 | 1 | 1 | 1 | 1.00 | 12.00 | P1: Will | 5 (blocker) | 4 Essential Task | ||||||||||||
3 | 4 | Startup Screen | User clicks into the tab and was confused about all the different brain names | 1 | 1 | 0.20 | 0.80 | P2: Sean | 3 (major) | 3 | ||||||||||||||||||
4 | Onboarding: Start the Brain | 4 | Startup Screen | User did not save the brain. | 5 | 1 | 1 | 0.40 | 8.00 | P3: Yiyang | 2 (minor) | 2 | ||||||||||||||||
5 | Scenario 1 | Task 1: Add 1 neurons and set it to "highly active" | 4 | Run Time Screen | User did not realize that they need to go to "Design Mode" to make changes to the brain. | 3 | 1 | 1 | 1 | 1 | 0.80 | 9.60 | P4: Julie | 1(suggestion) | 1 Not Essential At All | |||||||||||||
6 | 4 | Design Mode | User is confused about what do the numbers after "A," "B," "C," "D," and "ID" mean. | 1.5 | 1 | 1 | 0.40 | 2.40 | P5: MJ | |||||||||||||||||||
7 | 4 | Design Mode | User is confused after clicking on "add a neuron", because nothing happens | 2 | 1 | 0.20 | 1.60 | |||||||||||||||||||||
8 | Task 2: Add 4 new neurons and set each neuron to "quiet" | 3 | Design Mode | User thought that they had to click "Add Many Neurons" if they wanted to add more than one neuron, not realizing that this button allowed them to upload a new brain. This required user to start new brain from scratch. | 4 | 1 | 1 | 1 | 1 | 0.80 | 9.60 | |||||||||||||||||
9 | 3 | Design Mode | User mentioned that they skipped the buttons because they didn’t know what it meant. Button labeling issue | 2 | 1 | 0.20 | 1.20 | Theme: Issue with button labels do not fit user's mental model. | ||||||||||||||||||||
10 | 3 | Design Mode | User tried to change numbers while adding a new neurons, they wanted to see how changing numbers will affect neurons. | 1 | 1 | 0.20 | 0.60 | Theme: No way to undo work, cancelling, giving user a chance to fix mistakes | ||||||||||||||||||||
11 | 3 | Design Mode | User tried to delete multiple neurons at once but failed to do so. | 2 | 1 | 0.20 | 1.20 | Learnability: Decent | ||||||||||||||||||||
12 | 3 | Design Mode | After clicking on "Add Many Neurons," a drop-down menu with a list of terminologies (Brain names). The user is confused about these terminologies and unsure what these terminologies refer to. | 1 | 1 | 1 | 0.40 | 1.20 | ||||||||||||||||||||
13 | 3 | Design Mode | User tried to undo the last action performed (uploaded a existing brain) but failed to do so. | 5 | 1 | 1 | 0.40 | 6.00 | ||||||||||||||||||||
14 | Task 3: Extend an excitatory axon from the presynaptic highly active neuron to each of the quiet neurons. | 4 | Design Mode | It is too easy for user to add a new neuron by accidentally clicking on screen. This requires user to constantly delete accidental neurons. | 3 | 1 | 1 | 1 | 1 | 1 | 1.00 | 12.00 | ||||||||||||||||
15 | 4 | Design Mode | User had difficulty remembering which neuron is "active" vs "quiet." This required user to either delete/create new neurons, or rely on memory to remember which neuron is which, or reset each neuron's type. | 4 | 1 | 1 | 1 | 1 | 1 | 1.00 | 16.00 | |||||||||||||||||
16 | 4 | Design Mode | User did not know which direction the neural signal was flowing (postsynaptic to presynaptic or presynaptic to postsynaptic). This required user to delete axons and recreate the axon extension. | 5 | 1 | 1 | 1 | 1 | 0.80 | 16.00 | ||||||||||||||||||
17 | 4 | Design Mode | User tried to add neurons on the sensors (eyes, wheels) | 2 | 1 | 1 | 0.40 | 3.20 | ||||||||||||||||||||
18 | 4 | Design Mode | User was confused about what the number 100 indicates when creating an axon. | 2 | 1 | 0.20 | 1.60 | |||||||||||||||||||||
19 | 4 | Design Mode | Four antennas show around the neuron after clicking on the "Extend an Axon" button. User thought he should select one of the antennas first to extend an axon to another neuron. | 1 | 1 | 0.20 | 0.80 | |||||||||||||||||||||
20 | 4 | Design Mode | User mentioned there is no easy way for them to cancel the selection of a neuron. | 3 | 1 | 1 | 1 | 0.60 | 7.20 | |||||||||||||||||||
21 | 4 | Design Mode | User complains that adding axons for 4 neurons is simply repetition work. User expect to multi-extend axons instead | 2 | 1 | 0.20 | 1.60 | |||||||||||||||||||||
22 | Task 4: Increase each axon's weight (mV) voltage by increments of 10 starting at 0 to 30. | 3 | Design Mode | User thought that "Edit Properties" meant they could change the axon's weight, not realizing that they changed the neuron. | 3 | 1 | 1 | 1 | 0.60 | 5.40 | ||||||||||||||||||
23 | 3 | Design Mode | User clicked "Run Script" and could not find the "Cancel" button to go back to the main menu. | 3 | 1 | 0.20 | 1.80 | |||||||||||||||||||||
24 | 3 | Design Mode | User did not realize that they had to go to "Extend an Axon" then RECREATE the axon extension to edit the Axon Weight. | 5 | 1 | 1 | 1 | 1 | 1 | 1.00 | 15.00 | |||||||||||||||||
25 | 3 | Design Mode | User was confused about what an axon's weight means. | 1 | 1 | 1 | 0.40 | 1.20 | ||||||||||||||||||||
26 | 3 | Design Mode | Not being able to delete an axon: User accidentally added an extra axon while trying to edit the weight of other axons and had hard time finding a way to delete the extra axon before realizing that they need to set the value to 0 to delete. | 5 | 1 | 1 | 1 | 1 | 0.80 | 12.00 | ||||||||||||||||||
27 | 3 | Design Mode | Unable to undo the step of clicking the button "extending an axon". In order to undo this step, user need to finish creating the axon first then delete the axon. | 3 | 1 | 1 | 0.40 | 3.60 | ||||||||||||||||||||
28 | 3 | Design Mode | User thought it was weird to have nothing when the axon weight is 0, even though the user thought it kind of makes sense. | 1 | 1 | 0.20 | 0.60 | |||||||||||||||||||||
29 | Task 5: Go to the “Run Time” Dashboard and observe the patterns of the neurons. | 3 | Design Mode | Did not immediately realize the difference between "Runtime" and "Design Mode" and did not realize that user had to go to "Runtime" to see a reaction | 3 | 1 | 1 | 0.40 | 3.60 | |||||||||||||||||||
30 | 3 | Design Mode | User expected some feedback after clicking on the “save the brain” button to indicate the user had successfully saved the brain. | 2 | 1 | 1 | 0.40 | 2.40 | ||||||||||||||||||||
31 | 3 | Run Time Screen | Confused by what a "blinking" neuron means: Does it mean, "error message?" or that the neuron is "activated." | 1 | 1 | 0.20 | 0.60 | |||||||||||||||||||||
32 | Task 6: LEARNING OBJECTIVE: What is the minimum synapse weight to trigger spiking of the quiet neuron? | 3 | Run Time Screen | User did not notice the Neuron plot graph at the bottom of run time | 2 | 1 | 1 | 0.40 | 2.40 | |||||||||||||||||||
33 | Usability Task | Task 1: Can you change one of the highly active neurons to quiet? | 2 | - | ||||||||||||||||||||||||
34 | 2 | Design Mode | User tried to drag the neuron in order to move it before realizing that it doesn't work. | 3 | 1 | 1 | 0.40 | 2.40 | ||||||||||||||||||||
35 | Task 2: Can you move the active neuron to the right and group the quiet neurons to the left side of the brain? | - | ||||||||||||||||||||||||||
36 | Task 3: Can you delete a neuron? | 4 | - | |||||||||||||||||||||||||
37 | Task 4: Extra challenge: Can you delete an axon? | 4 | - | |||||||||||||||||||||||||
38 | Task 5: Can you save the brain? | 4 | - | |||||||||||||||||||||||||
39 | Task 6: [Once they save the brain, take them to the main screen] Can you upload the brain that you just saved? | 4 | Run Time | Did not realize that "Stop" would take user to onboarding screen. | 3 | 1 | 1 | 1 | 0.60 | 7.20 | ||||||||||||||||||
40 | 4 | Design Mode | User was confused about wheter or not they shoud save the brain before returning to the Run Time | 2 | 1 | 0.20 | 1.60 | |||||||||||||||||||||
41 | Scenario 2: Make the robot move forward when it sees red. | Task 1: Make a new brain | 4 | - | ||||||||||||||||||||||||
42 | Task 2: Place one neuron on the left side and another neuron on the right side of the brain. | 3 | - | |||||||||||||||||||||||||
43 | Task 3: Extend an excitatory axon from the left eye to the right neuron. Extend another excitatory axon from the right eye to the left neuron. | 4 | - | |||||||||||||||||||||||||
44 | 4 | Design Mode | Inconsistency of flow and menu when extending an axon from the sensory nueron and from the nueron inside the brain. | 3 | 1 | 0.20 | 2.40 | |||||||||||||||||||||
45 | 4 | Design Mode | User is confused about what "red-left" and "red-right" mean. User wonders whether he should select "red-left" when extending axon from the left eye instead of selecting "red." | 3 | 1 | 1 | 1 | 0.60 | 7.20 | |||||||||||||||||||
46 | Task 4: Extend an excitatory axon from the left neuron to the left motor and set the strength to 15. | 4 | Design Mode | User is confused about how the brain on the interface map with the robot.(ex: User is confused about which wheel on the platform matches to which wheel on the robot.) | 3 | 1 | 0.20 | 2.40 | ||||||||||||||||||||
47 | 4 | Design Mode | User was initially confused about what motor means. Proceeded to click wheels guessing that they can be motors. Because the user couldn't click on the motor, they were still confused where the motor is. | 2 | 1 | 0.20 | 1.60 | |||||||||||||||||||||
48 | 4 | Design Mode | User tries "attach script" thinking that will create a connection between neuron and the motor. It didn't work and the user was frustrated. | 3 | 1 | 0.20 | 2.40 | |||||||||||||||||||||
49 | Task 5: Test the brain by making the robot move forward. | 3 | Runtime Mode | User had hard time finding the eyes/camera of the robot. | 2 | 1 | 0.20 | 1.20 | ||||||||||||||||||||
50 | Scenario 2: Make the robot move backwards if it runs into an object | Task 6: Add a third neuron, and make it quiet. | 4 | - | ||||||||||||||||||||||||
51 | Task 7: Extend an axon from the distance sensor to the newly created third neuron and set the distance to medium. | 3 | Design Mode | User is confused about which neuron on the interface refers to the distance sensor. It takes the user some guessing to figure out which is the distance sensor. | 3 | 1 | 1 | 0.40 | 3.60 | |||||||||||||||||||
52 | Task 8: Extend an axon from the third neuron to the motor neuron. | 4 | User did not realize he had to extend the axons to both sides of the wheels. | 1 | 1 | 0.20 | 0.80 | |||||||||||||||||||||
53 | Task 9: Test the brain by making the robot move backwards. | 4 | 0.00 | 0.00 | ||||||||||||||||||||||||
54 | Scenario 2: Make the robot make a sound when it moves backwards | Task 10: Extend an axon from the third neuron to the speaker neuron. | 4 | Design Mode | Thought that the microphone was the speaker, so accidentally connected the neuron to the microphone. | 2 | 1 | 0.20 | 1.60 | |||||||||||||||||||
55 | Task 11: Run the Brain | 4 | - | |||||||||||||||||||||||||
56 | Task 12: Put an object in front of the robot again, does the robot make a sound when it detects the object in front of it and moves backwards? | 4 | - | |||||||||||||||||||||||||
57 | Extra Credit: | 4 | 0.00 | 0.00 | ||||||||||||||||||||||||
58 | ||||||||||||||||||||||||||||
59 | ||||||||||||||||||||||||||||
60 | ||||||||||||||||||||||||||||
61 | ||||||||||||||||||||||||||||
62 | ||||||||||||||||||||||||||||
63 | ||||||||||||||||||||||||||||
64 | ||||||||||||||||||||||||||||
65 | ||||||||||||||||||||||||||||
66 | ||||||||||||||||||||||||||||
67 | ||||||||||||||||||||||||||||
68 | ||||||||||||||||||||||||||||
69 | ||||||||||||||||||||||||||||
70 | ||||||||||||||||||||||||||||
71 | ||||||||||||||||||||||||||||
72 | ||||||||||||||||||||||||||||
73 | ||||||||||||||||||||||||||||
74 | ||||||||||||||||||||||||||||
75 | ||||||||||||||||||||||||||||
76 | ||||||||||||||||||||||||||||
77 | ||||||||||||||||||||||||||||
78 | ||||||||||||||||||||||||||||
79 | ||||||||||||||||||||||||||||
80 | ||||||||||||||||||||||||||||
81 | ||||||||||||||||||||||||||||
82 | ||||||||||||||||||||||||||||
83 | ||||||||||||||||||||||||||||
84 | ||||||||||||||||||||||||||||
85 | ||||||||||||||||||||||||||||
86 | ||||||||||||||||||||||||||||
87 | ||||||||||||||||||||||||||||
88 | ||||||||||||||||||||||||||||
89 | ||||||||||||||||||||||||||||
90 | ||||||||||||||||||||||||||||
91 | ||||||||||||||||||||||||||||
92 | ||||||||||||||||||||||||||||
93 | ||||||||||||||||||||||||||||
94 | ||||||||||||||||||||||||||||
95 | ||||||||||||||||||||||||||||
96 | ||||||||||||||||||||||||||||
97 | ||||||||||||||||||||||||||||
98 | ||||||||||||||||||||||||||||
99 | ||||||||||||||||||||||||||||
100 |