UAG August 23 2022 Usability Analysis.xlsx

	B	C	D	E	F	G	H	I	J	K
1	Task	Q No.		P1*(FA-18 Pilot) T1-T3	P2* (HA-60) T4-T5	P3 (Growler)	P4 (Growler)

2			Usability Issues					Notes	Possible Solutions
3	2	3	Did not find the "Summary and Tasking" page right away to upload the ATO. Clicked "create new" then "mission manager plugin"					P3 clicked create as well; P4 - "Summary and Tasking" was his last choice to find ATO	Need to understand why this wasn't as intuitive as we thought - more user feedback on ATO workflows specifically? Perhaps "Summary and Tasking" is not as intuitve or descriptive? Need to understand where users would think ATO/ACO management should go.
4	2	7	User first clicked "more" menu to find unique idenifier. Clicked A/C config option second.						This could be a product of "Unique Id" being a confusing term.
5	2	10	Asked if "generate autoroute" is the same as create route.						This could be an opportunity to improve how things are named or to provide contextual help around autoroutes and manual routes. Potentially we can provide a context help explaining what an autoroute is.
6	2	10	Asks how would user know the autoroute is correct?						See line 5. Context help can explain what is being calculated
7	2	15	After generating the route and viewing it on the map, would like a more zoomed in and centered around the route they just created					P4: zoomed on orbit or on route	Further define zoom levels. Each platform will have different levels that matter to them - user/platform pref?
8	3	17	Clicked "Create New" first, then the route plugin						This could be a result of the user just coming from the route page. In the system there is a way to create a manual route, however creating a manual route from the routes page is hidden.
9	4	24	Clicked on "Overlays" first.						Overlays is being reworked. There is a section for Shapes/Airspaces in the Overlay plugin. So users will be able to create a new shape collection from the Overlay Plugin in the new design
10	4	24	When overlay didn’t respond, clicked on "Create New" in the Navigation. They then went back to try to click on the "Overlays" first.						See line 10
11	4	24	Read the left hand navigation as items within the Event/Mission						If more than one user experiences this problem, might need to look into how we can convey that the left hand navigation is universal (other than Active Plans)
12	4	27	Confused on what moderaters meant by "bolt." But did not have issues navigating to the library to find it.						Could just be confused with the term
13	2	4	Expected ATO to be under "Missions"						Probably solved with training, but maybe we should elicit some feedback on if it should be its own thing under the mission/event
14	2	9	"I think route details has some uncertainty. I'm trying to get to another event. For route, I expected it to be in another section... (add orbit) Type in waypoint or lat/lon with relevant info"
15	2	14	Usually get aeronautical charts and additional views with viewing routes						Perhaps, we can give users the ability to easily turn on these preloaded charted from overlays. And/or utilize user pref for chart management. We will need to do some research and determine what kind of charts each community uses.
16	3	17	I would go to Routes --> Create Route --> Waypoint details --> Autoroute/plot
17	3	18	Would want to verify a/c configuration details before calculating route to make sure the calculations were correct						See line 5. Context help can explain what is being calculated
18	4	26	Didn't like the word commit in shapes - ok or accept would be better						LUXCE needs to standardize primary action buttons.
19
20
21
22			Positives						Analysis
23	1	2	Navigated to the library with no hesitation after receiving the direction to find "Training Event"						Intuitive
24	1	2	Click count:2						All users who participated in task 1 only took two clicks to navigate to the training event - navigation seems to be relatively intuitive
25	2	4	Liked how the "Add ATO" button lets them search for an existing ato and Import one
26	3	16	Clicks back and opens Tron with no issue						Event > Mission drilldown seems intuitive. User able to navigate to the higher event level with ease and then drill back down into a mission.
27	3	18	No issues finding route calculation button from the banner
28	3	19	Calculation means total miles, fuel, descent points, waypoints, headings						Double check what the current calc engine outputs.
29	3	21	Updated waypoint to Marshal from table with ease
30	3	22	No issues finding calc button						Calculate button is intuitive
31	2	6	Found Hammer easily
32	2	8	Found routes easily
33	2	10 to 13	Autorouted and viewed on map with ease
34	3	20 to 23	Tabular seemed very intuitive
35	3	23	Green checks are very intuitive to show successful recalculation						Green checkmark is pretty intuitive
36	4	28	No issues finidng clone within the "more" menu. Both pilots understood the difference between "clone" and "clone and open."
37
38
39
40			General Observations/Discussions						Analysis
41	1	1	Did not click on login input fields						Prototype issue, not usability
42	1	1	Logged in with CAC button						Prototype issue, not usability
43	2	4	Discussion around if NOMS can integrate with website where ATO is dowloaded from - but concerns around slowing the system down.
44	2	7	"Unique Identifier" was a confusing term.					P4: Use L16 TN instead of tail number as unique identifier. Tail number doesn't matter. NOTE: This came up organically with the "Please find where you could change the Hammer11's unique identifier" question.	If possible, use side no or tail no depending on if airforce or navy
45	2	10	Trained in JMPS that creating/generating a new route will discard the old route - users have been trained on this						UP allows for creating a new route and for overriding the current one.
46	2	10	Will only create new routes for cross country missions					Is this platform specific?
47	2	10	Use waypoints only. Create waypoints in Foreflight with overlays on, and then bring those points back to JMPS						Do we know how these waypoints are brought into NOMS? Is there currently a "chart" that noms can display that will assist the user in creating waypoints similar to how users are used to with other systems? When the user was talking about this, was the user talking about autorouting or about drawing a manual route?	This was during the autorouting task, but I think he was referring to how he creates all his routes - he was the user pushing for fields to enter lat/long to enter all waypoints instead of dropping points in the manual
48	2	10	Vocalized want to include sky vector/foreflight into the system
49	2	11	User inclined to create new. Some worries about modifying someone else's route					Quote from P3 "I would always create new to not save over other people's work - it's a squadron rule"	See line 45
50	2	13	Clikced map icon in the nav tool bar
51	2	14	Yes, user would expect the system to take you directly to the graphic view						Might need to gather more research, but if more user feel the same way, after clicking "autoroute," the system should take the user to the map view and center on the route.
52	3	17	For FA-18s, they wouldn’t normally draw routes. They are waypoint dependent. Suggested dropping waypoitns with specific lat/longs. However, for Helos, their community finds value in drawing manual routes						Drawing routes seems very platform dependent
53	3	19	User suggested exporting these calced values to excel so they can use it for their kneeboard cards						Suggest LUXCE does a more indepth research into how different communities handle kneeboards. There are plans to incorporate a kneeboard tool into UP so exploratory research is recommended.
54	3	20	Clicked the Tabular left nav item first and when that didn't respond, user clicked on the tabular editor below the route drawing tool						Pretty sure both take you to the same place, so I don't think this is too much of an issue
55	4	25	As a Helo, he would draw airspaces a lot and like the drawing tool. However, they think something more precise where you can enter in the number of points you need and their lat/long (current system allows this in the plugin). They don't use polygons a lot because of the limited waypoints they are allowed in the cokpit so they use other shapes to get what they need.						Perhaps the placement of entering in the lat long needs to move somewhere that allows the user to draw and enter in lat/long as they go
56			As a FA-18, they would never draw an airspace like this. They are fans of the details view where you have a table to enter in the points.
57	4	25	Both Helo and FA-18 pilot agreed that importing airspaces is the most common/likely behavior
58		4	If I couldn't find something I needed/wanted, I would just search it
59	5	27	I would want to close out of everything else before I start another task						Ensure everything has an easy/consistent way to close out of especially on the map; believe we currently have this
60	2	11	User would normally select replace - saw it as his route for his mission so he should be able to do anything he wanted with it
61	4	26	Shape manipulations: color, transparency, name, needs to be 3D					Label names on the map? This should be a toggle because that could clutter the map; transparency
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100