Dominic Amato                February 15, 2015

Common Obstacles New Users Face with Unfamiliar Software

Objective:

To observe the struggles that new users have when faced with learning a new application. The observation should serve to inform future considerations of interface design whereas user interface learnability is concerned and how such obstacles can be removed or minimized.

Method:

The first objective was to screen potential subjects in order to find a suitable user who was familiar with computer technology yet would not be considered an expert or intermediate user. To do this I talked to friends who I knew to be “novices” and found their availability and what types of software they were familiar or comfortable with. In order to prevent them from anticipating the software they would be working with I gave them a list of software I had deemed easy enough to learn though may be unfamiliar to the users. This list consisted of MS Paint (for predominantly OSX users), Adobe Photoshop, Adobe Illustrator, Audactiy, Microsoft Expression Design 4, and SketchUp. Since most users were generally familiar with photo editing software, though would still strictly considered beginners, I went with SketchUp as it was an unfamiliar domain, 3d CAM software, yet still had a low learning curve.

In order to drive the user’s tasks I allowed them to first briefly familiarize themselves with the UI and then gave them the task of designing some furniture they might use. This included an example of some office furniture I had designed within a 20 minute time limit that utilized most of the basic features (drawing shapes, extruding and subtracting material, coloring, and beveling). During this process I recorded the screen and audio and encouraged them to vocalize issues they had with the premise that I would only help if they were truly stuck. I also took paper notes of my own during the observation and noted time frames in which the user came across obstacles to review the process again.

Once the task had been completed or the 30 minute time frame had been reached the activity was stopped and there was a quick debrief with the participant in which I asked questions related to why they struggled in certain areas and also to gauge their reaction with how they felt they had done or frustrations they had but may not have vocalized. The response from this debrief was also considered in the review of the screen capture during the analysis.

 

Results:

The participant in this study was a Mac user though the study took place on a PC laptop with only a trackpad. The activity took them 25 minutes before they gave up in frustration. The user had minimal issues using the tools they recognized but the features and actions of those tools were not what they expected and was ultimately why they gave up in frustration. The first expectation the user experienced frustration with was the paint bucket which they wanted to make a carpet for their model. They selected the bucket but it would not paint what they thought was the floor, they had not drawn any shapes to paint on, though they expected that from the 3d axis lines that a floor existed (figure 1.)

Figure 1: The paint bucket won’t paint on the ‘floor’

The next frustration was with the zoom function. The user wanted to zoom out and were quickly frustrated with the lack of feedback and clues as to how to use the zoom function. In SketchUp you press down the key and move the mouse up and down to zoom in and out, or use the scroll wheel on the mouse. The user would click and it would not zoom in or out. They tried familiar methods such as holding ctrl and clicking, which works in adobe environments, and resorted to pressing the minus key to try and get the minus sign to show up on the magnifying glass icon. With some guidance they eventually figured out the function though remarked it was “stupid.”

The rest of the time the user experienced slight perversions of expectations with the drawing tools but did not consider them to be obstacles. They experienced some difficulty using the circle tool as they wanted to draw the circle in a certain perspective, the reason why they were unable to will be covered shortly, but were able to overcome limitations of the tools using other tools to generate their desired results. They even remarked about certain features of the drawing tools saying that they were “too smart I don’t want to draw the lines its suggesting” as SketchUp tries to make shapes and so encourages the user to snap lines together in order to make the shapes.

Despite the initial frustrations, the user was intent on completing the task and felt confident for the most part though in the final minute they changed the perspective and revealed they had been drawing a 2D object the entire time and not a 3D one, see Figure 5 and Figure 6. This is why the circle would not draw in the perspective they desired as it was attempting to be drawn on a 2D plane that the user had mistaken to be a 3D model.

Figure 5: The final drawing from the initial perspective

Figure 6: The Rotated Perspective

After realizing their mistake after 24 minutes of drawing they had no interest in continuing and so concluded the observation. In the debrief that followed the observation the user was asked about certain things they thought may have caused the problems they ran into and clues that may have helped them catch their mistake earlier. One thought they verbalized was, “I wasn’t smart enough to figure out I was drawing lines in only one perspective and not in 3d space” and when prompted what clues may have helped them they responded, “I guess there aren’t any shadows if it was 3d there would have been shadows.” They also learned that the furniture they were drawing was over 5 meters in width and responded, “There should be a scale bar in the corner I didn’t know how big I was making things.”

Figure 7: The length of the arc is about 5899mm

Discussion:

From the observation is was clear that the user had used their past familiarity with photo editing or drawing software to inform their actions using SketchUp. The simplicity of the design and familiarity of certain features allowed them to feel more comfortable using the interface and only caused them to defer to asking for help when the actions they wanted to perform were not consistent with the mental models they had associated with those icons. Certainly the UI allowed for that familiarity but the underlying functions proved consistently problematic because they did not conform to standard actions that the user was familiar with. For example, the paint bucket tool worked when there was a shape as the user expected it to but the initial use of it proved frustrating because they wanted to paint the ‘floor’ which was hinted at existing because of the axis lines and previous applications allowing the user to paint the canvas as a first step in the process (e.g. MS Paint, Adobe Photoshop.)

Perhaps more egregious was the zoom tool which is a very familiar tool in most programs. This tool had a multitude of perverted expectations leading to frustration. When clicked it was unresponsive, it did not zoom in or out and gave no clues as to why no action happened. The icon/cursor itself never changed so that the user would understand what function was actually happening, this usually being a plus or minus sign being placed on top of the cursor. Further issues with the functionality was that the user would have to click and then drag the mouse up or down to zoom in or out, the major problem being that now the users mouse cursor is no longer where the desired area to zoom was and that they need to now perform multiple actions just to return the cursor to the area of interest. This functionality is slightly mitigated with a mouse since zooming is easy using the scroll wheel and even with a track pad can be zoomed using two finger scrolling, though such actions are never told to the user and does not excuse the frustrations caused by the issues listed above.

An issue that the user was oblivious to until pointed out after the fact was the sizing of everything. Being used for mockups and computer aided modeling, the size should be a very important component of SketchUp but the implementation is hidden and somewhat cryptic.

When you first open the program there is a greeting window that is more concerned about getting you to buy SketchUp Pro and managing the product license then actually setting up functionality that is important to the program itself. When the user opened SketchUp they just hit the start button without any consideration as to the setup of the modeling (metric vs imperial etc...) That is certainly a major issue when core functionality is hidden away for extraneous factors like product licensing. This is not aided once inside the program as can be seen in figure 7 though figures 1, 5, and 6 also show the size of the drawings being displayed in the bottom right corner.  This indication is well out of the range of the fovea and since it is small thin text, is unlikely to be picked up by the periphery vision. This is evidenced by the fact that the user in this study never once noticed it in over 25 minutes of use. Easy solutions would be to display the length of the lines drawn right by the cursor and within the fovea as user attention would be drawn to it and would be unlikely to become a distraction.

Overall I think the program has several issues covered by Donald Norman’s principles of good design. Visibility failed as the user never recognized that they had been drawing on a 2D plane until it was too late and then they no longer wanted to continue using the program. It provided false affordances in that regard. The conceptual model was actually quite good in terms of familiarity to the user, though it suffered because of a lack of feedback, like with the zoom issue, and poor mapping, as the user thought they were performing actions that was familiar to them from previous programs though they did not work the same in the context of SketchUp. Another example of this, though it never came up during the observation was the key command to undo an action. Typically mapped to ctrl + z or cmd + z, in SketchUp is alt + backspace which ultimately bookends why the program may not be such a good introduction to CAM software for users.

The UI tries to be too familiar but creates cognitive dissonance when the actions performed don’t match the actions expected. The pencil tool and functions like that are convenient but ultimately the program is for modeling in 3d space which might not be the proper place for a pencil tool. The most useful tool that the user never even touched once is the “push/pull tool” which is poorly named and isn’t explicitly shown to be a very important tool. That tool is the same as extruding in other CAM software and is essentially how one moved from a 2d shape to a 3d object. That tool is hidden in the middle of the tool bar and sort of gives the hint that it is for raising or lowering shapes into 3D objects but is easily glossed over because of its position. The tooltip isn’t particularly handy either as it says, “push and pull face entities to sculpt 3d models” which is cryptic, what is a face entity?

Figure 10: The push/pull tool

Perhaps most detrimental is that novice users who find some of these tools familiar will be apt to use them and get unsatisfactory results, similar to the user in this study. Why this is the most detrimental is that the user then blames themselves rather than the program for the mistakes made? When a program is hard to use and unfamiliar the user blames the software for being hard to understand, which is a fair criticism, and a better outcome than a familiar looking program that is hard to use.

SketchUp hits on a lot of the base design frameworks like the utilization of color, using icons over text, and stimulating recognition over recall but then fails in major things by relying too heavily on system ones faster and familiar conjectures and then perverting the outcome that is expected. The language used is jargon even for those familiar with the domain, actions often require poor use of Fitt’s law (i.e. the zoom,) and the timing expectations are thrown off by a lack of consistency and nonresponses from certain functions (again the zoom.)

Conclusion:

If I were to follow up on this study I would have used a mouse instead of the track pad as the user was unfamiliar with the two button trackpad leading to consistent input error. Another issue was the orange halo around the mouse cursor (figure 3, 4, and 10) that, while useful for keeping track during a demo, was a frustration to the user as it obscured their view and was an annoyance. Initially I wanted to compare the performance of the user in another environment but the frustration of the user in the ‘easier’ environment led to the omission of that component of the study. If ever there were to be a comparison between environments more users would be required and the environment would be switched between users instead of having the user do both as their frustrations would limit their desire to continue.

Certainly one thing to take away from this study besides changes to the future implementation is that consistency really matters and that proper feedback is instrumental to the success of the users. A users familiarity with certain icons and their typical functions need to be followed or their expectations will not be met and the resulting mistakes can lead the users to either give up in frustration or make them feel dumb, neither of which are ideal for software developers and even more so entry level software that is supposed to be user friendly.