In preparation for UDS, we conducted usability testing of Unity with general public users.  We are now better informed as to where we should expend further efforts to enhance the experience for users working with Unity.

I summarize my findings below.

Participants Selection

We asked an external recruiting firm to find 15 participants who answered to the following criteria:
1.  Mix of males and females: roughly equal
2.  Age:

• 7 people between the ages of 18 and 30
• 6 people between the ages of 30 and 50
• 2 people 50 or over

3. Employment:  employed full-time or to be full-time student
4. Involvement in marketing research: no such involvement in the last 6 months.
5.  Basic technology:  Each participant needed:

• To have a broadband connection at home
• To be a daily internet visitor, staying online for at least 2 hours every day for personal reasons

6.  Online activities:  Participants needed to have done all of the following while online in the last 2 months:

• Look for information
• Read comments written by others
• Read blogs written by others

In addition, they needed to have done at least 4 of the following in the last 2 months:

• Create or review a social networking profile
• Post comments or reviews
• Write a blog and conduct some activities on social networking sites
• Shop
• Share photos
• Play games
• Download music on a music player.

7.  Participants needed to have a strong interest in technology.
8.  Each participant needed to own at least one portable music device, one mobile phone, and one computer/laptop.

Of the 15 participants recruited, 13 were Windows users, 1 was a Mac user, and 1 used both Windows and Mac.  None of the participants was familiar with Ubuntu.

Methodology

During one-on-one sessions, Unity was presented on a netbook.  The goal of the sessions was to get participants to experience most of Unity’s features and functionalities.  I introduced the session with the following instruction:  “Imagine this is your new computer.  What do you want to do with it?”  I had songs and pictures available for them to import.  During the 60-minute session, I asked them to do some of the activities they normally do, which included importing songs and photos, engaging in social networking, etc..  However, some of the tasks were tailored to fit the specific interests and knowledge of given participants.  For example, if a participant was a professional musician, I proposed to him or her more tasks related to music management; similarly, if a participant was a student, more of the writing and reporting capabilities of Unity were explored.

Findings

What participants liked about Unity

The look and feel:  First impressions of Unity were positive and many comments pertained to its elegance.

One participant said:  “Simple and clear tabs on the side.  I don’t like to be crowded with stuff on the screen.  It looks quite approachable.”

The core concept of Unity:  Many participants intuited the new concept underlying Unity’s look and feel and direction.

One participant  said:  “I like the application idea and that it is application-oriented and you can go there and have a quite user-friendly interactivity.  I  like the tabs at the top and when I hover it tells me what it is and it is big and clear, easy to see.”

Workspaces:  Many participants were unfamiliar with the concept of workspaces and were intrigued by its overall potential, and its friendliness.
One participant said:  “What you can do is that you can have different applications in different workspaces.  This is a nice feature.”

Dash:  Participants liked the simplicity of the dash and its look and feel.

Software Centre:  Generally, the software centre was seen as impressive, particularly its large number of available free software.

Usability issues

Performance of Unity

Participants were challenged by various functionalities and conceptual design features of Unity. It is noteworthy that one factor stood out as principally responsible for many usability problems: this was Unity’s slowness and suboptimal responsiveness during testing. The level of performance in this regard significantly impaired the flow of use and the user experience. Testing was done on a mid-range netbook that users were likely to own, and consequently the user experience in this regard was similar to what users would have experienced outside of the usability lab.

Multitasking  – Multitasking on Unity is disconnected and difficult at times

Task flow is interrupted: While working on a task, participants wanted to have all the documents and websites they were using easily available to them.  For them, the task was the unit of organisation of all resources and tools.  Thus, while working on a task, participants expected that Unity would provide them with a representation or visibility of what was available to them and how to easily access what they needed at any given point.  Unity does not, however, make evident the resources and tools users have at their disposal — whether it be multiple documents, programmes or websites. In a word, while Unity relies on users to keep track of the resources they are currently using, users are habituated to relying on the software to “keep track for them,” by making the resources highly visible, for example, by means of tabs. With Unity, resources are hidden from view.

Overall, participants found the navigation between documents to be cumbersome. Often, they looked for a way, a back button or breadcrumb, that would navigate them through their open applications and  documents.  They didn’t find either. As a consequence, they ended up going through the files and folders section to access their open documents. Clicking on the OpenOffice icon to access a document already open was not immediately discoverable.

One participant noted: “[navigation] seem[s] awkward.  I’m not getting around as quickly as I should.  The icons are supposed to tell you but I don’t know what they are.  That is a problem.

Another said: “It seems a bit pedantic to have to go through the application menu to navigate.”

Poor visibility of applications and of windows that are opened: Participants experienced difficulties with the following:

Minimising a document: When participants minimised a document, the document seemed to have disappeared when they expected the mininised document to be shown at the bottom of the screen.  In Unity, there were no traces of the document on the screen and participants could not find it again.

“I’m not entirely sure how to get back into my new document [after minimising it].  [It is] not clear that it opened a new page for it.  I don’t know how to get back to it.  I don’t know, it’s weird.  I expect it to be a tab and be able to switch between the two.”

Knowing which documents are currently opened:  Participants made it clear that they not only wanted to know if their documents were opened, but also wanted to know how many documents or windows were open at any given time.  But they did not always see the white triangle that indicates that a programme is running or that documents are opened.  Consequently, they were not aware of what was available to them.  In short, the general indicator was not enough for their needs for awareness.

Being able to distinguish between similar documents:  In the exposé view, documents are very small and, when 2 documents are similar, participants could not tell which was the one they wanted without opening each one individually.

Identifying a document:  When participants were in the process of writing a document, they looked for the name of the document they were working on in the top menu bar, but couldn’t find it.  They were also looking for such information in the exposé view.

Being aware of multiple open documents or windows they needed to consult or modify: Almost all participants wanted to organise their work by opening several windows and physically positioning them in a way to maintain an awareness of all of them.  For example, one participant opened Facebook and a word document.  She wanted to move the word document so she could expose the top part of Facebook.

“Can you move the window down at all?  I like to have a little bit of space for organisation.  I want to be able to move the window down.  It would be nice to [be able to do that], especially if you want to start working, you want to organise your environment.  You want to feel the sense of relief and organisation before you start working and [the sense] that what will come to you will be manageable.”

Immediate access to documents they currently are working on:  Exposé is not readily discoverable.  The majority of participants could not find the exposé feature.  As it is not a feature they are used to having, they need a prompt to help them explore it.

Exposé does not allow users to work on documents side-by-side.  Participants expected to use the exposé feature to be able to cut and paste between documents directly without having to bring one document up in full screen and then having to navigate to the next document by means of the workspaces icon.  As it is, exposé is useful for viewing only.  Participants would have liked to have had the option to directly navigate and conduct operations (e.g., cut and paste) between documents.

Documents cannot be resized and placed side-by-side:  No participant could find a way to resize his/her Open-Office documents in such a way that they could be placed side-by-side while working on both at once.

Overview of what’s going on: Many participants wished they could have a view of what resides in various parts of their computer, as is facilitated by Windows’s “my computer”.

Back button:  Many participants needed to “go back” to a document or an application that was currently open.  But the back button doesn’t work coherently and seemed to bring participants to arbitrary pages.

“The thing here, backward and forward buttons don’t work.  Back button doesn’t get me back to where I was.  I don’t know where it is taking                 me, I’m frustrated.  The back button is taking me somewhere [else].”

“If you can’t navigate you feel like a fool and you feel ashamed even if you’re alone in your room.”

“I’m struggling to find short-cuts and a back button.  Right now, to me, it feels very clunky to go from one place to another.”

Document management

Open documents: As already indicated briefly, participants didn’t know how many documents they had opened.  The white triangle indicator, meaning that the programme is working or that at least one document is currently open, is too general for users’ needs.

Delete documents:  Participants could not delete existing documents from their files and folders.  They first tried to drag and drop documents in the waste basket; then they right-clicked to see if they had such an option.

Copy and paste:  Copy and paste from one document to another didn’t always work for participants.
“Copy and paste is not going to work.  I’m not impressed, it must work.  I’m one for copying and pasting.”

Lack of feedback and guidance

Unity is often slow, and as a result participants tended to be confused about what was going on. They didn’t know:
1) if the system was slow and would eventually produce the desired effect, or
2) if the system had not registered the participant’s command and as a consequence, they should repeat their action, or even
3) if the system had crashed.

Unity needs to clearly show if it is engaged in a process or not.

“Has it crashed?”

“Let’s make sure I clicked it.  [She opens a document.  She’s getting confused navigating between documents and she closes the document without saving it.]  When I close, it should ask me to save it.  This is not good [that it doesn’t].  You rely on the computer to save the document or not.”

Search

When entering a search term in the search bar, there is no feedback to show that Unity is indeed engaged in a process.  Many participants were confused by this.

“How do you know this is searching?  Is there any sort of symbol to tell you if it is doing something?  I would like to see if there is a symbol that says the computer is working so I am not clicking around.”

Other issues:  Interactions with the launcher, finding the dash, lack of reference to understanding Ubuntu features

Interactions with the launcher

Adding an icon to the launcher:  Many participants were not able to add a short-cut of an application to the launcher.  They used various strategies but failed:

1. They tried to drag and drop icons directly from the applications section to the launcher.
2. They right-clicked, expecting an option to copy.
3. A few participants noticed that when an application is open, it appears in the launcher, and when they close it, it disappears.  They were confused by this feature, but at the same time perceived it as a clue.  So they tried to drag and drop the icon from the applications section onto the similar icon visible in the launcher.

Changing the order of icons in the launcher:  Most participants failed to reorganise the order of icons in the launcher.  They selected an icon and tried to drag it upwards within the launcher; that is, they did not realise that they needed to remove the icon from the deck first before they could relocate it.

Deleting an icon from the launcher:  Participants did not succeed in deleting an icon from the launcher.  Most selected the icon and then dragged it into the waste basket at the bottom of the launcher.  They did not see that they could remove an icon by dragging it horizontally away from the launcher.

Finding the dash:  The majority of participants who found the dash found it by accident. They were not sure what it was, and didn’t know how they had gotten there if they accidentally had.  As a consequence, they were not able to find it again. The problem for participants here was that the logo is not recognisable to them.  Of course, they were not familiar with Ubuntu; but, equally important, the logo is less visible in size and colour than the icons contained in the launcher.  It took participants a while to see it at all.

“The logo should stand out more.  It would be nice if the icon itself [was] coming at you, more of a three-dimensional thing.”

Lack of reference to understanding Ubuntu unique features

Software centre: Almost all participants were surprised when I suggested that they install a game that was not currently installed on their computer.  All of them immediately went to the internet.  When I told them that such games could be found on Ubuntu, they were confused and many didn’t know where to find them.  They went to the applications section, where there remembered a section called “installed” software.  They expected either a section below this one with a list of uninstalled software or (at the very least) a link to the software centre.

Moreover, most participants expected to be able to click on an icon in the dash, for example ‘music,’ and directly access their albums and songs.  They were a bit surprised to be taken to music applications instead, especially in light of the interactions of the other icons in the launcher, which  are quite different.

“I expect to see music files, names of songs or albums, anything you’ve downloaded.  Titles.”

Familiar short-cuts: Participants who normally used short-cuts to navigate didn’t find the familiar short-cuts and were frustrated.

“There are things that are nice to keep from one OS to another, like right click and short-cuts.  It’s nice to have two possible solutions.”

Changing the wallpaper

Many participants did not succeed in changing their wallpaper because the default screen of appearance was open in full screen by default.  Because of this, the action of immediate wallpaper change, as users select a wallpaper, was not visible to them. After participants had selected a wallpaper, they expected a step to confirm that they wanted their wallpaper to be changed.  Instead they were given  other options that were not relevant to what they were trying to do, and at which they had already (but unbeknownst to them) succeeded.

Search

The search functionalities felt limited to participants. When searching, they didn’t know what the field and scope were that were covered by the search engine they were using.

Grey icons in launcher

Participants thought that the grey icons in the launcher were inactive, especially when they clicked on them and the system was very slow in responding