How to approach usability testing

In order to deliver a clean, fresh, and — more importantly — effective user interface, usability tests are required. It is very unlikely any designer, regardless of his reputation and skills, will be able to design a good product without doing some kind of research and testing.

Usability testing is a technique used for evaluating a product by testing it on users who are part of the respective target audience. Testing is used in many fields, but today we will focus on user-centered interaction design and how to test when designing and developing such a product.

Every product has an intended purpose, and the scope and aim of usability testing is measuring if a product meets this purpose with regards to a user.

The four principles

Behind every usability test there are different goals, which pertain specifically to the observation aims of the tester. The results can be treated as a control measurement or a baseline. Because several tests can be conducted throughout a period of time, all the results will be compared with the baseline test results.

The four principles behind usability testing are:

• Efficiency – the tester measures how much time and how many steps are required for the user to complete basic tasks (find a product, add it to the cart, read the feedback and ratings, ask questions, buy the product. These would be basic tasks for a mobile app such as Amazon.
• Accuracy – how many mistakes do users make when trying to perform these tasks and how fatal are the mistakes? Sometimes, with the right information, the mistake is recoverable.
• Recall – after a period of non-use, how much does a person remember about the interface and the browsing process?
• Emotional response – how does the user feel about the tasks he had to complete? Was the person stressed or confident, and would the user recommend the product to a friend?

These are general principles used in testing user-centered interfaces, but it is important for the tester to set his own usability goals. Based on these, he will be able to closely monitor the subject and interpret his mistakes or gestures.

Wrong interpretation

Some people interpret the term “usability testing” incorrectly. Just gathering opinions on an object (or a device or an application) doesn’t mean anything more than market research. This type of research is definitely not usability testing, but rather quantitative research.

In order for such a procedure to be labeled as usability testing, it requires involving a systematic observation under controlled conditions; this determines how well users (always part of the target audience) can make use of the product. Knowing that 86% of the questioned users mentioned that “the application works fine” doesn’t mean you tested the usability of your application and the results were mostly positive. This only means the majority of the questioned individuals seem to think the application works fine, but this is not enough information for you to use for improving the interface.

One key aspect of usability testing is to involve the users as much as possible. Instead of asking them what they think about how a mobile interface looks, ask them to perform some actions. There are many aspects affecting the browsing process and most users will not be able to name or discuss them, but they will most definitely be able to show it to you while using the interface.

Methods

There are several methods to usability testing available, and I will go through most of them — or at least the most important — hoping to give you an idea of which one is more suitable for you and your purposes.

When testing a product you need to create a realistic situation in which the participant has to perform a list of tasks using the product you are testing. During this, observers should watch carefully and take notes as quietly as possible. Different props such as paper prototypes, scripted instructions and pre or post-test questionnaires are also used to gather information and feedback about the product you are testing. The think-aloud testing method, co-discovery learning and eye-tracking are usability testing techniques that can be used throughout these methods.

Hallway testing

This is a general methodology working with a limited number of people, ranged between four and six. The name of the testing comes from the idea that participants to the test should be random people who pass by in the hallway. This method can be used when your product is not necessarily aimed at a specific target.

Hallway testing should be employed early in the design phase. Test quickly and test often! This means that you will need to go out there several times. The process is quite simple: test on five persons, go back to the drawing board and solve the issues. Go out and test again on five other people, get back inside and solve the issues. After testing three or four times, the number of critical interface mistakes should be narrowed down significantly, and you then can start focusing on developing the product and its features. You will need to test again at some point in time, but knowing you solved most of your interface issues should allow you to focus on the development phase a bit more.

The reason behind using totally random people is because you don’t want to test your product on individuals who are somewhat familiar with your product and its interface. You want people who have never seen your interface before, so they all start from a common ground. Moreover, this way you can test newcomers — who are most of the time the easiest to lose — as their level of interest and motivation is not high enough yet. If someone who has not used the application before is very happy with it and handles all the tasks easily, it means most of the people who will use your application will do the same.

Remote usability testing

This methodology can be used when the product you test on has prospective users in different parts of the world. Bringing them together would pose real financial challenges and might not be possible for a freelancer or a small company.

Experts concerned by these issues came up with this methodology — which facilitates evaluations and testing being done remotely — with the user and the tester separated over space and maybe even time. Video conferencing is a way of doing this kind of testing, while another one could be by employing remote applications such as TeamViewer or WebEx. Both of them involve users who have a personal computer and an Internet connection. This way the tester can follow the participant’s movements, but not their reactions and emotions.

The tester can automatically get a collection of user’s click streams, user logs of critical mistakes, incidents that occur while interacting with the interface, and even subjective feedback by the users.

The good part about this kind of testing is that it is carried out in the participant’s own environment, which means he will be very confident in his abilities, and you will be able to simulate a real-life scenario testing. Clearly, the biggest advantage of this remote testing methodology is that it allows you to work with people from all over the world without many costs for transport and logistics.

There are several tools a designer can use for remote testing. WebEx and GoToMeeting are the most popular, but pretty much any remote tool would do the job.

Regardless of how well the tools would work, carrying out a synchronous remote testing is a bit more difficult than it looks like, as managing linguistic and cultural barriers through a computer might decrease the efficiency of the test. Interruptions and distractions in the participant’s environment are other challenges that are pretty much impossible to solve from the other corner of the planet.

Expert review

This is another methodology for usability testing and requires bringing in field experts to evaluate the product in testing. The challenges of this method are mostly financial and logistical, as it would cost a lot to bring in experts from different areas.

There is also an automated expert review methodology, which is based on the same principle, only it would be done through the use of different software.

A/B testing

A/B Split Testing is probably one of the most well known experimental approaches to user experience and interface testing. It aims at identifying the elements of a webpage that increase the user’s interest or engagement.

The method is called A/B testing because there are two versions of a website/interface (the A and the B version) that are compared. They are always identical, except for one variation (which can be an element such as a button, contact form or image) that might impact a user’s behaviour.

During the testing period the website is monitored through tools such as Google Analytics. In this period the two versions, A and B, change randomly, which means that you can come on the webpage and find a header image, then refresh the webpage and find the other header image.

The methodology is mainly used behind the scenes to maximize profit, reduce drop-off rates and increase sales. Amazon pioneered the methodology, but companies like eBay, Google, Walmart, Microsoft, Netflix and Zynga are also known for employing this method to increase the profitability of their sites.

Although this is mainly used for e-commerce websites, A/B testing can easily be used in interface design as well; and it can be as effective as giving testers an overview of which interface is better between a choice of two or more.

How many users to test?

Carrying out several tests with a limited number of participants is much better than testing once on a larger number of subjects. This translates simply into many quality tests instead of few quantity tests. Around five subjects for each test should be enough to help observers get enough information to work with for a period of time.

The argument behind this theory is that once you find out few people are confused by a feature or a website, you gain less from testing the same interface on even more people, as they will most likely be confused by the same elements. The solution is to solve the issues and then go out there and test again on a limited number of subjects. You need to repeat this process several times to get the best out of it.

There might be some downsides to this theory, many experts say. Usability usually applies to a larger sample of the population, not only to a specific set of users; this means that interface problems might be undetectable by the first group tested. However, carrying out one or two tests with this limited number of subjects is not what this theory suggests.

These tests should be carried out every week — maybe even twice per week — during the design process. The longer the design process is, the larger the number of tests has to be. During this whole process even a number of subjects between 50 and 100, or sometimes even larger, could be tested.

It would be more effective to test subjects across a broad spectrum of abilities in the second phase of testing. During the last tests, as the design should already be smooth, you could narrow the observations down and start testing at your own target audience.

When conducting usability tests, it is also important to notice the things that work well — not only the ones that don’t —and keep testing them over and over again. The theory behind this is that elements not working well should be eliminated, but elements that work well and are enjoyed by users should also be paid attention to. Try to keep them the same as they were in the first instance, because they obviously work well. Focus more on the ones that fail instead of trying to change and improve the ones that are already functioning successfully. There is time for that later on during the post-development processes.

Conclusion

Usability testing is something worth carrying out if you develop an interface and hope to achieve some kind of success with it. It might not be worth investing in it when developing a simple website; but I would personally always involve some testing if developing a mobile application, because it is much more complex.

Usability testing can also be done more or less for free; you do not necessarily need to invest a big amount of money in logistics. If you feel you only need to test on a smaller scale, use your friends and relatives for it; it would all be free or very, very cheap (chocolate cake is always a winner).

As you can see above, usability testing is something you can do in many different ways and you have to determine which way is right for you and your purposes before starting. It might seem like a very complicated process in the beginning, but even a beginner should be able to carry out such a test and get something out of it. So if you are in the middle of your design process, do not hesitate to go out there and do some testing — I promise you it will improve your interface and your users will be much happier with it.

Do you carry out usability testing on your designs? Which testing methods do you favor? Let us know in the comments.

Featured image/thumbnail, testing image via jurvetson

Christian Vasile

Christian Vasile is a Romanian web designer and online publisher based in Denmark. He has a crazy passion for design, usability and user experience and shares his thoughts with WDD from March 2013.