Part 1: Introduction to usability testing
Part 2: Preparing your usability test
Part 3: Running your usability test
Part 4: Usability test evaluation
Who Should Evaluate the Usability Test
Evaluating usability tests is not a complicated activity to carry out. You will need to dedicate some time to it, especially to focus on issues that were revealed from obtained responses. You can either evaluate the usability test in a group or, as is the common practice, the UX designer or researcher evaluates the test by himself and also edits the prototype. If there were other people that observed the testing and made notes, the UX designer or researcher can use these notes as a resource when conducting their evaluation.
Another practice that has proven to be useful is for the test observers to review the findings after each respondent. This way, you can check if the observers came away with the same impression of the test or if they noticed something you may have overlooked.
Which Findings You Should Pay Attention To
Not all issues you come across during the usability test are necessary to be fixed. You should always try to determine which issues are worth working on and which most likely resulted from the respondent’s misunderstanding of the instructions. As mentioned above, you can end up with a respondent that was maybe not part of your target group, and thus you should not consider feedback from that person as a complete failure, as a respondent from your target group would probably have a better understanding of the task at hand. If this happens to you, try to do another test with someone from your target group.
Practice makes perfect, but even without practice, not everything users point out during the test is automatically an issue that needs to be fixed. You should always investigate why users did not understand the task and where the core problem is. Did I choose a bad wording? Did I use a component that is not clearly visible in the prototype? Or are there too many components that are distracting?
Be an Investigator
If the respondents are motivated by monetary remuneration, they will try to reward you for the fee either intentionally or unintentionally. Keep this in mind during the evaluation, as sometimes respondents would rather not finish the assigned task but will feel obliged since they have been paid to do so. Focus on their facial expression and actions during testing – whether they seem confused or not. Try to find out what they would do if they had to deal with this situation home alone. Maybe the respondents would say that they would probably give up on the task because they got lost during the process. Or they may say that they would need to call the help desk to help them deal with the issue.
Sometimes, when you speak to respondents about this, saying that „At that moment it looked like you got lost in the process“ they might respond „Not at all, I was just thinking, but the process was clear to me“. This sort of response could be caused by the respondent's gratitude for being paid, or maybe they are simply embarrassed that they could not understand the instructions. You should be able to identify when a situation like this arises and appropriately mark the task as incomplete.
Conclusion
The usability test is a great way to find out if your product is understandable and usable for your users or if there are any shortcomings in your product or service. However, usability tests do not answer the question if the product, function or service you are developing is actually useful for users or if they are even interested. To answer these questions, you should conduct other inquiries and experiments, which is a topic for another series of articles yet to come!
Make sure to conduct usability tests on a continuous basis in order to detect as many issues as possible during product development.
With just a bit of practice, patience and determination, you’ll have all the tools you need for successful usability tests.
