Part 1: Introduction to usability testing
Part 2: Preparing your usability test
Part 3: Running your usability test
Part 4: Usability test evaluation
Usability Test Agreement
Before you start preparing for the test itself, make sure that you have provided respondents with a Usability Test Agreement. Such an agreement will clarify all rights and obligations covered in your mutual relationship with the respondents. The following information must be included in the agreement:
- Identification of parties taking part in the testing
- Purpose of the testing description
- Sum of remuneration and its payment method
- Statement about the handling of personal information solely for the purpose of test processing
- Statement about handling records (audio, video, …) solely for the purpose of test processing
- Consent of the respondent for test recording
- Respondent’s obligations to fulfill tasks assigned during the test
- Respondent’s obligation to maintain confidentiality about information that they see or hear during the test
This list is not exhaustive. If you decide to conduct any type of research or experiment – usability tests included – it is always a good idea to consult a legal expert in your respective field to help you create the agreement. Afterwards, you can use this same agreement template for future tests.
Give respondents a call in advance
When recruiting respondents, conduct a preliminary phone call with them to collect the necessary information for testing. Make sure that the respondent actually matches the requirements you set for your test. You definitely do not want to accidentally let an unsuitable respondent who ends up not being part of your target group participate in the test. Including irrelevant candidates in your test may jeopardize the results or insights of your predefined target group.
Inform the respondent of any crucial information in advance during the phone call, E. g. how the test will be conducted, if it is recorded, how the remuneration will be paid, etc. You can also send the respondent the testing agreement in advance and sign it in person at the usability test, thus saving precious time on the day of the event. In case you are conducting the usability test online, sending the agreement as a soft copy is the best way to get the agreement signed by the respondent in advance.
Pre-Test Preparation
To start off, it is essential to determine the ultimate goal of your usability test. Should the respondents be able to easily complete a full transaction on your site? Is the test about looking up contact information on a company's website? Or is the goal to successfully pay for a service?
Once you have your goal in mind, you need to design tasks for your respondents that will lead towards confirming your goal as well as verifying that your product is usable. During the test, you will assign the prepared tasks one by one to the respondent and evaluate how well they were able to complete them (or not), as well as where they encountered any difficulties.
Before beginning of the testing with your respondents, it can be beneficial to go through the test first with a colleague, friend or relative. This way, you can preemptively find mistakes with the tasks you intend to give to your participants and correct them ahead of time.
After the test, ask the respondents to describe their actions and feelings during each step of the task and, if they encountered any difficulties, to describe them out loud. At the end of the task, you can always ask the respondent for feedback, such as if they would have changed anything, what issues did they perceive, etc. The respondents’ observations can provide you with useful insights that you might otherwise overlook.
Make up a scenario
It is useful to have a short story or situation prepared to introduce the test to the respondent. For example, „your goal is to test how easy it is to find a specific phone number on the website of a travel agency.“
Instead of asking leading questions such as „How would you find a phone number on the website of a travel agency“? instead use questions such as „Imagine that you are looking to book a vacation and you want to schedule a personal meeting at the agency’s office. You do not know if the agency has a branch in your town. How would you contact the agency to find out“? Thanks to this wording, you will be able to test for both the amount of effort to find the particular phone number but also the customers’ preferred method to contact the agency.
If the respondent suggests a different way (e. g. contact via e‑mail) outside the parameters of your usability test, you can always lead the respondent back to the desired task with your next question.
Avoid Leading Expressions and Questions
Questions you ask should not deliberately lead respondents towards your test goal. To avoid this, it is useful to clarify exactly what you want to test for. Take a look at your question and evaluate whether it contains content that is too leading towards the goal or even contains key words that you want to test for. If you ask the respondent, „Where on the website can you find the contact information?“ you are guiding them directly towards looking for contact information. If you have a ’Contact’ section in the menu, the respondent will be able to complete the task within seconds. Alternatively, if you ask them, „How would you call customer support?“ you decrease how leading the question is.
Do Not Fragment Your Questions
Furthermore, it is useful to give the full task to the respondent at once and wait to see how they complete it. If you lead the respondent through the task with step-by-step guidance, you can avoid many issues that would otherwise occur. There is a difference between „purchase a large red jacket at the online store“ and step-by-step guidance „Find the men’s jacket section – choose the red color – choose size ‚large’ – put the item into the shopping cart“. The latter is fragmented and the respondent is led by clear instructions which are too easy to follow. By providing the respondent with detailed guidance, you lose the opportunity to discover shortcomings and flaws in your process.
Device Preparation
The ideal situation is when you test the respondent on a device that is well-known to them. When testing a mobile application, it is best to assign the respondent to the operational system they are used to. If the respondent uses an iPhone in their daily life, their habits are different than those of an Android user. If you develop a mobile application, it is crucial to include both operational systems because their users have different habits.
Imagine that you are an Apple enthusiast and you decide to test your website on a Mac, Windows users will have trouble trying to use a Mac. They can feel lost on the website because you chose a different operating system than they are used to.
If your product varies across operating systems, it could be time-consuming to create multiple prototypes for each operating system. In that case, you can invite respondents from your target group that are also familiar with the particular operating system that you have designed your prototype for. Of course, you do not have to make this distinction between respondents, but keep in mind that some issues could be related to unfamiliarity with the given operating system and should be evaluated as such. Therefore, the golden standard is to prepare the environment that closely resembles the user's experience as much as possible. That will ensure that the environment will not be unfamiliar for the respondent, and you should get more relevant feedback as a result.
