How we used 5 second test to make Cuddle.ai’s alerts more intuitive.

I joined Cuddle.ai as an UX Research Intern along with Vaidehi Vartak (who soon became a very dear friend) for the summer of 2019. During our term as interns, one of the challenges given was to test the “intuitive-ness” of the detailed view of Cuddle.ai’s alerts.

Cuddle.ai delivers personalised insights identifying areas of business that may need users’ attention. Users receive timely alerts when Cuddle.ai discovers anomalies that are relevant to their needs. We were testing the detailed view of these alerts pages that are of the following four types:

‘Intuitive’ essentially means something that is easy to use and understand. This meant that we had to devise a test that would check the following:

1️⃣ If the users could grasp the necessary information from the alert, and
2️⃣ If they could recognise the context of that information.

What is a 5 Second Test

In a 5 Second Test:

1️⃣ Participants are provided with context
2️⃣ A screen is shown in for a few seconds
3️⃣ Participants are asked questions on the test screen

Below you will find an example of a 5 Second Test.

Why 5 Second Testing?

1. A well designed screen should be able to communicate key information within 5 seconds: research shows that an average human brain can grasp about 7 key points in 5 seconds give or take.
   Less than 5 seconds would lead the to glance over the screen without making much sense of it. More than 5 seconds would lead the user to view the screen more critically.

2. It is quick to plan and conduct: The name ‘5 Second Test’ itself is enough to indicate that it is quick. One could cover an entire 5 second test with an individual user in less than 4 to 5 mins. That means you could get a large data set in small period of time.

3. 5 Second Tests could be conducted in any environment with any available tools. In our case, we conducted the test going from desk to desk at the Fractal Mumbai office. We used a mobile prototype which we made on Adobe XD. This helped us to conduct the test efficiently and on the go.

How we conducted it at Cuddle.ai

• Planning to execution: 3 days

• Number of users: 84 participants

Step 1

Defining content and validating the use of a 5 second test

To avoid derailing from the aim of the user test it is very important to clearly define the scope (content) of the test screens.

Once content is defined and outlined, it’s a golden opportunity to take a step back and validate the use of a 5 Second Test. In our situation our screens had simple data of a single situation and did not require any further interactions hence we used a 5 second test.

When not to use a 5 Second Test:

• Reading and comprehension of large texts

• Comparing two different pages

• Predictions and analysis of next steps

• Feature heavy pages that need navigating. Such as homepages and scrollable pages

Step 2

Deciding the type of 5 Second Test

Referring to the diagram below you can decide which type of 5 second test is it that your problem statement demands.

In our case we had to judge intuitiveness of the screens. This meant that we had to test if the most important parts of the alert screens were being comprehended as expected. So, our selection naturally was a Target Identification, 5 second test.

Step 3

Writing the instructions and context for a 5 Second Test

Good instructions have:
▸ One idea per sentence
▸ Not more than 3- 4 sentences
▸ No answers to the subsequently asked questions

Context is what gives a situation to the 5 second test. For example, let’s say that I show you some numbers like this,

there is not much that you can understand from it. The same chart will make more sense to you once a title is given to it; ‘Number of TT games I played over the the last six days’.

Step 4

Making the prototype

We used an Adobe XD prototype as it was portable, it helped us maintain consistency and because could be accessed easily on a mobile device. Here is the prototype we used for your reference.

There are a few limitations to a 5 second test prototype such as, the prototype cannot be scrollable or have hidden, unclear text.

Step 5

Writing the Questions

While writing the questions there are three things to keep in mind

1. Importance of communicated information: The page being tested will have some information that are more important than others. Having a clear list of test page contents by order of importance helps in making decisions.

2. Keeping the number of questions as low as possible: During a memory testing test such as this an effect known as the Reverse Polaroid Effect¹ is in play. According to P. Doncaster this means that maximum memory state is right after the screen in shown and the state keeps reducing as and when time passes.

3. Ordering the questions: Due to memory erosion, the first question answered will be most accurate; hence, the order of questions is important. You could refer to the priority list and ensure that the most important aspect is questioned first.

Step 6

Running trial tests to avoid misunderstandings and non-responses

We conducted trial tests with our internal team to ensure we were corrected early, and so we could implement the changes fast.

Trial tests are important so that problems are pointed out before tested with actual participants. If these problems go unnoticed, their effects can be irreversible and lead to skewed results. Trial tests could be conducted after the test is completely ready or during the process of making the test.

Step 7

Defining your test participants

Depending on the screens you are evaluating, there may be users who’d be more eligible to take the test than others. For example, if you were testing if users were easily able to find the logout option from a screen you would want to test on users who have logged out of the application vs. new users.

In our context, we shortlisted users who have received at least one of the aforementioned alerts after referring to our data on Countly.

Keep in Mind

Any Answer would erode memory. This means that if you ask the participant their names just before you ask them the questions it would drain some of their collected memory too.

5 Second Test Checklist

If you are reading this because you are planning on conducting a 5 second test then I have one more thing that could help you. Below I have made a checklist for planning a 5 second test which you could use as a reference. Feel free to change and modify the checklist to suite you.

References

• Doncaster, P., 2014. The UX Five-Second Rules. 1st ed. USA: Elsevier

Thank you for reading!