01-May-2023
At my organization, like many others, people often use the term 'UT' very loosely. Suhail Gupta a colleague and friend described it very correctly in a card set he made with all of Urban Company’s colloquial terms.
As the only user experience researcher on the team, I decided to address this by setting up a 30-minute workshop on the topic. In this blog, I'll describe how I convinced the team to be more specific in their use of the term.
The problem with generalisation
We've all experienced situations where the meaning of a word or phrase is misunderstood due to generalization. For example, saying "I crashed last night" to your mother might make her think you were in a car accident, when you really meant that you were exhausted and went to sleep.
Similarly, not all teapots are created equal, and "blonde" can mean different things to different people.
These examples demonstrate the problem with generalization. It can lead to misunderstandings, loss of important context, and confusion when a term is applied to diverse situations to mean a diverse set of things.
The term 'UT' is one such example that has been generalized, with my team using it 253 times in messages and 67 times in various documents in the past year alone.
The meaning of ‘UT’
To address this, I asked my team members what 'UT' meant to them. The responses varied between 'user testing,' 'usability testing,' and even 'union territories' (which generated a few laughs). This confirmed that the term had indeed been generalised.
After some discussion, we established that in the context of our work at Urban Company, 'UT' mostly refers to usability testing. However, it's important to check which type of testing is meant whenever the term is used. With this understanding, the goal of our workshop was to create a shared understanding of usability testing in our team to ensure clear communication.
Types of usability tests at Urban Company
At Urban Company, we conduct usability tests before and after revamping specific categories such as women's spa or laser hair reduction. We also test product capabilities such as routines (a capability to automate services that repeat every month) and packages (a pre-curated set of services that users can choose from). We conduct similar tests on the partner-side app as well.
The generalisation of insights
However, generalization doesn't just apply to the term 'UT.' It can also creep into the insights generated from loosely designed usability tests. Generalized insights can be dangerous because they can lead teams in different directions. For example, with our routines capability there was an insight such as the one below. Teams considered changing the term to 'subscription' to improve user understanding, while others wanted to create a detailed video to teach users about routines.
Each of these directions justify the insight above however, when you make the insight above a little bit more specific the scenario completely changes.
To avoid these misunderstandings, it's important to define insights before starting to solve for them. Insights from usability tests can be either discovery problems (e.g., participants didn't see or find the subject of the test) or comprehension problems (e.g., participants didn't understand the subject) or engagement problems (e.g., participants saw, skimmed but did not spend time on the subject). The solutions to these three types of problems are different, so it's imperative to define the insight before attempting to solve it.
Note: Follow the order of issues while identifying the kind of issue your insight falls under.
Let’s practice with a few examples. Could you note which type of problem each of the following usability issue falls under? Answers to these can be found at the end of the blog.
Qualifying criterias for usability tests
Finally, it is also important to understand that to maintain the quality of insights, it is important to maintain a few qualifying standards for usability tests. At Urban Company, we follow an extensive guide to UTs for content revamps. This guide includes standards such as ensuring that the tasks are realistic and relevant, recruiting participants that match the user persona, conducting tests in a controlled environment, and using a variety of metrics to evaluate the test results. However let me summarize some basic criteria that one can note to maintain the standards of an usability test.
01: Qualifying factors while recruiting
02: Qualifying factors while conducting
03: Qualifying factors for analysis
I'm sure you're all thinking like my audience at the workshop, that it's a good idea to do a quick check of your ideas with anyone you can find. And you're right, there is some merit to that. However, let's be honest and call those 'gut checks'.
Answers to the practice examples:
Example 1: Comprehension problem
Example 2: Discovery problem
Example 3: Engagement problem