Customer research
How talking to users will accelerate your growth
User research reveals obstacles and opportunities that intuition alone cannot uncover, helping teams create more useful and effective experiences.
Very few things are more valuable to Marketing and UX professionals than user testing.
I clearly remember the first usability test I coordinated, almost 20 years ago.
It was transformative to see how the lessons from that morning, spent interacting with real users, helped us create an e-commerce experience that was far ahead of the competition at the time.
That story has repeated itself across several projects since then.
Over the years, the practice known as usability testing has gained better names and broader scopes: User Testing, User Interviews, UX Research, Prototype Testing...
These names are better because a user can help you understand much more than the usability of an interface. They can offer insights about product positioning, differentiators, copywriting, the roadmap, and much more.
Even decades later, the process that lets users tell (or show) you what you need to do is still invaluable.
Why is user testing so powerful?
Because nothing replaces the experience of watching real users get frustrated with your product (or your Marketing as a whole).
In a meeting room, there is always a considerable degree of abstraction. Even in the most data-driven companies, discussions often drift into the realm of “I think”:
📣 “I think this is not a high-priority problem.”
📣 “I think this part turned out well.”
📣 “I think it is clear why we are better than the competition.”
📣 “I think the page already has all the relevant information.”
And in this world of assumptions, the HiPPO (Highest Paid Person’s Opinion) always wins.
By the way, if you need to convince leaders to make improvements, user testing is also very effective.
Presenting quantitative data to justify a suggestion of yours is “OK.” But showing a few seconds of screen recording of a single user failing miserably with an interface often does more to engage a leader and get changes made.
Why? Because, as I said earlier, nothing replaces the experience of seeing users get frustrated with your brand.
How to run user tests
Types of tests
Tests can be in person or remote. With all the resources available today, you will probably move faster without losing anything relevant by choosing remote testing.
You also need to decide between moderated and unmoderated tests:
Moderated: you, as the moderator, follow the user in real time.
Unmoderated: the user receives written instructions, and their actions are recorded for later analysis.
Most of the tests I run are unmoderated. They let us move much faster and, as long as the instructions are clear, work almost as well as moderated tests.
How many users should I recruit?
For the kind of qualitative test we are discussing, it usually is not worth involving more than five participants.
A study by NN Group shows that there is very little correlation between the number of users in a testing session and the number of insights found.
In other words, you can identify most of the insights “available” with very few participants.
If you need a large number of users, other methods may be more efficient, such as heatmaps, screen recordings, or five-second tests. I will talk more about these methods in future newsletter editions. 🙂
How should I recruit users?
Anyone has the potential to deliver valuable insights in a test. But I recommend prioritizing people who truly represent your target audience. Here is the order I use:
Excellent: a real user or customer, or someone who represents your ICP (Ideal Customer Profile).
Good: people who resemble your ideal audience and could become your customers.
OK: any “ordinary” person.
There are several ways to recruit these people. Two simple and effective options are:
Add a pop-up to your website or product inviting users to answer a few questions about themselves. If their answers indicate that they fit the profile you are looking for, contact them and invite them to the test. A combination of tools such as Hotjar and Google Forms makes this process quick and easy.
Email your users and invite them to answer the questionnaire, following a similar process.
Here is how Nubank did exactly that in a recent email:
Guidance for participants
Much of a user test’s quality depends on the guidance you give participants.
Whether the test is moderated or unmoderated, it is important to tell users things like:
They are not being tested; the product or interface they will see is.
They should be as honest and transparent as possible. When I am not the creator of what is being tested, I make a point of telling users so they feel even more comfortable being honest.
They should say their reactions and questions out loud.
The test will be recorded, but the recording will not be shared publicly.
For more detail, I strongly recommend Steve Krug’s usability testing scripts, perhaps the world’s leading reference on the subject.
Running the test
After the initial guidance, it is time to get started.
The exact process will vary depending on what you want to learn. But I usually like to have three moments:
As soon as the user opens the website or product, I want to hear their first impressions. What is this interface about? What can you do here? What is your initial feeling? Understanding those first reactions is crucial because many users will use the first few seconds of interaction to decide whether you deserve their attention.
I ask them to complete the most important actions on the site (or the actions I am evaluating). For example: buy a product, find information, complete onboarding, reach the aha moment, and so on.
I try to understand the user’s view of the experience as a whole. What was especially good or bad? Were expectations met and questions answered? What impression does the brand or product leave?
Analyzing the results
After all the tests are over, I recommend saving the raw materials (recordings and all notes). With help from AI, create an executive summary with the key insights and the recording clips related to each one. As I said earlier, this can help engage other stakeholders.
Many of the insights in your analysis will be obvious, handed to you by the users. That is the point, right?
To make sure nothing important was missed, pay special attention to these questions:
Did users quickly understand where they were? If they had trouble understanding where they were and what they could do, they will probably abandon the process in real life.
Do users understand your value proposition? If they did not understand the benefit offered by the product, feature, or page, you need to fix that immediately.
Did users miss any information or an answer to an objection? This is a classic landing page problem, especially on pages designed to “get straight to the point.” Many users have an objection in mind, and pretending it does not exist will not help. You need to present an effective counterargument.
Did users fail to notice an essential element? They may not notice (or scroll far enough to find) the killer differentiator or powerful social proof you worked so hard to create.
Did users misunderstand any information? The fact that information is present in your copy and read by a user does not mean it was understood correctly.
Did users believe what they read? As with the previous point, saying something does not guarantee that users will believe it. If in doubt, ask users directly what they did or did not believe in the interface. You may be surprised.
User tests do not have to be complicated
Despite all the tips I have shared in this article to get the most out of user testing, I want to be clear: tests definitely do not need to be complex or detailed to create value.
I have a story that became well known back when I was at Supersonic.
More than 10 years ago, when we were starting work for a large real estate company in Belo Horizonte, I took advantage of a visit to my parents and simply asked them to use the website to find the ideal property for them (something they were actively looking for at the time).
In 30 minutes, just by watching my parents, I found several insights that were essential to one of Supersonic’s first major case studies, which increased the client’s conversion rate by more than seven times.
Try it yourself. Start simple if you need to. I guarantee you will like the results.