Jesse Stevens
Jesse Stevens
Senior Product Strategist

Experience

Guide to user testing – types & timeline

The only way to get a product right is through user testing. 

Assumptions are a wonderful place to begin designing products, but product solutions must be evaluated with real users to see if these assumptions solve the user’s core needs.

In fact, pairing user testing alongside market and product fit is the trifecta in building your product strategy. When teams do these things, they are more efficient in every other area of development, accelerating product roadmaps and finding ROI faster. 

This knowledge only comes from solving your user’s underlying problems, which comes from testing.

In this guide to user testing, we’ll take you through the types of user testing, when each should be used, and how to use them in a realistic product life cycle. 

user testing guide focus group

Types of User Tests 

There are two categories of user testing, qualitative vs quantitative.

Qualitative testing is learning about a user’s interpretations and feelings, while quantitative is number-based and measurable. An example might be interviewing a single user about their needs and motivations vs A/B testing a single interface element or user flow. 

When conducting your quantitative data, be sure to account for statistical significance, otherwise, your results could be by chance and not a real consensus of users. 

For the best results, you need both qualitative and quantitative testing. Here are the top user tests and how they help.  

Usability testing

One of the most important and popular tests, usability testing asks a user to complete a particular task with feedback while a moderator watches and documents. These kinds of tests are essential because they put a real human behind a device.

With this, you can learn about their problems, how they navigate, how they search for information, and what they like/dislike about a certain product. 

Direct observation

With direct observation, you observe without interaction. Seeing a user easily find information or struggle to complete a sign-up can shed light on how easy your application is to use. 

Prototyping

With prototyping, your user is reviewing a prototype, not a full-fledged app. Sometimes these prototypes are low fidelity wireframes, sometimes they’re high fidelity and clickable. Since development is expensive and time-consuming, you don’t want to begin developing without testing some kind of prototype. 

A/B testing

Comparing two options to find the superior one can be a fast way to iterate. Is a user more likely to respond to a “Contact Us” CTA or “Let’s Talk?” Does a blue button work better than a red button? With A/B testing, you can get useful data and make informed decisions. 

Treejack testing

Treejack testing is ideal for information architecture testing, which is essential for any complex website or product. You can ask questions like, “if you wanted to find the location nearest to you, what page would you click on?” and then record all interactions. 

Eye-tracking and heat mapping

Eye-tracking and heat map testing can be expensive, but it’s worthwhile for some products. With these tests, you can measure where your users are looking and where they’re clicking on your app– to understand if you are providing the right information in the right place. 

In-person vs remote

You can conduct all of these tests in-person, remote, or automatically. Remember to take these different environments into account when analyzing data, because different atmospheres can affect users’ opinions and how they interact with technology. 

user testing scaled 1

When & how often should a team test?

Like most answers to design-related questions, “It depends.” 

Minimum viable products (MVPs) need different testing compared to a legacy app. A single-page web app needs different testing versus a large e-commerce store. So, unfortunately, there is no one size fits all solution. 

A robust discovery session is a good practice. This process helps strategists understand the problem space and craft a program to tackle the issue. It also helps designers determine how to test and when. 

However, we can provide some general guidelines which are:

Test often

A strong design process starts with discovery and definition, before diving into ideation. Then, repetition is essential. 

Ideate – Prototype – Test as needed

Ideate – Prototype – Test as needed  

Every time you iterate, you need to ask yourself, “should we test this?” While a single color swap won’t require standalone testing, a new feature will. 

You also have a budget to consider. Testing every small thing will cost you time and money, so have discussions, ask questions, and keep testing on your team’s mind. 

Lightweight iterations = quantitative tests 

There are always exceptions to this rule, but it’s generally true. 

A “lightweight iteration” is difficult to define, but it’s likely something that can be A/B tested and decided upon with analytics. 

Heavyweight iterations = qualitative tests 

Also exceptions here, but when you do more intensive iterations, like developing new features or launching to a new audience, you need to sit down and talk to users. Understanding not just what they like, but why they like it can be incredibly insightful. 

Internal vs external testing 

If you’re in the very beginning stages of product development (think wireframes), we recommend testing with internal stakeholders. 

Don’t go outside of your organization until you have high-fidelity prototypes. After all, every product needs stakeholders to sign off before digging into the design. 

user testing emoney visual dept scaled

Real product testing: eMoney 

Rooted in comprehensive financial planning, eMoney’s products strengthen client relationships, streamline business operations, and drive overall growth. 

eMoney’s goal was to extend its product to a new audience: millennials. 

Together with eMoney, we came up with five concepts that could potentially reach their target of millennial users. We then kicked off a series of design sprints, where we built out prototypes for eMoney concepts and tested them with real users. It became clear that users gravitated towards one of the concepts more than others, which was a mobile app that could provide financial wellness advice and complement the existing eMoney software-as-a-service platform. 

This was the beginning of what became eMoney Incentive.

We learned through testing and development that Incentive appealed to a wider audience beyond millennials.

Because of this rigorous testing, eMoney’s pilot program created a strong pipeline of interested users, including current users, retirement plan advisors, and employers.  

If your product needs a user testing strategy put in place, reach out to the designers and product innovators at DEPT®. 

Sign up for the DEPT® product newsletter

Insights

Stay in touch!

Personalize your experience

We use functional cookies to make the website work properly, analytical cookies to measure your behavior and marketing cookies for ads- and content personalization. We collect data on how you use our website to make our website easier to use, but also to tailor or personalize communication in advertisements, on our website or apps. Data collected through marketing cookies is also shared with third parties. By clicking accept you agree to this. More information? Read our cookie policy.