Qualitative or quantitative usability tests: what’s the difference?

Birth and evolution of usability tests, the different ways of carrying them out and the comparison between qualitative and quantitative approaches

The usability test is an experience analysis research methodology based on the direct observation of a person’s behavior and interaction with a product.

In this article we see how they were born and the two main types.

Usability test: birth and evolution

Usability test was born in the engineering field in the 1970s, in a context where human-machine interfaces were becoming increasingly complex and sophisticated. In particular, the term “usability” was coined by Jakob Nielsen, one of the pioneers of usability engineering.

Initially, usability tests were mainly used in the context of computer systems design, to evaluate the ease of use of an interface and to identify any problems or difficulties of use. In the following years, user tests have been refined and extended to other fields, such as the design of products and services.

This type of user testing can be carried out today in different ways. In fact, with “usability test” we mean multiple test activities which however differ significantly in the type of research and therefore also in the type of output. It follows that not all usability tests are the same and that not everyone tests usability in the same way.

Qualitative or quantitative usability tests: definition, features and benefits

To clarify this definition we will focus on digital products for simplicity, but we know that tests can also be carried out on physical, hybrid or even stores and environments.

Qualitative usability tests

At their birth, usability tests were considered as a qualitative tool. Nielsen himself will define that with 5-8 participants about 80% of the usability barriers can be found. The qualitative approach makes it possible to understand not only “what happens”, but also to have indications on “why” certain choices by the user happen.

This activity makes it possible to optimize the investment, not just identifying the problem, but starting a process of understanding the motivation that generated it. This approach allows greater efficiency in the resolution phase, because it is possible to avoid all those solutions which, although different, do not solve the key problem of the future user’s experience.

Quantitative usability tests

Quantitative usability tests developed later to reach more people with a tool that allows them to be compared. These are activities provided remotely in most situations, by sending a link, which requires that the activities be carried out on a deferred basis by the participants.

Qualitative vs Quantitative Usability Tests: A Comparison

Defining upstream whether one tool is better than another would lead to partial conclusions. At the center of our approach there are always people, with their specific needs. Our goal is to deploy the tools capable of offering the answers the company needs from time to time.

In general, we can say that quantitative tests generally have a lower cost than tests of a qualitative nature and offer the possibility of applying statistics, but reach minimum levels of depth. In most cases they only answer the question “what happens”, without offering information on “why”.

Quantitative tests have the advantage of allowing an objective comparison. This approach can be useful when large companies need to compare the usability of digital products to understand macro differences in very different cultures, in a short time. In all other cases, we would like to suggest using usability tests in a qualitative way, so as to obtain higher quality information and enhance the investigation characteristics of a tool as it was conceived.