Product development is a tricky endeavor because it requires a lot of research, investment, and resources to do well — yet, you don’t know if it’s going to bear fruit until much later in the cycle. Many organizations have learned this the hard way, spending incredible amounts of time and money on a new product idea, only to discover months or years later that it doesn’t actually work in the marketplace. This can be demoralizing, and in some cases — it can be an existential threat to the business.
As such, a whole industry has developed around idea validation to try and avoid this result by collecting valuable information ahead of time that acts as a predictor for the success (or lack thereof) of any new product. Typically, the way this is done is by getting out there and interacting with your customer base to better understand their needs and the potential demand for this product.
You might, for example, phone a portion of your customer base to run a basic survey or create an in-person focus group to discuss potential features and gather opinions. Either way — you’re collecting qualitative data in the form of customer opinions which are valuable in some contexts but can also be misleading if that’s the only thing you’re looking at.
The Limits of Qualitative Data
When you’re engaging with a customer one on one and asking their opinions on potential new products, it can be tempting to think that the customer’s views represent those of your target audience. After all, what better way could there be to get into the minds of your customers than to talk to one in the flesh! Unfortunately, this sort of thinking can be very misleading because when you’re dealing with qualitative data like this, there are various factors that limit its usefulness:
- Context. As humans, our behaviour varies widely due to context. Because we are social creatures, we have been conditioned to act in certain ways in certain situations because of the social norms in that space and the way we want to be perceived. This is to say that a customer in a formal focus group might share an opinion that doesn’t actually match reality because they feel the social pressure around them to respond in a certain way. They will often tell you what they think you want to hear. This is not helpful for product development, because you’re trying to figure what a customer is actually likely to do when you start asking for their credit card. That’s what really matters.
- Anecdotes. It’s not feasible to collect qualitative feedback from large proportions of your target audience and so you’re always going to have to rely on the few people you can get to respond appropriately. These conversations are going to feel more valuable than they actually are because, in essence, they are merely personal anecdotes. They don’t necessarily reflect the views of your entire customer base and in many cases, they can be completely opposite to what your most valuable customers actually want.
- Conflict Avoidance. When you are engaging with a customer, they are unlikely to give you negative feedback directly out of politeness. Most people don’t want to get into a position that might be difficult or uncomfortable so they will often be biased towards the positive rather than giving their actual honest opinion. This can give you a false sense of confidence that doesn’t actually play out when you bring the product to market and customers can act anonymously. That’s when you’ll see the true data as to whether they value what you’re building.
It’s clear that qualitative data can be useful in the early stages of idea conceptualization to come up with potential features, components, and functionality that might be valuable. However, the limitations above should be taken seriously — and qualitative opinions should not be used as a defining metric for deciding whether a certain product idea should be developed. There’s just too much at stake to hang your hat on a few anecdotes.
Instead, you should be using real, objective, quantitative data.
The Value of Quantitative Data
To get a more objective and statistically significant sense of whether a product will garner the demand that it needs, it is much more valuable to collect quantitative data at scale. This is achieved through running various demand tests without your customers knowing they are a part of one. This ensures that they are going to provide honest feedback because they don’t feel any personal attachment to the outcome, and so their purchase intent or lack thereof is going to be genuine.
The best way to collect this data is through the methodology of pretotyping which is a very simple, but effective way to identify product demand early before you invest significant time and resources. For example, you might aim to communicate your idea in its simplest form on a landing page and then measure the behaviour of visitors to that page to determine interest levels and various other metrics. With enough scale, you can get an accurate sense of how your customer base feels about the product and that is much more valuable as an indicator for whether a product is going to succeed.
After all, the numbers are what matter here — that’s where the truth lies. Don’t get carried away with the stories you hear from individual customers. What you’re trying to figure is how they are going to respond when they ask you if they can pay for it. And that truth only comes from quantitative data.
If you’re looking for a tool that enables you to validate your product idea using pretotyping, Horizon has all you need in one single platform to run a successful test and provides you with powerful and robust quantitative data that will accelerate product development decision-making.
.webp){kind=logo}
