The Dyson Zone—a wearable air purifier combined with noise-cancelling headphones—illustrates the potential risks of focusing on R&D-driven innovation without early consumer validation. Despite years of development and the creation of 500 prototypes, the product initially launched at $949, only to be repositioned later as noise-cancelling headphones at a significantly lower price point due to limited consumer interest. This R&D-driven innovation by Dyson highlights where a Painted Door Test could have helped the team identify consumer interest and willingness to pay earlier. It might have prevented the spending of millions on internal research and development without first validating market demand. On the other hand, there are also limitations when it comes to testing market demand early-stage via Painted Door Tests. In this article, we will delve into both when to use a Painted Door Test for consumer research and market success prediction, and when not to.

‍

What are Painted Door Tests?

Before we dive into the restrictions and potentials of Painted Door Tests, let’s clarify what a Painted Door Test is and where they originate from. If you already know the drill, you can just skip this section.

Painted Door Tests are based on the Pretotyping Methodology introduced at scale by Google

The core of this Pretotyping method, i.e. Painted Door Tests, is to acquire first-party data on consumer behaviour by presenting the consumer with the option to engage with a product or service. This allows you to measure their actions—such as usage or purchase intent—resulting in more decisions being tested and ultimately leading to higher success rates.​

‍

4 out of 10 People act differently from their stated behaviour in shopping.

Or differently put: Actions speak louder than words. The discrepancy between what individuals state and the actions they ultimately take is especially observable in the shopping behaviour of consumers. A study among US online shoppers found that 38% of shoppers are not actually following their previously stated behaviour (Say-do gap among U.S. active online shoppers 2022, Statista 2023; https://newproductsuccess.org/new-product-failure-rates-2013-jpim-30-pp-976-979/). This phenomenon called The Say-Do Gap poses several threats for market researchers. If you’re interested in the topic, you can read our blog article about the challenges of the Say-Do Gap for researchers and how to navigate through it here.

In brief, Painted Door Tests can help bridge the Say-Do gap by validating hypotheses based on consumer opinions or initial market research with actual purchase intent insights data from consumers interacting with real-life buying journeys.

‍

How Painted Door Tests Work

Painted Door Tests, sometimes referred to as Fake Door Tests, Smoke Tests, or Landing Page Tests, are a powerful tool for gauging consumer interest in new products. These tests engage potential customers by presenting value propositions and tracking their behaviour across the typical buying stages—awareness, consideration, and decision. By measuring interest and engagement at key points, they offer a quick, efficient, and resource-light way to validate market demand before launching a product.

Typical Use Cases of Painted Door Tests:

• Assessing Market Demand: Is there a real need and interest among my target audience for a new product or service?
• Evaluating Purchase Intent: Will consumers actually buy the product or service?
• Determining Price Sensitivity: What price are consumers willing to pay, and when does the demand begin to drop? For more insights, check out our blog on how pricing can impact operating profits and how Painted Door Tests can assist in optimising it.
• Identifying Profitable Target Markets: Which market or region shows the greatest interest in a product or service, measured by purchase intent?
• Choosing the Best Features, Value Proposition, or Branding: Which branding or feature options resonate most with consumers in a real-world context?

What you need to run a Painted Door Test

‍

For more information on how Painted Door Tests work in detail and how they differ from usual A/B testing, you can check our blog article here.

‍

Painted Door Tests Can Be Applied To Existing And New Product Developments

Painted Door Tests offer strategic flexibility across both new and existing products by validating strategic decisions in pricing, features, value proposition, branding, and target groups. 

For new products, they help businesses assess financial viability of the overall business case by collecting insights on customer acquisition costs, identifying highest-demanded product features, and discovering the most resonating value proposition with real consumers before actually developing or entering the market. The teams that conduct Painted Door Tests for new products are most commonly innovation, new product development or corporate development/strategy units.

In the context of existing products, these Painted Door Tests guide enhancements such as optimising pricing strategies for an increase in profits, adding new features, variations or line extensions, or even validating the necessity of rebranding efforts with the target audience. Product managers or market researchers use Painted Door Tests for strategic decisions on existing products.

By targeting narrow and detailed market segments through social or search ads, they also enable innovation, product and market research teams to understand where the highest consumer interest lies, facilitating more informed decisions for both product development and market expansion strategies. This method ensures that every product decision is backed by real consumer behaviour, minimising risk of failure and sunk costs, and maximising the chances of market success for every product decision.

If you’re interested in use cases of Horizon clients for predicting market success of strategic product decisions for new and existing products, we are linking more of Horizon's use cases here.

‍

Data Points Collected To Validate Purchase Intent By Consumers Through Painted Door Tests

In a Painted Door Test, various data points along the customer journey of the test can be leveraged to understand consumer behaviour and gauging market demand for the product decision at hand. From audience sourcing through platforms like Meta and Google to call-to-action button clicks and email conversions after resolution of the test, each stage of the funnel can provide valuable insights. Metrics such as click-through rates (CTR), unique landing page visitors, and call-to-action conversion rates (CTA) clicks offer a glimpse into initial interest and can be used to derive more concise data points than traditional research (survey, focus groups, etc.) for the business case to make the strategic product decision. As consumers proceed through the journey, selecting products, aiming to add them to the cart, opting in with their emails, and confirming interest, businesses can measure close to real-life engagement and intent. These stages help paint a comprehensive picture of market demand, allowing companies to make informed decisions on whether to proceed with product development at all or in which way. Below you can see where in the customer journey of a best-practice Painted Door Test the data points you need can be collected.

‍

If you’re looking for more in-depth information on data that you can gather with a Painted Door Test, we have created a detailed blog article in the past around the topic here.

‍

When Not To Use Painted Door Tests

Alright - we have covered when Painted Door Tests can actually help teams in innovation, product or market research to predict market success of strategic product decisions. Let’s cover the restrictions. 

While Painted Door Tests can be a powerful tool for validating market demand, consumer interest and willingness to buy with real consumers pre-market, they are not always the right fit for every situation. In certain contexts, using these tests could lead to misleading insights. Understanding when not to rely on Painted Door Tests is just as crucial to increase the ROI of your product decisions as being aware of when they are most useful. Below, we explore specific scenarios where teams should avoid this method to ensure they make the best decisions for their product development process.

‍

1) Products Or Services Requiring Complex And Lengthy Decision Making

One might wonder why one of the key values of a Painted Door Test, acquiring the audience from what is probably the largest panel in the world,  social media and search engines, would not work for LinkedIn ads/B2B products. In complex decision-making environments like B2B markets, Painted Door Tests often fall short of capturing the true stages of the buying process. These markets typically involve multiple decision-makers from various departments, longer sales cycles, and extensive evaluation of various factors such as long-term ROI, compatibility with existing systems, legal compliance, or even impact on social and environmental responsibility. A Painted Door Test, with its short field time of 7-14 days won’t reflect the deeper, multi-layered considerations that drive final purchase decisions for say a SaaS product for finance departments. Therefore, in environments where decisions are not made by a single individual but by teams or departments, Painted Door Tests may provide skewed or incomplete insights.

2) Products Or Services With High Prices and Investment-Characteristics

For products with highly unique customer journeys and particularly high-priced items, Painted Door Tests can also be misleading. In the case of high-priced consumer goods like cars or complex investment plans, customers often undergo an extensive research phase before making a decision. They consider numerous factors, including financing options, long-term value, and even testing the product in real life. This lengthy, careful decision-making process doesn’t align with the quick, impulse-driven behaviour typically measured in a Painted Door Test.

3) Products With Strong Restrictions On Digital Ads Platforms

Dating, tobacco, alcohol - products that face strict regulations on digital advertising platforms are poor or impossible candidates for Painted Door Tests. These restrictions can severely limit the reach and effectiveness of online ads, skewing test results and preventing teams from collecting a significant sample size, leading to unreliable data. In such industries, advertising policies on platforms like Google and Meta often require careful navigation, and campaigns are subject to rigorous approval processes - for good reason. If teams still manage to publish advertisements, this can result in limited audience exposure or messaging that doesn’t fully convey the brand’s or tested product’s value and benefits, more precisely you’re not able to communicate the test stimulus properly. As a result, the data gathered through Painted Door Tests may not reflect true consumer interest or market demand, making it mostly impossible to draw meaningful conclusions to incorporate into the business case and making the strategic product decision with real-life/first-party consumer data.

‍

Effective Methods For Market Research When You’re Not Using Painted Door Tests

Conjoint Analysis:

+ Provides detailed insights and can help in predicting market acceptance

- Consumes time, can be complex, and is usually expensive

Panel Research:

+ Can track consumer changes over time and offers in-depth insights

- Usually risk of panel conditioning

Surveys:

+ Scalable, easy to analyse, and usually have a wide reach

- Limited depth, potential bias, and low response rates

Focus Groups:

+ Can explore nuances in consumer preferences and interests

- Time-consuming, hardly scalable, and influenced by group dynamics

Ethnographic Research:

+ Provides deep contextual insights and uncovers hidden behaviours

- Subject to researcher bias and subjectivity

Sentiment Analysis:

+ Quickly processes large volumes of data and uncovers trends

- Can miss nuances and is dependent on data quality

‍

If you’re looking for a partner for your qualitative research, the Horizon team would recommend to take a look at Bolt Insight. Bolt’s research AI automatically identifies consumers, conducts 30-minute individual in-depth interviews, and provides you with an actionable report summary, all within 24 hours.

‍

Painted Door Tests can effectively validate consumer interest, market demand, and purchase intent in many situations. However, they are not suitable for every product or market context. Complex decision-making processes, unique customer journeys, and industries with strict advertising regulations are some situations where other research methods might be more appropriate. By understanding the potential and limitations of Painted Door Tests, teams can make better product decisions and combine this method with more qualitative approaches such as conjoint analysis, panel research, or ethnographic studies to gain comprehensive insights into consumer behaviour.