In this chapter we will get a grasp on the fundamentals of what a fake door test is and the basics of how they work.
In later chapters we will look at how to setup your fake door tests in more detail.
What is a fake door test?
A fake door test is a market research method that gathers insights through the use landing pages. They are typically used before a product exists and validate ideas through data points such as conversion rates and cost-per-lead.
Let’s break down in more detail what fake door tests are.
Fake door tests are an amazing way to base any strategic product decision on - from testing product concepts to decision in the later lifecycle of a product, i.e. line extensions. A fake door test allows to gather high-quality insights for the decision making, before building any part of the product. They work by running landing pages and tracking how people use or interact with that page to gather behavioural data on purchase- and usage intent.
They allow you to quickly test (new) product concepts without putting a lot of time and money into developing a prototype, hence why they are sometimes called ‘Pretotypes’. You can even run multiple tests using them to develop the perfect product, through brand, pricing, target group, feature and value proposition testing. Thanks to the DNA of the insights, which are unbiased and behaviour-based, fake door tests allow to make any product decision with customer centricity at the very heart.
Takeaway
Fake door tests are:
A market research strategy to collect behavioural data
A way to test product concepts before they are built
Also called landing page tests, Pretotyping and painted-door tests.
How do fake door tests work?
Fake door tests work by directing people to a landing page for a product or feature that doesn’t exist yet, from this they gather data on the behaviour of people visiting the page.
A fake door test system would look like this:
It’s a simple setup that gathers essential behavioural data. It does this through the use of targeted advertising on platforms such as Facebook to bring people to your products landing page and tracking their engagement with the page.
The data gathered is based on the interaction with the advertising and the landing page. This creates a full view of the consumer as they progress through the funnel you’ve setup. The resulting data can tell you the full story of a user and the demand for that product.
A typical fake door test is all about testing multiple versions of a product to determine the most optimal version to develop and launch. This is done through the use of variants, for example, you might want to test the features of your product.
To do this, you would set up a landing page with each bucket of features on and direct people to those landing pages, this may be between 3 and 6 different variants. By doing this you get to collect data and discover the usage or purchase intent for each set, helping you identify the most optimal version to develop.
Takeaway
Fake door tests work by:
Building landing pages and directing traffic to them
Collect behavioural data at every stage of the user journey
Testing product concepts before they are built.
Chapter 2:
The Benefits of Fake Door Testing
In this chapter we will look at the reasons you should be running fake door tests in more detail, and how the testing process fits into your current consumer insights process.
Why should I run fake door tests?
You should be running fake door tests because of the unbiased behavioural data that they gather. They give you much more reliable insights that surveys and can validate any consumer insights you have gathered previously.
Fake door tests provide a variety of highly detailed behavioural data that can only be gathered through doing fake door tests, which can help you:
Develop consumer-centric products
Build better business cases for new products
Forecast a products cost and return better
And all of that, for a fast and cost effective setup.
Unlike other market research methods, fake door testing collects data from the direct behaviour of the users viewing the ads and visiting the landing pages. This translates into tangible demand data based on real-world data.
This is important because surveys and other consumer insights methods can only collect biased and opinion-based data. When looking at questions in a survey, it is upon the respondent to evaluate the product and their attitude towards it. This results in all kinds of biases being placed on the response, such as, confirmation bias, functional-fixedness bias, anchoring bias and more.
With fake door testing, you remove this problem as the user believes that the product is real and they are viewing a page to purchase or sign up to the product. This behavioural data can therefore not be biased as they are actively trying to signup to a product or attempting to purchase it.
2. Fake door testing validates your opinion-based data
Consumer insights data is typically opinion-based, which can lead to products being developed that shouldn’t be. Fake door testing helps resolve these issues through the behavioural data it provides.
Once you have gathered your initial consumer insights, you can setup fake door tests to validate the hypothesis you have from that initial data. For example, if you hypothesize that the product needs a certain set of features based on the opinion data you have gathered, you can test multiple sets of features in a fake door test to gather behavioural data and validate whether those features are required or not.
3. Fake door testing is quick and cost effective
Running surveys or focus groups requires a tremendous amount of setup to design the test, find participants, run the test, collect the data and then analyse each response to bring the story and themes out of the data.
Fake door testing on the other hand can be setup, ran and have perfectly viewable results within a little over a week. This is because the setup is very straightforward, as we will go through later on, and the data collected is based on behaviour, so it’s more robust..
All of this means it requires a lot less time and money to run, which means you can run more tests than ever to make sure all of your product decisions are valid, minimising the risk of failure. In fact, some companies are running over a 1000 tests a year to support all kinds of strategic product decisions.
Never miss expert insights and case studies on market success prediction with our monthly newsletter
You should use fake door tests before you build a new product or make any kind of product decision. They will validate your idea or concept through the data they gather and help you understand the market demand for that product decision, before going into development.
With fake door testing is also known as Pretotyping, which places it before a prototype. The idea here is that you are testing your product concept before creating any prototype or MVP to understand the market demand for that product. Helping you make strategic product decisions.
However, with fake door tests, you need a direction on what you are going to test. This means they can’t be used in the same way as traditional consumer insights that can gather a broad range of data. The best option with fake door tests is to be as focussed on testing particular aspects as possible.
This means it fits perfectly between your initial consumer insights like surveys and a prototype.
Chapter 3:
Pros and Cons of Fake Door Testing
In this chapter we look at the challenges that fake door testing has and how to overcome them.
What are the risks of fake door tests and how to avoid them
The risks of fake door tests are mainly around managing consumer expectations, because you are gathering data on a non-existing product, you have to make sure you are clear with the experiment otherwise you will be seen as lying or a scam.
The risks, while may seem damaging are easy to navigate if the proper procedures are put in place.
Conceived as false
The idea of fake door tests is that people don’t know it’s a test, which can lead to issues if they find out. They can feel cheated and that the brand has been lying to them.
This can easily be worked around by making sure you have good messaging in place during the test and if a user goes through the whole flow.
Examples of this could be:
The product is out of stock right now - keep updated via email
Great for physical products that could be bought online. It will seem like the demand for this product is high and so you’ve run out of stock, no problem there.
This product isn’t available yet - we will let you know when it’s ready
Great for digital products and apps.By placing these messages at the resolution stage, it managing expectations of the audience in a way that doesn’t blatantly say they have been part of a test.
Brand reputation
With the idea of fake door testing being seen as false, you might be thinking that has a negative effect on your brand reputation? When in fact, it can be the opposite, you just need to manage expectations well.
Most consumers will welcome being part of a test if it means a better product can be produced for them, it’s like backing a Kickstarter campaign. They put skin in the game to get the product they want made.
The key to overcoming this challenge is to resolve the test effectively so your consumers will get excited about the product.
Brand bias
One challenge that can come up with fake door testing, and really the only bias that can be attributed to testing with landing pages, is brand bias.
If you have a well known brand then results can have a bias on them because they recognise that brand and love it and will accept anything the brand makes. There are multiple ways to navigate this challenge through the use of off-brand and on-brand testing strategies.
Off-brandfake door testing
With this method of testing you can run tests using a new brand that does not exist yet or is not associated with your original business. This will allow any insights gathered to be clear of any brand bias and will focus on results attributable to the product only.
You can also use brand bias to your advantage through on-brand testing. This method works perfectly for running a brand test as you can test multiple brands to see which has the most favourable insights. The setup for this could be your main product brand vs. a subsidiary brand vs. a completely new brand to test which has the higher demand.
On-brand testing is also great for developing strategic product decisions.
One example of this would be if you already had a successful product in the market, but there is an opportunity to develop a new variation of it, it could be a different colour or size. Using on-brand tests would allow you to test the demand for this new version based on people already familiar with the existing product. Allowing you to develop a business case on if this version should be developed or not.
Whether using on or off-brand testing you will always need to have GDPR-compliant privacy policies and imprints. This means, whichever brand you use on the face of the landing page, your real and main company, i.e. the legal entity will always be visible in other areas such as the imprint.
Incognito testing allows you to test without disclosing your company in the imprint or as the provider of the landing pages. With our incognito testing service, Horizon provides the imprint and legal infrastructure for your ads and landing pages so that your company brand or legal entity is not associated with the research project by consumers and competitors. This allows you to get to a consumer-validated product decision even faster.
Not having laser focussed test parameters
Fake door tests are the perfect solution for gathering behavioural data for your product, if you setup the tests in a focussed manner.
If you have ever run A/B tests before you will understand the principle of making as few changes as possible between each test so that you can attribute any changes in performance to that change. It’s the same with fake door testing.
To get the optimum performance out of your tests you need to focus on what aspect, singular, you are testing for in each round of testing.
For example, you might want to test a new set of headphones. You want to test the pricing, features and colours of this product.
With fake door testing you can test for all of those aspects and get amazing results, but you need to separate those into different rounds of testing. This will help keep your results attributable.
You can see the diagram above how you flow from one test to the next, this is the ideal setup. You run a set of tests for pricing for instance, create multiple landing pages where the only change is the price, you get clean data. Then you move onto features, same idea, landing pages with the only change being different feature sets. Then repeat this for as many aspects as you want to test.
Chapter 4:
How to Run Fake Door Tests
In this chapter we will look at the process of running fake door tests, and how we setup our tests with a 10 step strategy.
How do I run fake door tests?
Running fake door tests requires you to decide the insights you want to gather, then you need to create a landing page and some adverts to direct people to that landing page. It’s a very simple process that doesn’t take long to setup and run.
We have a fully detailed blog about setting up and running fake door tests, but in essence there are 10 steps to making sure you have our fake door test running efficiently and effectively.
1. Decide on the mission for the test
When thinking about your fake door tests, you need to keep them laser focussed as mentioned earlier. To do this, you have to understand the purpose of the test, what the problem you are trying to solve with these tests is.
The way we handle this is to develop our testing mission, which can be phrased like this:
‘We are trying to decide X. To help us make this decision, we are going to test Y and analyse the data points Z.’
X is going to be the reason you are testing. It will frame the entire test and help you understand the data at the end. It shouldn’t have any influence on the end results.
‘We are trying to decide if we should launch a blue version of our product.’
Y is going define what aspect you are testing. This will help restrict the testing criteria to a narrow focus and giving you more meaningful data.
‘We are going to test 3 different colours of our product against one another’
Z is defining the data that is most relevant to answer X. What you are looking for here is the data points that are most likely going to answer your question.
‘Analyze the data point of double opt-in conversions’
To put this together into one test mission, it would look like this:
‘We are trying to decide if we should launch a blue version of our product. To help us make this decision, we are going to test 3 different colours of our product against one another and identify the most attractive ones by comparing conversion rates on the button click “buy now”’
2. Write your test hypothesis
Now we have our test mission, we need to write the direct hypothesis for the tests. This is going to be based around what we believe will happen based on our previous market research.
An example of a hypothesis could be:
‘From our consumer insights research, we believe that launching a blue version of our product will increase sales by 25%’
This statement supports the mission of the test, but is there so you have something to validate with the behavioural data gathered.
3. Design the fake door test parameters
We now have to look at the test mission and hypothesis and design how our test is going to run.
With our example mission of ‘testing 3 different colours’ we would need 3 versions of our landing page and advert, one for each colour. These are called variants, and you need to create one for each version of the product you are testing.
We advise you run a minimum of 2 variants and a maximum of 6. Any more than that and the data gets hard to read and the spread of data is too thin. Any less than 2 and you don’t have any valid comparison that can validate your hypothesis.
4. Design landing pages
Designing the landing pages is one of the most important areas to put some time and focus on. There are two key areas to think about with designing your landing pages:
4a. Minimum changes for maximum attribution
A key issue we have found is people making multiple changes to their landing pages between variants. This is the worst thing you can do because you can’t attribute positive or negative data to the changes you’ve made. The best option is to change only the elements that NEED to be changed between variants.
For instance, if you were running our example feature test, you would only change the colour of the headphones and any wording relating to the colour, everything else, every word, sentence and branding stays the same.
4b. Design it once, design it well
Because of the use of variants and keeping changes between to a minimum, you don’t need to design 3 individual landing pages, you design 1 page and replicate to make the minor changes. This is going to save you a lot of time, but means you should be spending more time on that first design to make sure it is a compelling page.
5. Create your ads
We need to get people to visit the landing page:
In a short amount of time
In enough quantity to provide valid data points
In our target audience
The best option for this is digital advertising such as Facebook ads. So we need to develop our ads for each test variant, in our example test we would have one ad design, with a variant for each colour of headphone.
It needs to promote the product like you would with an existing product, so have great ad copy and imagery or video to convince people to click on it.
6. Create your audience
Now you can pull it all together in the ad platform of your choice and create the audience to send your ads out to.
Typically you only need to setup one audience based on the current customers for your brand or an idea of who that audience is. You can run target group tests with fake door testing as well, which would mean you setup multiple audiences and send them to their various landing pages to see which performs better.
There are some guides to follow with your audience:
50,000 is a minimum sized audience to get significant results
2 to 3 million is a good sized audience for the best results
Include as many interests that are relevant to your market as possible
Exclude interests to make sure you aren’t getting outliers.
7. Create follow up content
Now we have our main flow together, you need to setup the content that comes after someone has completed your test by either clicking the button or filling in your form, basically making the ‘purchase’.
This is typically a:
Confirmation page or lightbox
Confirmation email
Resolution landing page
They will appear to the user in that order, so when they have ‘purchased’ they will be taken to the confirmation page that will say ‘sorry this is out of stock right now’, they will then get an email to confirm their email address and finally be taken to the resolution page to confirm that they have signed up to receive emails.
If they go through all of these steps it is classed as a double opt-in which can be categorised as a highly engaged user, meaning their data would be of most use.
8. Connect it all together
We now have everything ready to go, we just need to make sure we can collect the data in some way.
You can build it and connect it all together with a stream of tools like Unbounce, Mailchimp, Google Analytics, Facebook Ads and more. This will allow you to collect all of the data and then you will need to pull it all into one place and analyse it.
Or, shameless pitch, you could use Horizon for your fake door tests. It connects all of the tools up in one place and presents the data to you in an easily manageable format.
9. Run tests
All that’s left is to press play and get those tests running. Be aware of seasonality as this can effect your results as well.
You should aim to let the tests run for a week to allow for sufficient data collection.
10. Analyse insights
Once the results have been collected and returned you now need to review the data in whichever platform you have chosen to gather it all in.
We now need to refer back to our mission and hypothesis statements to analyse the right data and see if it has proven or disproven our initial thinking.
Never miss expert insights and case studies on market success prediction with our monthly newsletter
The lay out of your fake door test landing pages needs to have specific calls-to-action that drive action on your landing page. You also need to tell the consumer what the product is about in enough detail to make it seem like a real product.
Designing the perfect fake door test landing page is fairly simple, it just takes a little bit of time and thought.
There is a flow that works best for sales landing pages and fake door test landing pages are no different. They should be treated as a live sales page, the only difference is that instead of actually converting to sales, they are converting to purchase intent data.
Header
You need to develop a header section that is interesting both from a visual aspect and a copy aspect. These two go hand in hand to garner engagement from the user.
The content needs to be clear and concise, show an image of the product you are testing and tell the user how it will help them in their lives. Ultimately, the page is about them, not about you or your product.
You also want to add a call-to-action here to encourage people to complete the test.
Benefits
Next section talks about the benefits of the product and how they can affect the users lives.
The content here is really what will convince someone to ‘purchase’ the product and needs to be worded so it is convincing. This is the section that will get buy-in from the user so it is one of the most important areas.
Features
Now you can list any specific features that you want to pull out that could convince someone to purchase.
CTA
Next you will want a mid-page call to action. This is a prompt section to try and get them to purchase the product.
It should be highly visible and breakout from the page to encourage users to stop and read it, make sure it grabs their attention.
Process
Now you have the process section where you tell a user the process to using the product, it could also relate to one of the main benefits of the product as well.
Purchase
Finally you end with the purchase section where they give their details over to receive the product. Obviously they won’t because its ‘out of stock’ (speaking physical products), but this is the main conversion that is being tracked in your tests.
Chapter 5:
Fake Door Test Examples
In this chapter we will look at examples of how fake door tests can be used to gather unbiased behavioural data.
Fake door test examples
You can use fake door testing for many different reasons, aspects of a product and industries. Here are some examples of how you can use fake door tests.
Testing pricing sensitivity with fake door tests
Pricing sensitivity testing with fake door tests is the most reliable and efficient way to find the optimal pricing point for a product ahead of market release.
With methods such as using the Van Westendorp Price Sensitivity Meter, you ask people typically four questions to determine what price they are willing to pay for the product. This is a flawed system because the respondents aren’t actually putting any money down, or have any ‘skin in the game’. It relies on them to have an unbiased opinion on their reaction to actually wanting to buy the product.
With fake door testing however, they believe the product is real and are willing to give something to access that. That is a much higher indicator of the demand for a product at that pricing point because it is based on real world behaviour.
A test setup to test price sensitivity for 3 variants would be:
2 master ads that display the price of the product
3 variants of each ad
6 ads in total
1 master landing page
3 variants, 1 for each pricing point
The goal of the test would be to see which pricing point has the highest demand through the analysis of double opt-in conversions.
Never miss expert insights and case studies on market success prediction with our monthly newsletter
In this chapter we look at all the questions we get asked on a frequent basis.
Are fake door tests landing page tests?
Fake door tests are also called landing page tests, facade tests, smoke tests, painted door tests and even Pretotyping.
Fake door tests go by many names, but they all do the same thing.
Fake Door Testing Takeaways
Fake door tests are a consumer insights method to gather behavioural data through landing pages.
They work by directing people to landing pages and gather data by tracking their interactions with it.
The benefits of them are being able to gather behavioural data and validate your opinion based data like surveys.
You should run them before you build a product or are making any strategic product decision, but after your initial consumer insights testing.
The risks of fake doors tests are consumers feeling ‘scammed’ if the test hasn’t been communicated well. You shouldn’t be scared of admitting it’s a test to benefit them, but equally you don’t have to let them know.
The process of running a fake door test is to develop what you are wanting to test, or your test mission, designing your content for the test and analysing the data afterwards to validate the reason for testing.
Your fake door test should always include a reason for testing, without it you will not understand what the data means and how to act upon it.
Fake door tests can also be called Pretotyping, landing page tests, façade tests, smoke tests, painted door tests.
.webp){kind=logo}
