Before you start building the world’s most awesome app, it’s a good idea to find out if people want it. I know it’s hard to believe others might not love your idea, but unfortunately, not everybody likes the same stuff. If we find out others have interest, you can avoid wasting time and money building something nobody wants.
The Facebook ads experiment will give you a high-level understanding around the following:
- if you have a strong unique value proposition
- if people desire to learn more about your idea, and
- which audience cares the most about it
This experiment will not give you proof - this experiment will give you confidence. Wouldn't it be nice to have confidence that someone other than your best friend likes your idea?
The Experiment
The Facebook ads experiment leverages Facebook’s large and diverse user base and its rich data about those users. With this data, you can target specific audiences and uncover the level of interest in your idea.
Each experiment will involve running 2 ads to A/B test against. Each ad has a design that highlights the unique value proposition and targets a specific audience. In each experiment, you isolate a variable to uncover which one garners interest. Those variables could be the unique value proposition, design, or audience.
Getting set up
In order to run this experiment you will need a few things:
- An idea for an app.
- At least one Unique Value Proposition (UVP) - it’s okay to have more than one.
- A well-defined audience that will be using your app.
- A Facebook account
- Optional but highly recommended: Facebook page *
- Optional but highly recommended: Splash page **
* If you want your ad to appear in the Facebooks users’ news feeds, you will need a Facebook page and a website. A simple Facebook page with an image and a single post saying coming soon will cover the page. A simple splash site stating your UVP and a "coming soon" will work.
**If you do create the splash page, you could incorporate a currency exchange. This would allow people to pay or sign up to be on a waiting list. Here is an example of a somewhat in-depth splash page to test a currency exchange http://www.nillyapp.com/
Running the ads
To run the ads, visit https://www.facebook.com/business/products/ads. You will need to log into your Facebook account to create the ads. I won’t go into great detail about how to do actually create the ads since the UI changes so often, but I will cover the main points.
Ad Type
Ideally, you want an ad that is meant to generate leads. You could create an ad that drives people to a Facebook page, but you wouldn’t benefit from the possible currency exchange of a splash page. Regardless of what type of ad you pick, make sure that you are charged per click. Clicks are a low-level form of validation.
Audience & Placements
Chose an audience that is narrow. A large vague audience makes it more difficult to determine who is interested in your idea. You have the option to run ads just on Facebook, on Instagram, or on their entire ad network. If you believe that one of those areas doesn’t apply to you, then feel free to cut it out. Just make sure that you are consistent with all the ads.
Budget & Schedule
Each ad should run for at least 7 days. I tend to spend about $100 to start. It's really up to you.
Design & Content
Design and content encompass the format, media, page, and links. You will want to create a simple ad that shows your UVP and has a call to action. Here is an example ad:
Taking action with each test
For the best results, you'll want to run at least 3 A/B tests. If the first 3 A/B tests return mixed results, you may want to run up to 6 A/B tests. I'll do my best to guide you from where to start and then based off of your results, where to go next.
Determining success
Bare in mind that these A/B test will return a single data point for you to compare results - clicks per impressions. You have to determine what success looks like. Different ideas should have different thresholds. If your idea is hard to execute and you need many users, the amount of clicks per impressions you need might be very high. If your idea is easy to execute and needs a few users, the amount of clicks per impressions you need might be low.
Try to answer these questions to determine a success threshold:
- How difficult do you think it will be to execute on your idea?
- How many users do you think you will need to be successful?
- How much competition is there for your idea?
- What is the average lifetime value of a customer?
- What is the average cost for this kind of product?
- What is the size of your audience?
Things to avoid
Avoid the desire to get clicks! Keep in mind that just getting clicks isn’t going to teach you anything - you need to know why you got the clicks. This is not about advertising best practices. You can throw words like “free” and “limited time” that will encourage clicks, but we don’t want that. We want an ad that tests only the value of your idea and the audiences that see it.
One of the most valuable outcomes is to identify that your idea doesn’t resonate with anybody. You save all the time and money it takes to build something nobody needs.
Experiments
Value proposition with style variable
This test will help you identify if your unique value proposition garners interest from a particular audience. You can run a single ad by itself, but I prefer to run 2 ads with the same content but different styles. This helps create a variable and a control to use for comparison.
Example:
Variable and controls
- same audience (control)
- same unique value proposition (control)
- same time frame & budget (control)
- different aesthetic style (variable)
Analyzing outcomes
If both ads get the same amount of clicks per impressions, and that amount of clicks per impressions makes you feel confident that enough people like your idea, then you've got some validation. You can stop experimenting if you'd like, or if you want to be more thorough, you can move on to the second test: testing the audience.
If one ad got way more clicks per impressions than the other, you could be concerned that the unique value proposition isn’t strong enough and that the ads visual style is what garnered the clicks. You should run a UVP A/B test to isolate the style as the enticing factor.
If neither ad gets enough clicks to make you happy, you should run this test again with a different UVP.
Value proposition with value proposition variable
This test is to help us identify the interest of our unique value proposition. Unlike the solo UPV a/b test, here we are going to pit two UVPs against each other. This test is great to run when you have competing ideas, or when you are not sure which aspect of your idea people are most interested in.
When creating this ad, make sure that the ad is simple and focuses mostly on the unique value proposition.
Example:
Variable and controls
- same audience (control)
- same time frame & budget (control)
- same aesthetic style (control)
- different unique value propositions (variable)
Analyzing outcomes
If one ad got way more clicks than the other, and the number of clicks per impressions makes you feel confident that this audience likes your idea, then you should be good to go. You know that one of your value propositions is good, and you know that an audience responds to it.
If both ads get the same amount of clicks per impressions, and that amount of clicks makes you confident that the idea is good and the audience likes it, you might want to do two things: one - run a solo UPV test with each UVP to see if the style was the draw, or two - pick the UVP that is coupled with the least risky to execute business plan.
If neither ad gets enough clicks per impressions to make you happy, you could try three things: pick two completely different UVPs and run this again, try a different UVP and run the solo UVP tests, run this test again with a different audience.
Value proposition with audience variable
The purpose of this experiment is to determine which audience responds best to the UVP. This experiment is best if you think your UVP is great, but the audiences you have tested thus far, are the wrong audiences.
This test should be run only after you have run either the value proposition with style variable or the value proposition with value proposition variable experiment. You will use the same value proposition and style for both ads, but now you’ll run them on two different audiences.
It might seem that you could run only one ad this time, and compare results to the previous experiment, but we would then be creating an experiment with two variables: audience and dates the ad ran. It is important to note that dates can affect click through rates - holidays, times of the year, days of the week, can all affect click rate. It is best to have only one variable, so you should run two ads for the most accurate results.
Variable and controls
- different audiences, one should be the same as a previous experiment (variable)
- same time frame & budget (control)
- same aesthetic style from the previous experiment (control)
- same unique value propositions from the previous experiment (control)
Analyzing outcomes
If one ad got way more clicks than the other, and the number of clicks per impressions makes you feel confident that this audience likes your idea, then you should be good to go. You know that one of your value propositions is good, and you know that an audience responds to it.
If both ads get the same amount of clicks per impressions, and that amount of clicks makes you confident that the idea is good and the audience likes it, you should be good to go. You know that more than one audience resonates with your value proposition.
If neither ad gets enough clicks per impressions to make you happy, you could try two things: pick a completely different UVP and run this again or pick a different UVP and run the UVP test.
Summary
The Facebook ad experiment is a great place to start to build confidence that your idea is resonating with a larger group of people. It is by no means perfect, but it is a good tool for de-risking your product idea up front. Remember, the biggest resource you have is your own time. Be sure to take steps up front to understand if your product can be successful.