The Ultimate Guide on How to AB Split Test your Amazon Listing
This article is not covering Amazon Experiments; instead, we are focusing on how to run AB Split testing Surveys for your Amazon listing with tools like Sellametrics.
If you're still interested, read on, and you won't be disappointed.
If youre selling on amazon for some time, you know that a well optimize listing is one of the biggest assets you can have in this bussines, and truth to be told, not 2 listings are the same, what worked for somebody else, may not work for your product. Thats why your goal should be to make decisions backed with real data and remove the guesswork.
As a marketing strategy, split testing forms the foundation for conducting advanced scientific testing. Therefore, a solid split testing approach should be at the heart of every conversion rate optimization (CRO) campaign, especially for Amazon Sellers.
Split testing can give you an edge over your competition by removing the confusion and clarifying your message; remember confusion damages the conversion rate. Also, if you're optimizing your product copy, title, images, or bullets – a split test will help you identify versions with a higher potential to engage your audience. More engagements equal more sales.
While running a split test isn't sophisticated, the challenge is understanding how and what to test. That's what this guide is all about. First, we dive into split testing, deconstructing the various terms that confuse most marketers and CRO strategists.
Let’s jump right in.
Contents:
What is A/B Split testing?
Split testing, also commonly referred to as A/B testing, is a technique that involves comparing two creatives to determine which version is more appealing or performs the best. The goal of split testing is to impact your conversion rate success metrics. It can be click-through rate, user sessions, sales and revenue, value per visit, and so on.
Anything visual can be split testested, For example for websites, you do this by sending a portion of traffic to a web page, where each visitor is shown a different experience, at random. You then track their experiences and measure their impact on the outcome.
In the Amazon space we run AB split tests with specialized tools like Sellametrics Surveys, you upload 2+ versions of a creative and run it againsts a targeted demographic.
In split testing, there are two + variations, hence the initials A/B. ‘A’ represents one version while ‘B’ represents a different version.
In scientific words, Version A is the original version, also known as the ‘Control’. It is the original, unaltered page that you wish to compare to a different version. Version B is known as the ‘Variation’ or ‘Treatment’. It represents the improved version of the page and challenges the control version.
The Different types of split testing
Many marketers seem to use the terms split testing and A/B testing interchangeably. But the two terms don’t mean the same thing and the concept is different.
That’s why we need to make a clear distinction between split testing and A/B testing.
Let’s clarify what each term means.
Split testing.
In split testing, you manually create the different versions you would like your visitors (sampling group) to see. Here, you’re not just making a few changes to one variation. You create totally unique versions. Also, users get to see the same versions, allowing you to gather insights and pick a winner. That means, you compare the activity for both versions, simultaneously.
Here is how split testing works:
You create several completely different logos or landing pages that you show to a portion of visitors at the same time, as one total experience. You can compare two or more versions, which your audience will see as one total view.
You would then redirect your traffic to both the “Control” version and the variation. Here, both the control and variations have different designs. It could be a different image, headline, copy, call-to-action button or URL.
“If you’re testing multiple landing pages with different headlines or CTAS, you’re doing split testing.”
One approach in split testing is to show users both version A and version B when they land on the page. But some split testing tools have an option that lets marketers host the landing pages on a different URL for comparison. With this approach, you run one version for some time, say one week – and the other version for another week. Then you would compare the results from the two sessions.
Split testing tools leverage on a team of respondents, who provide detailed, written feedback on which version they like. Once the poll completes, your split testing software will present the winner. A winner is the variation that got the most votes from those participating in the survey.
Once the test completes, you can use the winning page or logo to run an actual A/B test. Here, you would use the winner of your split test as the Control variable. Then make minor improvements to the control like changing the placement of your CTA.
N/B: Split testing can involve comparing more than two options. You can have A/B/C/D options.
Classical A/B testing
In contrast, an actual A/B test uses scientific principles to compare the control and the variation. It involves comparing the control version to the variation, but in this case you make minor changes to a specific parameter. You could also make a variety of small, single changes to the control version. Here, the versions being tested do not have a major difference.
As an example, you test the impact of changing the color of the checkout button from green to blue. Here, you’re testing the impact of changing one element on a single page.
Let’s say you start by doing a split test for two landing pages and you get a winner. If you wish to fine tune the winning version to determine which elements will positively impact conversion metrics, performing A/B testing would be more reliable. You would test a single element at a time, where you’d have the control and one variation.
It’s worth noting, in randomized controlled A/B tests, you’re not gathering feedback from respondents. Rather, you’ll be testing with your website traffic to track consumer behavior and optimize for their experiences.
Multivariate testing
Also, you can change many components simultaneously, which is an advanced type of A/B test, known as multivariate testing. Here, you would test the combined effect of changing several variables at the same time.
So when running a multivariate test, you could change the control version to have different images, color buttons, color schemes, navigation bars, form lengths, and so on.
A multivariate test is best recommended when testing the impact of changing many elements for a single page but this technique needs huge traffic.
Why Split testing is important for your product
Split testing is a fundamental strategy in conversion rate optimization. You can use split testing to optimize many components of your website, including the home page, landing page, or blog posts. Split testing can be helpful if you want to know which copy, headline, layout, or book cover will favorably impact conversion numbers.
Split testing can steer your marketing campaign in the right direction. Instead of going with assumptions, you’ll be implementing actions backed by quality data/feedback.
Let’s say you’re selling on Amazon or shopify. A split test process can help you know which content design or landing page version will best move your audience.
Running a split test lets you understand how customers will respond to different experiences on your website, Amazon listing, or App. You can then determine the best way to optimize your site or product to create the interactive customer experience. Hence, as you optimize your conversion funnel, you also get to involve potential customers in the process.
Not only that, a split testing approach (like the one we follow) can help you gain valuable insights from your audience – insights with the potential to bring more dollars. You get to understand your customers’ experiences through written feedback.
You also can get to learn about customers’ concerns you had not anticipated. Respondents provide detailed feedback on why they preferred a specific version over the other.
Split testing come with these three mighty benefits:
Appropriate for low traffic sites. Split testing is the best technique to adopt if you’re testing with a low traffic site. That’s because conducting an A/B test or even multivariate test is not only challenging but also it needs one to test with a large sample size. That means you need huge amounts of daily traffic, which is not the case with split testing. You can test with as few as 50 respondents.
Increased revenue. Optimizing your conversion funnel by running designs that have proven to influence user behavior will more than likely translate to better conversion rates. That means a higher turnover.
Gather many insights within a short time. Also, split testing makes it possible to test multiple variations of the same parameter, be it a title, image, or logo. Testing many variations simultaneously allows you to gather combined insights in a short time unlike running a traditional A/B test.
Step-by-step on how to perform a split test
Performing a split test is quite straight-forward. You only need to have a SellAMetrics account to get started and polls prices start at just $40.
Step 1: set your goal and decide what to test
The first obvious step is to determine which elements to split test. It could be a product image, a logo, book cover, an app icon, or an entire landing page hosted on different urls. If you have a clear purpose for your site or campaign, it will be much easier to define quantifiable success metrics.
Step 2: Create variations
Create different versions of the page, icon, or image you wish to split test. While a split testing tool will help you set up the different experiences, create the different versions to test beforehand.
Step 3: Create a poll
Log into your split testing software and click on ‘Start a new poll’. This will display the various targeting options. Our tool is easy to use and you can start a poll as soon as you have the different versions to split. You can compare up to eight unique options with an audience size of up to 200 respondents.
Step 4: Write an unbiased poll question
Your poll question is like a hypothesis that guides you to identify possible solutions, often through research. The question you pose to respondents should have no positive or negative bias. Structure the question in such a way that it won’t influence participants to go with a specific answer.
Examples of unbiased questions include:
1. Which logo between the two images captures your attention?
2. Which button color do you like most?
3. Which headline intrigues you?
4. Which book cover would you be compelled to buy?
Some tips to remember include:
Avoid using double-barreled questions. These are questions structured to ask two disparate things but only a single answer is expected. Example: which headline and subheading would you prefer?
Avoid basing your questions on a personal feeling or opinion.
You should not structure your questions to be in the form of an assumption.
Step 5: Next, wait for the test to collect feedback and analyze it
Split testing with SellAMetrics is fun and exciting. The poll takes between 15 minutes to 1 hour. But you can have quality feedback within minutes. The SellAMetrics panel will review your written question and compare the different versions submitted. You’ll be able to track respondents’ feedback as they post it.
Once the test completes, you’ll get an email notification with a link to the test results page. The split test software will show you the most voted version.
N/B: A/B tests and multivariate tests involve scientific hypothesis testing, also a key fundamental of conversion rate optimization. But designing an A/B test needs a solid foundation of statistical hypothesis testing. These types of A/B tests also follow a specific procedure to design, run, and analyze the test results. In this guide, we will not discuss statistical concepts and procedures involved in A/B tests and multivariate tests.
Also: SellAMetrics does not rely on statistical hypothesis testing like in classical A/B testing procedures. Rather, it’s a split test tool to compare A/B/C/D options of an image, videos, or logo.
Statistical hypothesis testing is where an analyst relies on the nature of the data, usually existing traffic, to test if it supports a given assumption.
As an example, the assumption could be: Adding a promotional message at the checkout button will increase the number of sign ups.
You would then collect data from your existing traffic and then test if this hypothesis will have meaningful results.
What do you A/B test?
Here some things you can split test using our tool:
Visual media.
Images, infographics, and videos are a powerful way to build engagements and impact conversions.
Using an appealing product photo that communicates its value can drive conversions. An image gives your customers the feel of the product you’re selling.
Let’s say you’re selling a flower pot – and you have two product images. One image is showing a colorful pot and the other is showing the same pot with a flower sprouting inside. The image with a flower sprouting inside the pot might appeal to potential customers. That’s because it creates the ideal picture of how the pot will look like. Comparing the images or info-graphic through a split test will help you know which version is more appealing and why respondents prefer that particular version.
Here is an example showing split test of an image:
2. Titles and descriptions.
Titles tell your audience what your product is about. By looking at the title or description of a listing, web visitors can tell at a glance if the product they’re about to click on will align with their needs and expectations.
For example, you can split test two different titles for an Amazon product. Let’s say you are selling women’s sweatpants on Amazon. You could have two different titles that you can split test to find out which version will have a positive impact on clicks, search volume, or sales.
Title 1: “Women’s Oversized Sweatshirt.”
Title 2: “Cotton Oversized Sweatshirt with Los Angeles California Printed.”
3. Book Cover designs.
One critical aspect of marketing your book is a well designed cover. It can mean the difference between making profits or losses. A well designed cover should persuade the reader, capture the storyline and reveal little while giving away just enough. In one particular survey, 79 of the respondents said the cover matters when deciding if to buy a specific book. As an author or publisher, split testing will help you know which cover design will make the best impression. The best part is: you can split test more than two potential cover designs.
4. Blurbs or book descriptions
Other than the cover, the book description is another critical element in marketing your book. Target readers will be swayed if to spend money by reading the book description. You can split test different book blurbs, asking the SellAMetrics panel to vote on the best blurbs for a storyline or title.
5. Amazon Coupon offer % VS $
Ad creative
An Ad creative can be in the format of an image, video or audio. Split testing your Ad creative for Google or Facebook can help you know which version of Ad to use for Facebook or Google marketing. After all, you’re spending money to run the Ads – you might as well have an appealing creative that readers are guaranteed to click on.
Logo designs.
Differences between A/B testing and our split wizard tool, SellAMetrics
- In A/B testing, you test with your existing website traffic. In contrast, SellAmetrics has a panel of U.S based respondents. The respondents get a small stipend to participate in the survey. When running a poll through the SellAMetrics panel, you can ask them about images, graphics, texts, headlines, content layout, and so on. Our panel gives detailed responses on why they preferred A/B and vice-versa.
- A/B testing is deeper than splitting traffic between two web pages. It takes time to set up split tests, and the tests need to run for longer periods. With survey polls like in split testing, targeted results can trickle in within 15-30 minutes, at most 2 hours.
- In A/B tests, subjects are not aware you’re monitoring their behavior. A visitor cannot tell if the site has other versions besides the page they are interacting with. Our panel at SellAmetrics gets to see all test variations being compared. The panel provides feedback based on the poll question you’ve asked.
Split testing limitations and best practices to observe
1. Should you run a split test or a traditional A/B test?
Deciding if you should split a test or A/B test can be confusing.
Traditional A/B testing involves gathering a lot of data for the test to be reliable. Hence you need a lot of website traffic and time to run these tests. And if you’re driving traffic through Facebook or Google, it can cost you thousands of dollars.
Let’s say you want to test different elements like the CTA, copy, image, and design layout – A/B testing would be the best option.
But if you’re dealing with a case of less traffic or you don’t want to wait for test results for weeks, split testing is the best option, no doubt. You can use split testing to determine which version of a book cover, logo, title, or product description is more appealing or engaging.
Also, if your goal is to make huge website changes, go with split testing.
It is also the ideal option if your goal is to figure out which version of a book cover, logo, title or product description is more appealing or engaging.
As a recommendation, start with split testing images, videos, book covers, titles, or descriptions… then use the best versions to run a classical A/B test.
2. Create an unbiased question
Again, a critical step to your split testing approach is creating unbiased questions to pose to the SellaMetrics panel. Your question must not be biased, either positively or negatively.
3. Analyze respondents’ feedback in detail
It’s easy to look at the incoming stats and declare a runaway winner. But your split test should go beyond that. It makes sense to closely analyze feedback posted to understand why users preferred a specific version. Dig deep into the comments to identify recurring themes and reactions. You will find useful insights that you can leverage to improve your ideas.
Final Words: “Alleviate doubt, guesswork, and making assumptions – get the real facts with SellaMetrics split testing wizard.”
Split testing is a confusing domain in marketing. The confusion sets in when many marketers fail to make a clear distinction on the various definitions used in the online marketing industry.
Split testing could mean different things to marketers. It could mean having unique variations of the same page. Alternatively, it could mean having two completely unique versions of a web page.
Regardless, insights gained from a slit testing session will be valuable when optimizing your website design, product, or app. Make smart marketing decisions based on hard data rather than go with assumptions.
In fact, split testing will help you build a foundation to run effective controlled A/B tests.
With SellAMetrics you can gather both quantitative and qualitative data. Such data will help you predict user behavior and even implement smart marketing tactics.
Slit Testing Frequently asked Questions
1. Who is split testing for?
Anyone with an online business can take advantage of A/B testing. It can benefit E-commerce sites, media/content sites, lead generation sites, donations sites, publishers, designers, and indie game developers.
2. How do I split the traffic for an A/B test?
You will find many A/B testing programs to help you with this. Also, you could have a developer set up the code for you in the backend.
3. How do I create the variation or different user interfaces for an A/B test?
4. How do you know what to test in A/B testing?
First, gather quantitative and qualitative data about how your site is performing. It will help identify which metrics to improve and which elements to test.
5. How long should a split test run?
You should run the test long enough to gather data from the Panel. The split test tool will alert you when voting is complete.
6. When should you do a split test?
A/B testing can prove beneficial when you want to boost your site revenue conversions or when you’re changing the price and billing options. You can test how a change in price or product image will affect conversion numbers. You can also A/B test when you’re redesigning some parts of your website like the layout, media, and copy.
7. When should you not do a slit test?
If testing for minor changes. Example: changing the CTA button color. Here, A/B testing is more effective.
8. Do I need to have a degree in statistics to do A/B testing?
No, but you need a good understanding of statistical hypothesis testing to be able to analyze the results and make decisions.