What is A/B Testing?
A/B testing, also known as Split Testing is an impressive method to figure out the best option for marketing and promotional activities in a business.
A/B testing is a simple method to send multiple versions of a mail in a campaign and determine which one is the best so that you can send the winning campaign to your subscribers for assured higher interactions.
Send and compare two or more versions of a mailer to see which one performs better.
Further, A/B testing helps save time, energy and money when compared to the traditional practices. Send and compare two or more versions of a mailer to see which one performs better. You can choose a fraction of subscribers from the contact list who will receive the test campaigns. Based on the responses, the winner will be decided and the winning campaign can then be forwarded to rest of subscribers.
Why do you need A/B Testing & how it can help you improve?
Planning is always necessary to execute any plan in any business.
Perfectly planned A/B Testing can help yield effective results in marketing by filtering the ideal elements. These elements can then be combined to create a successful marketing campaign.
Improve your approach towards your subscribers and increase your business revenue. Let your customers decide what they want to see and based on this decision send the most efficient emails.
Here, A/B testing is associated with email campaigns. Important elements in an email campaign include the subject line, sending domain, from/reply-to email/name, email content/body, and certain customizations. These elements decide if the receiver will open and interact with your email or not.
Choose What To Test?
There is no limit to what variant you can choose from the email campaign. You can test each and every possible element but that will make this process very long. So, instead of testing all elements, focus on those which are important for your subscribers or which are important for a subscriber to open an email.
- These important variants can be:
Subject
From name From Name/Reply-To
Email Content
Multi variant Customized Case- Above 4 are the most common chosen variants for this A/B test but you are free to use as many as 10 variants. You may not require more than these variants as this number is the largest for chosen variants. The system will automatically assign different colors to each variant for easy differentiation among the group.
Pre-requisite for A/B Testing
Before starting the A/B test process we need to make sure that all the basic ingredients are added. These basic ingredients are:
Contact List
Email Campaign
Control Panel Guide to perform A/B Testing
A/B Testing is basically a 7 step process.
We will discuss each step one by one. This step required some basic information in order to create the A/B test campaign.
-
A/B Testing - 7 Steps1. Basics
1 Name Mention the name of your A/B Test Campaign. The name can match to your email campaign name so that it is easy to track the link and results.2 Choose Type 4 options are mentioned. You can choose only one at a time. If you wish to use other options then you can create another A/B Test Campaign.3 Number of Variants Maximum 10 variants can be chosen. Understand the requirement and according to it choose the variants.2. Components
1 Write a Subject Line In the previous step, we have chosen a subject line to perform the test with 3 variants.2 From/Reply-To Mention the sender's name along with from & reply-to email ids. If there are multiple domains, you can choose one out of the list.3 Dynamic Tags While writing the subject line you can even customise it using the dynamic tag values from the drop-down. All the custom fields which are added while creating email campaign will be listed in this drop-down menu.3. Design
1 Design for All Variants For 3 chosen variants with the subject line we need to choose the design for this testing. Either create a new design or choose from the existing ones.2 Drag-n-Drop Design Builder Design your own customised template by simple drag and drop editor. As the name says, you need to drag the building blocks and prepare the design. Refer to this link.3 HTML/Simple Text Editor You have your designer team to prepare your templates then simply browse the templates or import through links in the editor.4 Popular Designs There are 100s of templates already added in the panel. You can browse through the categories and choose any template you wish to use.4. Recipients
1 Choose the contact List Choose the contact list/s who will be the final recipients of this A/B test campaign. You can choose more than 1 contact list.2 Add Segmentation If the chosen contact list includes segmentation then you can apply this segmentation as well.5. Winner
1 Select the size of your Test If there are 1000 subscribers in the contact list then the system will take 10% of this number (which is 100) to send the A/B test campaign. The winner variant will be chosen to be sent to rest of subscribers. Refer to image for more information.2 Winning Condition Suppose you chose 'Total Open' as your winning condition, then it means that the variant which received maximum opens will be the final winner. Similar is the case with other winning conditions too.3 Running Time of Test You can choose the minimum time to run the A/B test campaign. This time can be in minutes, hours and days. After the time test will complete and the winner will be announced. Choose the time based on size of the variant list.
Set Priority of Variants Out of 3 variants you can choose the priority of variants so that in case there is a tie then the system will declare a winner based on your priority or choice.6. Review
1 A/B Test Details Review page will show the basic details under this heading. If you wish to make any changes then use the 'EDIT' button and save again.2 Template Details As we have chosen a subject line for A/B Testing template details will enlist all the variants individually. Again you can use 'EDIT' button to amend changes if any.3 Sender's Details 'From name', 'from email id', 'reply-to email id'.4 Content Details HTML Design and text content which you have chosen in 'Design' step will be listed here. If you wish you can cross check the template and design and edit it too if required.5 Recipients Information Here the name of contact list along with a total number of subscribers in it will be displayed. If any automation or segmentation is applied to that contact list then that too will be mentioned with EDIT facility.6 Variants Details Size of A/B test along with the chosen winning condition is mentioned plus the run time of the test.7 Choose Schedule Choose to Schedule campaign now or in future.7. Run & Result
A complete report of each and every subscriber activity will be presented. This report will give you the winner name along with the other statistics like total open, unique open, click, unsubscribe, interaction rate. As your chosen winning condition is a total number of opens, the winner is decided based on total opens.
Quick, Easy & Interactive Setup
An intelligent testing environment where it is very easy to setup your split test in a few minutes
Drag to set priority order of variants in case of a tie to choose winner.
Slider to distribute contacts between variants and final-run.
Manually select winner at anytime.
Manually select winner at anytime.
Real time selection of contact.
Insights of A/B Test
The split test may help you to better understand the nature of subscribers. You can learn from test results about,
How to design an effective subject line?
Is it helpful if you use Name of the subscriber in your subject?
How does a From-Name effect the open rate of the campaign?
Is your mailer designer worthy to be paid?
What is the best time range to send emails to your customers?
Some Important Points
Sometimes we get confused with options or doesn't know which combination is best to get highest user response.
A/B testing (sometimes called split testing ) is comparing two/more versions of a mailer to see which one performs better.
You compare two/more mailers by sending the variants to a fraction of subscribers at the same time. The one that gives a better result will be used to send to the rest of the subscribers.
You can select the percentage of subscribers on which this test should run.
Then, set a winning condition for variants. When test will complete, the system will select the winner according to the winning condition. Rest of the emails will be sent by the Winner variant.
You can set the priority of the variants so that in the case of a tie, the system can choose the winner according to your choice.
Final Total number of Recipients
When you select a contact list or segmentation for A/B Test then it is quite possible that size of these lists may change during Wait-Time (generally when wait time is long). So total number of recipients may change at the time when the campaign is sent with Winner Variant. For example, at the time of creating A/B test, there are 1000 contacts in the selected contact list. The test is run on 10% of the total contacts, i.e., 100 contacts, and wait time is 10 days. Within 10 days contacts increased by 100. Now total contacts are 1100 in the list. So after 10 days, final campaign will be sent to 1000 contacts, i.e., ((1000-100)+100)