A/B testing (also called split testing) is a form of statistical analysis in which you test two versions of the same thing to see which one performs better. For example, you might be trying to find out what drives more people to sign up to your service, and the difference in the ‘a’ and ‘b’ test groups might be as small as a single word in the ‘call to action’ on your website landing page, or it might be two completely different versions of your website. There’s a more complicated version of this, called multivariate testing, where instead of a/b groups, you test a/b/c/d/e/f groups to see which combination of multiple factors produces the desired result.
Why do a/b testing for videos?
There’s a lot of talk about how changing one word in your call to action or on your sign-up button can potentially increase conversions by a statistically significant amount, but is it true? In our experience, yes, rarely by very much however, but when you add up a lot of small tweaks, it can make a big difference.
If you want to split test a video for your social media campaign, mailout or website landing page, Biteable has made it easy to produce multiple versions of a video quickly, with options to duplicate an existing video and re-edit the new version.
How to do video a/b testing
There are a number of software tools you can use to a/b test the effectiveness of your digital marketing. In this section we’ll show you how to a/b test with Facebook, and how to use some of the other great a/b testing tools for marketers.
YouTube
Although YouTube Analytics does not support ab testing, there are some third-party services who can manage it for you. TubeBuddy does it (although it’s hard to work out how to access their service if you don’t know where to look). YouTube does have its own analytics, of course, so it would be possible to post two versions of a video and compare the results yourself.
Mailchimp
Mailchimp is an email management tool that allows you to easily set parameters for ab testing on mailouts to your mailing list. Make two slightly different videos to link to (Biteable makes it easy to do this) and find out if the day of the week affects your video click-through rates, how you ‘sell’ your video in the subject line affects engagement, or if the thumbnail or color of your button has any measurable effect. You can find out how it works here.
Facebook has made it really easy to split test your video ad. When you’re creating your ad in Facebook Ads Manager, in the Campaign Objective section, click on Conversions, Lead Generation or App Installs and you’ll get a small tick box that allows you to make your campaign a split test.
Google Analytics
Google have also made it easy to split test websites with Google Analytics for free. If you have a website with a video on your landing page, you can make two different versions of the landing page that contain different versions of the video see which one works best. Here’s a good article on how to do it.
What to a/b test
It’s amazing what kinds of things can make a difference. AdEspresso lists the most important factors in a Facebook ad campaign as (in descending order of importance) image, post text, placement, landing page and headline. Here, then, are a couple of things to consider testing:
Thumbnails. If your video doesn’t autoplay, it will probably display a thumbnail to give viewers some idea of the content. If it’s possible to change what thumbnail will display or even choose it yourself, it could have a big impact on whether or not people choose to watch your video. Color. There’s a lot of theory online about how color can influence a viewer’s perception. It’s hard to quantify that, but what is certain is that a good color scheme, no matter what it is, will make you look more professional, so it certainly won’t hurt to AB test different color schemes.
CTA. Every marketer knows that a good call to action describes what you want the viewer to do next, but it can be more or less effective depending on the exact wording. Try to minimize any sense of obligation or hassle, but use words that describe a dynamic action rather than a passive one. For example, friendly action words or phrases like “learn more,” “give it a try,” or “sign up for free” sound a lot more personable than “submit” and “enter credit card details”.
Title. Your video title is of paramount importance, especially if you’re posting it on social media sites. On YouTube it’s how people find your video, and on Facebook and Twitter it will have to quickly define what the video is about and grab the audience’s attention as they scroll down their feed.
Location. The position of your video on the web page may determine whether or not people watch it. Many businesses use a video as nearly the sole feature on their landing page, as there are many advantages of having a video front and centre on your website. Remember, the more distractions you provide, the less likely people are to click on your video, so experiment with things like placement and what distractions surround it.
Length. As a basic rule, the longer the video, the less likely people are to watch to the end, or even watch it at all. Between 1-2 minutes is optimal, but you should always make your video as short and to-the-point as possible while retaining the essential information. Test different lengths to make sure you’re hitting that sweet spot between providing enough information and going too long.
Introduction. Your introduction should be short and sweet, it’s a balancing act between telling your audience who you are and what’s in the video, and not going on too long and losing your viewers. Ad Age says that “nearly 20% of the audience that starts watching a given video clip will abandon it within the first 10 seconds”. A/B testing might help you get the balance right.
Music. You’d be surprised how music affects consumers’ buying behaviour. Music can also completely change the meaning of a video sequence, so it seems obvious this would be a good thing to split test.
A/B video testing can make a difference
Big companies like Google and influential people like Barack Obama have used split testing to great success. It’s a commitment though, as you’ll need to test and change multiple things for it to make a noticeable difference. With today’s tools, however, it’s relatively fast and easy to do, and we learn important things about our target market with every iteration, so it’s time to get out there and set your first experiment in motion.