How to Implement an Email Testing Strategy Published May 2, 2018 Can you predict how your audience will respond to every aspect of your email marketing? You may have data that gives you some clues, but—unless you’re a mind reader—the answer is probably no. This is where email testing comes in. In this blog post, we explain how to implement an email testing strategy and get better results from your marketing. Why is email testing important? Email testing helps you understand how variable elements in your strategy impact engagement. It is a smart way to understand which tactics get the best results. Testing allows you to continually refine and improve your approach. This is the fastest route to increased engagement and conversions. Without testing, you may make the wrong assumptions and waste time tweaking the wrong tactics. For example, say your open rates are low. You might assume this is due to your subject line, and spend ages improving it, only to get the same results. Little did you know; your open rate was actually because of send time. Your subject line had nothing to do with it. Testing ensures we take a data-led approach to optimisation. It guides our efforts, making sure we improve the right things. The things that will actually improve results. What should you test? So, which elements of your email strategy should you test? Here are some crucial things to consider: send times send frequency subject lines templates messaging colours call to action button text button style use of images from line Varying each of these email elements may change the way your audience engages with your emails. Humans are complex and emotional beings. Something as simple as the colour or position of a button may change their response to it. Email A/B testing helps you figure out the most effective version of each key element, so you can optimise results. How to implement an email testing framework Now you understand the value of testing and which elements you need to consider. But how do you put email testing into practice? Here’s a step-by-step guide to implementing an email testing strategy: 1. Setting a measurable objective First up, set up a measurable objective. To do this, you need to decide which metrics matter most. These may include: open rates click-through rates click-to-open rates unsubscribe rates website conversions revenue Decide which metric you are look at and make a note of your objective. For example, “to improve open rate”. 2. Creating a hypothesis Next up, create a hypothesis to test. A hypothesis is a theory about how your audience will behave. A good hypothesis includes: what you are going to change what you think the outcome will be why you think that will be the case For example, “If we change send times to the evening, then open rates will increase, because that’s when people have finished work.” Try creating your own hypothesis using the same format: “If ____, then ____, because ____.” 3. Deciding what to test Once you’ve created your hypothesis, you need to decide exactly what to test to meet this hypothesis. In our examples, this might be changing send times from midday to 7pm. 4. Considering sample size Now you need to consider the sample size needed to get meaningful results from you test. Testing on too small a sample size won’t tell you a great deal. That said, if you have a large audience, you don’t want to test on a large sample. This would mean sending potentially ineffective marketing to a large number of people. If you have a small audience consider testing on 10 to 20 percent of them. For audiences of 10,000 or more, 5 percent may suffice. Here’s a handy calculator to help you work out what sample size to use. Don’t forget that your sample needs to be randomly selected so your results are not biased. 5. Creating testing calendar Next up, you need to create a testing calendar. This helps you outline your testing strategy. You might need to run multiple tests of the same hypothesis to find a meaningful result. You may also have multiple hypotheses to test. A calendar helps you prioritize these and work out which order to test them in. 6. Setting the test live Now comes the time to create the test and set it live. Our drag-and-drop editor makes it really easy to create and adapt templates to support your testing strategy. Being able to implement your test quickly is extremely valuable especially. You don’t want your testing strategy get in the way of your business as usual email marketing. Testing should support your overall strategy, not hinder it. 7. Collecting data and reporting Next up, comes data collection and reporting. Our own email marketing platform makes easy-work of reporting. You’ll get beautifully simple reports to show that your team and those higher up can digest easily. If you’re A/B testing subject lines you’ll get notified about which variation was performed best and whether a winner could be declared by statistical significance. We recently launched a new reporting feature that makes it even easier to see the results and trends across multiple campaigns. 8. Retesting If your results are borderline, it might be hard to prove or disprove your hypothesis with confidence. That’s when retesting is a good idea. For inconclusive results, try running the test two or three times to draw meaningful conclusions. 9. Optimising Once you have conclusive results, you can prove or disprove your hypothesis. This allows you to optimise your strategy accordingly. Sometimes your test may not have the desired result. Results could go down but don’t feel disheartened. A failure to prove a hypothesis should be seen as a success. Going back to our example, if you prove that 7pm send times do increase open rates, you can now change send times to this time across the board. Takeaway We hope our guide to implementing an email testing strategy helps you get yours up and running. This is your first step to optimising results and boosting your bottom line. Email testing is easy to launch at the push of a button with our email marketing platform. To see them in action, book a demo via the button below.