Guessing is the slowest way to improve your emails. You might feel sure that a shorter subject line works better, or that a red button beats a blue one — but feelings are often wrong, and the only way to know is to test. A/B testing, also called split testing, sends two versions of an email to small groups, measures which one performs better, and sends the winner to the rest of your list. It is the single most reliable way to lift your results over time, because every improvement is proven by your own readers rather than borrowed from someone else’s advice. This guide explains how to run tests that actually mean something.
What A/B Testing Is, and What It Is Not
In a basic A/B test, you split your audience into two similar groups. Group A receives the original version, group B receives a version with one change, and you compare the results on one metric you chose in advance. The winning version goes to everyone else. That simplicity is the whole point. A/B testing is not redesigning an entire email and hoping for the best: if you change five things at once and results improve, you will never know which change did it. It is also not a one-time fix. The best senders test constantly, because audiences change, and what worked last year may not work now. Think of testing as a habit, not a project: one small test per send, every send, and the improvements stack up.
What to Test First
Start with the elements that have the biggest effect and are easiest to change. Subject lines are the obvious first test, because they decide whether anyone opens at all — and our guide to subject lines that get opened gives you plenty of formulas to test against each other. Next, test the preview text, the short line readers see beside the subject, which many senders leave blank and waste. Then test the call to action: the button text, colour, and placement. A button that says “Get my free guide” will usually beat one that says “Submit”. After that, test the email’s structure: a short version against a longer one, one offer against two, an image-led layout against a text-led one. Send time is another powerful test, covered in our guide to the best time to send emails. Work through this list in order, and you will find wins quickly, because most lists have at least one weak spot hiding in plain sight.
How to Run a Valid Test
A test is only useful if it is fair. Change exactly one variable between the two versions. If you test a subject line, keep the email body, sender name, and send time identical. Next, make sure each group is large enough. On a tiny list, a difference of a few clicks is noise, not a result — as a rough rule, you want at least a few hundred recipients per version before you trust the outcome, and more is better. Split the groups randomly, and send both versions at the same time, so time-of-day effects do not skew the result. Decide in advance which metric decides the winner: open rate for a subject-line test, click rate for a content or button test, conversions for an offer test. Finally, give the test enough time. Do not declare a winner after an hour, when only your fastest openers have responded; wait until most of the response has come in, usually at least a day, before judging.
Reading Results Without Fooling Yourself
Small differences are usually meaningless. If version A gets a 21% open rate and version B gets 22%, that gap could easily be chance, especially on a small list. Look for clear, repeatable differences, and be suspicious of a single surprising result. When a test wins, ask why before you celebrate: did the winning subject line work because it was shorter, or because it mentioned a benefit? The reason matters, because it tells you what to test next. Keep a simple log of every test: what you changed, who it went to, and what happened. After a few months, that log becomes a playbook of what your specific audience responds to, which is far more valuable than any generic best-practice list. And when a test loses, that is still useful information — you have ruled something out, and you will not waste time on it again.
Building a Simple Testing Calendar
Testing works best on a rhythm. A practical approach for a small sender is one test per send, rotating through the elements that matter: subject line this week, call to action next week, layout the week after, send time after that. Over a month you will have four real results and a clearer picture of your audience. Avoid testing during unusual sends — a huge launch or a holiday email will behave differently from your normal sends, and results from those will not transfer. Also resist the urge to test everything at once. Multivariate testing, where many elements change together, needs a very large list to produce trustworthy results, and for most small businesses it just creates confusion. One variable, one test, one lesson, repeated steadily, beats an ambitious testing program that nobody finishes.
Mistakes That Waste Tests
The most common mistake is testing without enough recipients and then treating noise as a verdict. If your list is small, test the big things — subject lines and offers — and accept that you need patience. The second mistake is changing the winner and the loser mid-test, which invalidates the whole thing. Once a test starts, leave it alone until it finishes. The third mistake is testing things that do not matter: button shades and font sizes rarely move results, while the offer itself and the first two sentences usually do. The fourth mistake is copying someone else’s winning test without checking it on your own list. Audiences differ, and a tactic that worked for a large retailer may flop for a small newsletter. Finally, do not test and then ignore the result. If version B won clearly, make it your new default, and test something else against it next time. That is how small wins compound.
Frequently Asked Questions
How big does my list need to be for A/B testing?
You can start with a few hundred subscribers, but keep expectations modest. With a small list, test only big, obvious changes — like two very different subject lines — and look for large differences. As your list grows, smaller differences become meaningful, and you can test subtler changes.
Should I test every email I send?
Ideally yes, but keep it lightweight. One small test per send is enough. If testing starts delaying your sends or complicating your process, simplify: test only subject lines for a while, and add other elements once the habit is comfortable.
Conclusion
A/B testing turns email improvement from guesswork into evidence. Change one thing, split your audience fairly, decide the winner in advance, and write down what you learned. Repeat that every send, and your emails will quietly get better month after month — not because you found a magic formula, but because your own readers told you what works.