How to A/B Test Ad Creative

An A/B test of ad creative compares two versions of one ad that differ in a single element, such as the opening line or the image, shown to comparable audiences at the same time. You choose the success metric and the run time before launch, then judge the result only once the test has finished and the difference is clear enough to trust. This guide is part of our paid advertising guide.

It's about how to run the test, not what to put in the ad. For ideas worth testing, see our post on what makes ad creative perform. You'll need conversion tracking and two versions of one ad.

What an A/B test can and can't tell you

A well-run test answers one narrow question: for this audience, on this platform, at this time, did version B produce more of the chosen result than version A? That's useful because it replaces opinion with evidence, one decision at a time.

It can't tell you why people responded, whether the result will hold in a different season or audience, or what would happen with a third idea you didn't test. It also can't rescue a test with mixed variables. If the two ads differ in image, headline and offer, a winner tells you only that one bundle beat another.

So treat each test as one entry in a running record, not a final verdict. The value builds as you stack results and retest the ideas that matter most.

Steps

1. 01
   Write the question the test will answer.

   Put it in one sentence before you build anything, such as "Does opening with the customer's problem get more leads than opening with our offer?" Add what you expect to happen and why. A written question stops the test from drifting and makes the result easy to act on later.

2. 02
   Change one variable only.

   Keep everything else identical: audience, placements, budget, landing page and every other part of the ad. Change one element, such as the first line, the opening shot of a video, the image or the call to action. If you want to test a whole new concept, that's fine, but label it as a concept test.

3. 03
   Choose the success metric before launch.

   Pick the one result the test will be judged on, such as leads or purchases, and write it down. Clicks and views are easier to move but don't always lead to sales. Note a guardrail too, such as cost per lead, so a version that wins on clicks but loses on leads isn't crowned by mistake.

4. 04
   Split the audience fairly.

   Each person should see only one version, and both versions should get a comparable share of traffic. Use the platform's own testing tool where you can: Google Ads experiments offer a cookie-based split that shows each user one version, and Meta's Marketing API includes a split-test study type. Two ads in one ad set aren't a fair test, because delivery favours one.

5. 05
   Set the run time and budget in advance.

   Cover at least one full week, so weekdays and weekends are both included, and allow enough budget for each version to collect a meaningful number of the result you chose. Write the end date down. Deciding the length first stops you from ending the test the moment one version happens to lead.

6. 06
   Launch both versions together and leave them alone.

   Start them at the same moment so they face the same days, auctions and news. Don't edit budgets, audiences or the ads while the test runs. Google's own help warns that changes to either version during an experiment can make the results harder to interpret.

7. 07
   Read the result with its confidence range.

   When the test ends, look at the difference and how sure the platform is about it. Google Ads shows a confidence interval, with 80% as the default level, and marks results that are statistically significant. A range that spans both a gain and a loss means you don't have a winner yet.

8. 08
   Log every test, including the losers.

   Record the question, the variable, the dates, the metric, both results and the decision in one shared sheet, using the format below. Losing tests are as useful as winners: they stop the team from retesting the same idea and show which kinds of change tend to matter for your audience.

9. 09
   Act on the result, then test the next variable.

   If the challenger won clearly, make it the new control and pause the old version. If the result wasn't clear, keep the control and decide whether a bigger difference is worth testing. Then pick the next question from your list. Steady, simple tests build more knowledge than occasional big ones.

If you'd rather have tests planned, the versions produced and the results read for you, our in-house ads team makes the creative and runs the campaigns.

A simple test log

A test log doesn't need special software. One row per test, in a shared spreadsheet, is enough. Copy these columns and fill them in before launch, then complete the result fields when the test ends:

Test ID:          T-014
Question:         Does a problem-first opening get more leads than offer-first?
Variable changed: Opening line only
Control (A):      Offer-first opening
Challenger (B):   Problem-first opening
Success metric:   Leads (form submissions)
Guardrail:        Cost per lead
Platform / split: Platform test tool, even split
Start / end:      Set before launch, full weeks
Result A / B:     Fill in at the end
Confidence:       As reported by the platform
Decision:         Keep A / switch to B / no clear winner
Next question:    What to test next

The question, variable and metric are filled in before launch on purpose. If they're written after the result arrives, it's too easy to pick the metric that makes the favourite look good.

For search ads, the same log works for headline and description tests. We cover how responsive search ads mix their assets in how to write Google search ads.

When a result isn't meaningful

Most inconclusive tests fail for the same few reasons. Google's experiment help lists three: the test hasn't run long enough, the traffic split was too small, or the change didn't make a real difference. All three apply to creative tests on any platform.

Small numbers are the usual problem. A version that leads by a handful of leads after a few days can easily lose that lead by the end. The platform's confidence range is the honest guide: if it includes zero, treat the result as a tie.

A meaningful result also has to be relevant. A winner on clicks with no change in leads isn't a reason to switch. Neither is a result from a week that was unusual for your business, such as a holiday. When the audience is retargeting traffic, keep in mind that it behaves differently from new people; our guide to retargeting strategies explains why.

Troubleshooting

One version got almost all the spend

Cause: the two versions were placed in the same ad set or ad group, where the platform's delivery system shows the ad it predicts will do best. Early predictions can be based on very little data, so one version gets most of the impressions before it has had a fair chance. The result looks like a test but isn't one.

Fix: rebuild the test with the platform's testing tool, which splits people evenly between versions, as in step 4. On Google Ads, use an experiment with a cookie-based split. On Meta, use its split-testing option, checking the current setup in Ads Manager first. If you can't, run each version in its own identical ad set or campaign with equal budgets, and treat the result as a rougher guide.

The leader changed halfway through

Cause: with small numbers, early leads often reverse. One version may have had a lucky few days, or the two versions may suit different days of the week. Edits made during the test, such as a budget change on one side, can also shift results in a way that has nothing to do with the creative itself.

Fix: go back to the run time you set in step 5 and judge only the full period. Don't call a winner from a mid-test peek. Check the change history for edits during the test, and if there were any, note them in the log and consider rerunning the test cleanly. Next time, include at least one full week and leave the test untouched.

The test never reaches a clear result

Cause: the difference between the versions is too small to detect with the traffic available, the success metric happens too rarely, or the split gives one side too little traffic. A change of one word in a long ad, judged on purchases from a small budget, may never produce a confident answer in any sensible time.

Fix: accept the tie and keep the control, then make the next test bolder: a different opening, angle or format rather than a small tweak. Judge on an action that happens more often but still matters, such as lead form submissions rather than closed sales. Use an even split, and make sure the budget suits the length you planned.