How many creatives can a budget actually test? It depends on what you judge them on. A conversion needs thousands of visitors to tell two ads apart. A hook rate needs a few thousand impressions. The same budget therefore supports very different numbers of variants depending on the metric.
Enter your budget and your usual rates. The table shows, for each metric, how much volume every variant needs, what that costs, and how many variants you can afford.
Your numbers
Use averages from the last month for the audience type you are testing on.
| Judge each variant on | Volume needed per variant | Cost per variant | Variants your budget supports | Budget to test 5 |
|---|
How the table is built
For the first three rows the volume per variant comes from a standard rule of thumb for comparing two rates. The number of observations needed per variant to detect a difference δ between rates around p is about 16 × p × (1 − p) ÷ δ², at the usual 5% significance and 80% power. This is Lehr's rule, from Sixteen S-squared over D-squared (Statistics in Medicine, 1992). The calculator sets δ to your chosen share of p, so asking to detect a 20% relative difference on a 30% hook rate means a 6 point gap.
The fourth row uses a quick floor of 30 primary events per variant, the minimum we use in our decision framework before an ad has earned any verdict. It is a floor for saying anything at all, not a power calculation, and it is far smaller than what the conversion rate row demands. Compare the two rows and you can see why conversion tests on small budgets so often end in noise.
What the numbers usually show
On the defaults, a hook rate test needs under a thousand impressions per variant, so a budget of about €1,000 screens dozens of openings. A conversion rate test needs thousands of clicks per variant, which at these rates costs over €20,000 per variant. That asymmetry is the reason to test openings first. It is also why the research on openings matters: 1.5 seconds of genuine attention was enough to encode memory in the VCCP and Nelson-Field study of 72 ads, provided the brand was in frame. See the 3-second hook, deconstructed.
The wider point is about noise. Lewis and Rao examined 25 digital advertising experiments covering $2.8 million of spend and found only 3 had the power to separate a 50% return from zero. Small differences on small volume are not signal, which is why this tool exists. The full argument is in kill it or scale it, and a written method for the same calculation is in how many creatives should you test at once.
Treat the volumes as order of magnitude. Lehr's rule assumes a fixed sample size set in advance, independent observations and a randomised split. Ordinary ad sets are not randomised, because the platform shifts spend towards ads it expects to win, and people see ads repeatedly. Meta's A/B test tool randomises. Ordinary ad sets do not.
More variants make it harder. Comparing many variants against each other raises the chance that one looks good by luck. Use the figures as a minimum.
Rates move. The hook, click and conversion rates you enter are assumptions, and your real rates will differ by audience and placement.
Knack applies the same discipline automatically: creatives below the spend and volume floor are shown as too early to call rather than ranked, and each ad is graded on the metric its campaign objective implies. If you have not yet sized your overall plan, use the media spend calculator first.
Sources and method
- Lehr, R. (1992). Sixteen S-squared over D-squared: a relation for crude sample size estimates. Statistics in Medicine, 11(8), 1099 to 1102. Order of magnitude estimates at 80% power and 5% significance.
- Lewis, R. A., and Rao, J. M. (2015). The Unfavorable Economics of Measuring the Returns to Advertising. The Quarterly Journal of Economics, 130(4), 1941 to 1973.
- VCCP Media and Dr Karen Nelson-Field / Amplified (May 2025). Hacking the Attention Economy.
- Cost per click = CPM ÷ 1,000 ÷ link CTR. Cost per conversion = cost per click ÷ conversion rate. The floor of 30 events is a working convention, not a published standard.