Incrementality Testing for DTC Brands: How to Measure and Calculate Lift

Incrementality testing is an experiment that measures how many sales your marketing caused. You split people or regions into a group that sees your ads and a holdout group that doesn't, then compare their results. The difference is the incremental lift: the orders that would not have happened without the ads.

This guide covers how to calculate incrementality, the main test designs, how to size and read a test, which tools run them and which channels a DTC brand should test first. For how lift tests fit alongside attribution and marketing mix modeling, see our marketing measurement guide.

What is incrementality in marketing?

Incrementality is the share of your results that your marketing caused. Attribution answers a different question: it gives credit to the touchpoints that came before each order, whether or not they changed anything.

The gap between the two is largest for channels that reach people who were already going to buy. A retargeting ad shown to someone with a full cart, or a branded search ad clicked by someone typing your name, often collects credit under last click attribution for a sale that was already happening. Only an experiment can tell how much of that credit is real. In a study of 15 advertising experiments at Facebook, Gordon and colleagues found that observational methods often failed to match what the randomized experiments measured.

Incrementality testing vs A/B testing

Both are randomized experiments, but they compare different things:

An A/B test can tell you that creative B beats creative A while both are mostly reaching people who would have bought anyway. An incrementality test is what tells you whether either is worth paying for.

---

How to calculate incrementality

Five numbers come out of every lift test. Here they are with an illustrative retargeting test that held out 10% of the audience from the ads:

1. Lift. How much higher the test group's conversion rate is than the holdout's.

• Lift = (test rate − holdout rate) ÷ holdout rate = (0.30% − 0.24%) ÷ 0.24% = 25%

2. Incremental conversions. Test-group purchases minus the purchases it would have made without ads, estimated from the holdout's rate.

• Expected without ads = 0.24% × 900,000 = 2,160
• Incremental conversions = 2,700 − 2,160 = 540

3. Incrementality rate. The share of the test group's purchases that the ads caused.

• 540 ÷ 2,700 = 20%

4. Incremental ROAS (iROAS) and incremental CPA. With $30,000 of spend and an $80 average order value:

• Incremental revenue = 540 × $80 = $43,200
• iROAS = $43,200 ÷ $30,000 = 1.44
• Incremental CPA = $30,000 ÷ 540 = $55.56

5. Incrementality factor. Compare the test with what your reporting credited. If the platform attributed 1,800 purchases to this campaign in the same period:

• 540 ÷ 1,800 = 0.30

In this example the campaign reported a ROAS of 4.8 (1,800 × $80 ÷ $30,000) but returned 1.44. Whether it's worth keeping depends on how 1.44 compares with your break-even ROAS, not on the 4.8. The marketing measurement guide shows how to apply a factor like this to your day-to-day attribution reports.

---

Types of incrementality tests

Platform tests are the easiest to start. Meta's lift studies randomize people into test and control groups and compare their conversions, though Meta currently limits access and asks advertisers to contact their Meta representative. Google offers Conversion Lift based on users and based on geography; Google notes the geography split is noisier and usually needs a bigger budget, and is opened through an account representative.

Geo experiments don't need a platform's cooperation. Meta's open-source GeoLift uses synthetic control methods to build a comparison from untreated regions, and includes power calculators for choosing markets. Because the outcome is your own orders by shipping region, a geo test also counts sales the ad platform never sees.

---

Which channels should a DTC brand test first?

Test where the gap between reported and real results is likely to be largest, or where the most money is at stake:

1. Retargeting. It reaches people who already showed intent, so it is the classic case of credit for sales that would have happened anyway.
2. Branded search. Someone searching your brand name already knows you. The question is how many of them would click your organic listing if the ad weren't there.
3. Affiliates, coupon and cashback sites. They often appear at checkout, right before a purchase that was already happening.
4. Your largest prospecting channel. Even if it is genuinely incremental, it is where a wrong estimate costs the most.
5. Email and SMS flows. Owned channels cost little per send, but discount-led flows can train customers to wait for a code.

Write down before the test what you will do with each possible result: cut, hold or scale. A test that can't change a decision isn't worth the sales it costs.

How to size an incrementality test

Before launching, check that the test can detect the effect you care about. The smallest lift a test can reliably detect is its minimum detectable effect, and four things set it:

Tools help with this before you spend anything. GeoLift's power calculators estimate which markets and durations can detect a given lift, and Google sets the budget for a user-based Conversion Lift study during setup from the campaign's historical conversion data. If the numbers show the test can only detect a very large effect, test a bigger change, run longer or pick a higher-volume channel.

Hold everything else still while the test runs: no big promotions, price changes or budget shifts in other channels. Run whole weeks so weekday and weekend behavior are both covered, and long enough to cover the time customers usually take to buy.

---

How to read the result

Treat the result as a range. Every test reports an estimate with uncertainty around it, usually as a confidence interval. Plan around the low end. A result of "20% incremental, somewhere between 10% and 30%" means the campaign may be half as good as the headline number.

Statistical significance is about the range, not the headline. A result is statistically significant when the range doesn't include zero. A non-significant result doesn't prove the ads did nothing; it may mean the test was too small to tell.

Apply the result to your daily reporting. Turn the test into an incrementality factor and apply it to the attributed numbers you use every day, then compare the adjusted ROAS with your break-even point.

Re-test when things change. A factor measured in one season, with one set of creative and audiences, drifts. Re-test after big changes in spend, creative, targeting or the time of year, and keep a log of every test so you can see how each channel's factor moves.

---

Incrementality testing tools

Platform tools are the fastest way to start but only see their own conversions and their own ads. Geo tests take more setup and measure your actual store orders across every channel.

---

Where Upstack fits

Upstack doesn't run incrementality tests. It looks after the data the tests and the follow-up decisions depend on:

• Conversions the platform can count. Purchases are sent server-side to Meta, Google and TikTok, so platform lift studies aren't relying on the browser pixel alone to see your orders. Our server-side tracking guide explains why the pixel misses orders.
• An attributed baseline to compare against. The Attribution board credits Shopify orders by source under First click, Last click or Any click. Dividing a test's incremental orders by the orders the board credited for the same campaign and period gives your incrementality factor on a store-revenue basis, rather than on what the platform claimed.

---

Frequently asked questions

What is incrementality testing?

Incrementality testing is a controlled experiment that measures how many conversions your marketing caused. A test group sees your ads, a randomly chosen holdout group doesn't, and the difference in their results is the incremental effect.

How do you calculate incrementality?

Compare the conversion rates of the test and holdout groups. Lift is the difference divided by the holdout rate. Incremental conversions are the test group's conversions minus what it would have converted at the holdout's rate. Divide incremental revenue by spend for incremental ROAS.

What is a good incremental lift?

There's no universal benchmark, because lift depends on the channel, audience and how many people would have bought anyway. The useful test is whether incremental ROAS clears your break-even ROAS. A campaign with a small lift on a large, cheap audience can be worth more than one with a big lift on a few expensive conversions.

Is incrementality testing the same as A/B testing?

No. An A/B test shows two versions of an ad to two groups and asks which performs better. An incrementality test shows ads to one group and nothing to the holdout, and asks whether the ads caused sales at all.

Can a small brand run incrementality tests?

Yes, with limits. A small brand may not have enough conversions to detect a modest lift, so test bigger changes (turning a channel off rather than trimming it), run longer, or start with an email or SMS holdout, which costs nothing in media.

---

Key takeaways

• Incrementality testing measures the sales your marketing caused by comparing a group that sees ads with a holdout that doesn't.
• Lift, incremental conversions, incrementality rate, iROAS and the incrementality factor all come from the same test-versus-holdout comparison.
• Platform lift studies are the easiest start; geo tests measure your own store orders across channels.
• Test retargeting, branded search, affiliates and your biggest channel first, and decide what you'll do with each result before you launch.
• Read results as ranges, apply them to your daily attribution, and re-test when spend, creative or seasons change.

Request a demo and we'll show you your attributed baseline by channel before you run your first test.