Geo Holdout Testing: What It Is and How to Use It in Your Marketing

By Will Burghes, Head of Professional Services, DV Rockerbox

In this blog, we'll cover the basics of geo holdout testing, why you would want to run geo holdout tests, and how to work with DV Rockerbox™ to plan them and measure the results.

What Is Geo Holdout Testing?

Geo testing compares different advertising treatments in two different geographies. In its simplest form, one geographic area is exposed to the advertising being tested and the "holdout group" is not. The results show whether the advertising campaign affected the business's KPIs.

What Can a Geo Holdout Test Tell You?

A geo holdout test can give you important insights into whether your advertising is driving business results or whether you'd see the same results without the spend. Some examples of the specific things you can discover through geo holdout testing include:

• Overall: What is this channel's impact on revenue? In other words, if we stopped spending on this channel, what would happen to revenue?
• KPI-specific: What other results does the channel impact? Does it attract new customers more than repeat customers? Does it drive more visits to my site?

Best Practices for Setting Up a Geo Holdout Test

DV Rockerbox clients use geo tests to evaluate the impact of channels like OTT, online video, social, out-of-home (OOH) and search, and they work with DV Rockerbox analysts to plan the tests and measure the results. Based on our expertise, we recommend several best practices when preparing to run a geo holdout test.

1. First, identify the test and control markets. When selecting which markets to use, you need to pick sets of regions that resemble one another in terms of:

• • • Demographic composition
    • Overall sales
    • Seasonality

1. Make sure that the geographic areas you're testing are substantial enough and have enough conversions and ad spend to make the test statistically significant when compared to the minimum detectable effect.
2. Consider the timeframe you're using and make sure there are no geo-specific seasonal events, macro events or significant marketing strategy changes (e.g., opening a new flagship store with a big PR splash) that might impact the results.
3. During the test, compare the test group to a counterfactual generated from the holdout during the test period. You can calculate the counterfactual in multiple ways, including synthetic control, difference-in-differences, time-based regression or Bayesian structural time series models.

Measuring the Results of a Geo Holdout Test with DV Rockerbox

You can run geo holdout tests in any ad platform that can target audiences geographically, and measure the incremental impact of the tested channel against all other media you have in market.

When you work with DV Rockerbox to plan and analyze a test, we can leverage DV Rockerbox data to dig into the test's impact and remove many manual data-collection steps. Ultimately, DV Rockerbox delivers detailed insights into the test's performance, showing whether your ad spend is truly having the impact you think.

Choosing DV Rockerbox as a measurement platform gives you an always-on solution that gathers data on your marketing activities across every channel and geography. DV Rockerbox data can uncover incredible insights, and our analysts are here to help you make sense of them.

Next Steps

Contact a DV representative.

Learn more about DV Rockerbox.