Hold out Testing

In the rapidly evolving world of digital advertising, marketers are constantly seeking more effective ways to reach and engage with their target audiences. Key to this pursuit is understanding the intricacies of modeled audiences, conversion optimization algorithms, geo-testing, and incrementality testing. These strategies, when applied judiciously, can significantly enhance the effectiveness of digital campaigns.

The Rise of Modeled Audiences

One of the most prominent trends in digital advertising is the use of modeled audiences. Platforms like Facebook have led this charge, with a significant portion of their ad spend being directed towards what they call 'broad audiences' - previously known as lookalike audiences. This approach involves creating a pyramid-like structure of potential customers, starting with a seed audience, such as a company's best customers from the past six months. The platform then identifies potential targets, ranking them based on their likelihood to convert.

For instance, a fashion brand selling shoes can leverage signals from various shoe-related activities captured by pixels across websites. Facebook's algorithm can identify consumers actively looking for shoes, those who might be interested soon, and a broader audience who generally show interest in shoes. This segmentation ensures that ads are served to the most relevant audience first, enhancing the likelihood of conversions.

The conversion optimization algorithm plays a crucial role in determining the effectiveness of a campaign. It operates on a top-to-bottom approach, serving impressions to the most likely buyers first. This strategy aims to achieve strong last-click attribution, improving campaign metrics like CPM (Cost Per Mille) and encouraging increased ad spend. However, it's crucial to note that as you move down the pyramid, the conversion rates tend to decline, leading to diminishing returns in broader audience segments.

Geo-Testing: A Strategic Approach

Geo-testing offers a practical solution for testing and scaling marketing strategies. By categorizing different states or regions into tiers based on factors like penetration rate and conversion propensity, marketers can execute controlled tests. For example, finding a similar but smaller markets with similar markets as a larger one like California (a tier three state) allows for low-risk testing with scalable insights. This approach enables marketers to extrapolate findings from smaller markets to larger ones, ensuring efficient allocation of marketing resources.

Incrementality testing, or holdout testing, is vital in understanding the actual contribution of a specific marketing channel. By comparing control markets (where a particular media, like Facebook, is turned off) with active markets, marketers can measure the true impact of that media on revenue. For example, if a company observes a 26% drop in revenue in the absence of Facebook ads, it can infer that Facebook contributes 26% to its business.

The next step involves comparing these findings with platform-reported metrics. If Facebook Ads Manager reports a higher number of conversions than what the incrementality test suggests, the marketer can apply a multiplier to align reported conversions with actual impact. This multiplier becomes a critical tool in ongoing operational reporting, ensuring that marketers account for the true incremental value provided by platforms like Facebook.

Choosing the Right Attribution Model

Deciding on the appropriate attribution model is another crucial consideration. Whether a marketer relies on platform reporting, Google Analytics, or a media mix model, the chosen method must accurately reflect the impact of different channels. A heterogeneous approach allows for the integration of diverse data sources, offering a comprehensive view of a campaign's performance across various platforms.

Diminishing Returns in Marketing

The concept of diminishing returns is pivotal in marketing, especially when managing ad campaigns. Imagine your marketing efforts as a pyramid. At the top, the conversion rates are high, but as you progress down, these rates start to decrease. This phenomenon is due to the diminishing impact of each additional dollar spent. The first dollar might bring significant returns, but the next dollar is less efficient, creating a typical curve of diminishing returns. 

Consider a scenario where a brand is spending $100,000 a week on advertising. When they double this expenditure, the crucial question is how significantly the returns will diminish. For a new or smaller brand, it’s sometimes hard to see them hit diminishing returns. It could take 6 months to a year before they see it hit them. For larger brands, they can double their spend and barely see a spike in conversions. It’s akin to driving down a mountain; the slope's severity can vary greatly. This uncertainty necessitates rigorous testing to understand where your brand stands on this curve of diminishing returns.

Incrementality testing is a powerful tool used to gauge where your campaign is on the diminishing returns curve. It helps to determine how much the returns diminish with increased spending. For example, small and emerging brands might double their ad spend repeatedly without seeing a notable change in returns. This could be due to their large potential audience and the universal appeal of their products, like shoes or t-shirts. In contrast, well-known brands might see a steeper curve, where increased spending leads to higher costs per thousand impressions (CPM) and diminished returns.

Testing Strategies

There are various testing strategies, like geo testing and split testing, which fall under two primary categories: incrementality tests and scale tests. Geo tests are based on first-party data and offer high control and transparency, making them a preferred choice for many brands. However, third-party platform lift tests also play a vital role as part of a comprehensive testing strategy.

Beyond incrementality testing, marketers can employ advanced attribution techniques to refine their strategies further. These include:

• Marketing Mix Modeling: This technique evaluates the effectiveness of different marketing tactics and channels, helping allocate resources more efficiently.
• Multi-Touch Attribution: Although complex, this method provides insights into how various touchpoints contribute to conversions.
• Post-Purchase Surveys: These are increasingly used as a low-fidelity, cost-effective method for initial incrementality assessments. They offer directional insights and can be a stepping stone toward more sophisticated testing methods.

As digital advertising continues to evolve, understanding and implementing these advanced strategies becomes increasingly important. The key is not just in gathering data but in interpreting it correctly to make informed, strategic decisions. By mastering the art of modeled audiences, conversion optimization, geo-testing, and incrementality testing, marketers can significantly enhance the effectiveness of their campaigns, ensuring they reach the right audience with the right message at the right time.