Evaluate your data onboarding partners using match tests

Marketer demands for accurate data, superior audience reach, and effective targeting have not changed, but data onboarding provider performance across each of these areas has become increasingly important.

To stay competitive, companies are starting to conduct head-to-head “match tests.” What is a match test? It’s a way for marketers to evaluate data onboarding providers—a practice that increased dramatically in the past year. 

We see this shift as a good thing—as long as the match test is conducted appropriately, ensuring the metrics provided are a direct representation of each onboarding provider’s performance when compared to each other.

Many match tests are flawed, rendering metrics that might be interesting but aren’t an equal comparison of onboarding providers. These flawed tests fail to get at the heart of what is most important to advertisers needing a data onboarding solution. 

How to perform a match test 

Marketers supply each vendor with an offline CRM file to be converted into a digital audience to reach that same audience online. Once the offline data is ingested and matched internally, each onboarding provider outputs a match report containing various offline and online match metrics.   

Reach, accuracy, and performance: A holistic approach 

Reach

Evaluating audience reach, or what percentage of a CRM universe matches to at least one cookie, is a valuable metric to consider. However, it can be misleading if the platforms being evaluated in a match test reflect reach differently.

Onboarding providers may define “reach” as the number of offline users a marketer will be able to reach online, while DMPs or DSPs have different default expiration dates for the matched cookies that get reflected in their platforms.

Marketers should verify with the different vendors involved in the test that the reach metrics they provide align with each other. A reach test based on vendor metrics or DSP metrics only is not the same as a live campaign reach test.

Accuracy

Evaluated independently, reach may only tell half of the onboarding story, and it can incentivize vendors to supply the largest possible cookie audience without respect to cookie status, accuracy, and performance. To avoid vendor bias, marketers should ensure the shelf life of cookies is being accounted for equally across all vendors being evaluated in the test.

If one vendor supplies 30-day active cookies with a heavy bias toward 0-14 day cookies, but another vendor provides a 90-day active cookie audience, the matched cookies reflected within the downstream media partners will not be an apples-to-apples comparison.  

Cookie accuracy is an equally important aspect of match tests that many marketers choose to ignore because of the level of effort required to do so. But what if a significant percentage of the cookie universe is incorrectly matched? Choosing not to test accuracy could result in a marketer burning their media budget on poor-performing audiences that fail to reach the target audience.  

Performance 

A leading multichannel retailer recognized that historic approaches to conducting match tests could contain flaws. To avoid common pitfalls, the retailer organized its test to evaluate reach, accuracy, and performance across vendors resulting in a robust, multivendor test.  

1. To start, they divided their CRM universe into equal, randomly selected audiences and assigned one audience to each onboarding provider participating in the head-to-head test.
2. Once each provider received the associated audiences containing PII, they matched the offline records internally, translated them into cookies, and synced them to the same DSP.   
3. The DSP ran a live campaign against each audience, ensuring it used the exact same bid strategy for each. 
4. Media was run for each audience until the same number of orders for each onboarding provider being evaluated was generated.
5. To measure performance, the retailer analyzed the media spend required to generate those orders (cost/ROI).
6. For the accuracy component, the retailer compared the consumers who placed the orders against the original CRM audience initially provided to each onboarding provider, at an individual person level.  

Through this process, the retailer could comprehensively assess audience reach within the DSP platform in addition to targeting accuracy and performance in a “live” context at the impression level for each audience and across each onboarding provider.  

Alternative approaches 

There are alternative approaches for evaluating online accuracy if a marketer’s business is largely or entirely offline. For example, an advertiser can offer a substantial discount, but to receive an offer or coupon, users will be required to register an email address with the advertiser.

This approach collects PII against a specific ad that was served, measures the accuracy of who was served the ad, and evaluates live reach against the cookie audience.  

Put your provider to the test

There’s a lot at stake when choosing an onboarding provider who will be a close partner for many years. Marketers should always encourage partners to participate in a match test to substantiate the efficacy of their solution.

Using the tips outlined here, marketers can ensure they have a well-constructed test to identify the strongest performing provider across reach, accuracy, and performance.