What publishers need to setup a first-party data test

Testing is an integral part of building out a first-party audience strategy. As a publisher, it allows you to prove the value of your cohorts, shows advertisers that first-party data performs, and provides an opportunity to refine your data strategy. 

Whether you are answering RFPs that require first-party data audiences or testing third- versus first-party data, the results you get will enable you to prove the value and produce case studies to take your first-party audiences out to market to win new business.

There are numerous ways to design tests but following these simple rules will ensure all the hard work, and impressions, you put into the test are valid:

1. Select one variable

The most important rule when designing a test is that there must only be one variable (unless you are doing a multivariate test, which is a much more complicated setup).

When testing how your first-party data performs compared to other data sources, that one variable will be the audience. All other testing parameters must remain the same, including the budget assigned to each variable. These are campaign type, time frame — start and end date, impression volume, geo, and ad type. 

If you are wondering how to determine the right impression volume for a test, unfortunately, there isn’t an exact number but with audience testing the more the better. 

Think of it this way, if you have 50 impressions and one click compared to 50 impressions and two clicks the click-through-rate (CTR) is going to be much higher but the data pool will be so small that the learnings aren’t accurate or meaningful. This is a great resource if you want to learn more about defining test parameters.

2. Set success metrics prior to starting the test

Designing your test to have a measurable outcome is critical. The best approach is to work with an advertiser with who you already have a strong relationship so you can ask for transparency in the data on their side. Here are a few examples of metrics you can look at when designing first-party audience tests:

Direct Response:

• Cost-per-acquisition (CPA)
• Net-response-rate (NRR)
• Conversion rate (CVR)
• Click-through-rate (CTR)*

*While you can use CTR as a success metric, it is not always indicative of campaign success. If possible, use this as a secondary metric only.

Brand Lift:

• Lift in awareness, consideration, and/or intent
• Viewability, Audible and visible on complete (AVOC), and/or Engagement (e.g. time spent on the page)

Efficiencies

• Increase in scale
• Performance by browser (e.g. ability to target in Safari and/or Firefox)
• Cost-per-mille (CPM) reduction

To ensure you are gathering the right test information, and selecting the correct variables and success metrics, download this template for setting up a first-party data test.

3. Selecting the right advertisers

Work with advertisers that are reliant on third-party data but want to find a future-proof solution to their strategy. It may take a bit of education on your part to shift their mindset, but the end result will allow you to build deeper and more innovative partnerships.

4. Set expectations internally

As with all testing and learning, the final results may not always be what you expect. Even if you don’t see an improvement in the results, what’s most important is being able to prove that you can replace third-party data with your first-party audiences. You also want to use the results to showcase the variety of targeting options your advertisers can tap into.

5. Take your results to market

After your tests are completed and you’ve analyzed the results, you now have additional proof points that you can take to market. Add the test results to your sales collateral, update your media kit, and put together a case study showcasing the value of your first-party audiences. Ultimately, you’ll be able to use the data you’ve accumulated to help win more deals with advertisers and further educate them on why they should be including first-party audiences in their strategy. 

Our customers are testing and proving the value of their audiences  

Insider saw an 11% performance increase with first-party data. The team worked with a financial client with strict audience targeting parameters. The advertiser had previously relied just on third-party data. Insider conducted an A/B test on the company’s behalf that compared a campaign using first-party data with one relying on third-party data. Against the advertiser’s KPI, the first-party data segment outperformed by more than 11%.

Lia Sur, Senior Director, Programmatic & Data Strategy at Insider says: “It all comes down to educating buyers that the demise of third-party cookies is a big deal and not something that can wait until the last minute. 40% of inventory is already not targetable, so while third-party deprecation is big news, many advertisers are already not reaching this audience. This is a great environment to start testing so that when the change comes they are ready.”

Livingly saw a 5x increase in spend and a 10% improvement on viewability with first-party data. Livingly Media has taken an innovative approach to allow advertisers to shift to its first-party data. It provides advertisers with the option to test their first-party data against existing third-party audiences. Brands can see the results for themselves and are empowered to make data-driven decisions.

After comparing inventory, one large CPG brand naturally gravitated towards using first-party data to reach niche audiences and inventory across all browsers. The brand increased spend with Livingly, as the publisher helped them reduce reliance on third-party data and wasted impressions while also increasing scale using premium accurate data. Jonathan Penn, SVP Revenue at Livingly, said: “It’s best to go after advertisers who are currently using third-party data [when looking for test partners] and to not limit testing to only contextual. Do your homework and ask the right questions ahead of time.”

To get started, we’ve created this template to ensure you gather the information you need to set up a successful first-party data test.