The days when people were content to buy a demo on TV are long gone. From addressable to custom linear audiences, many advertisers are now making use of targeted audiences on TV, sourced from online segments, first party, or second party datasets.
It’s great that we can buy these audiences, but it’s important to understand they are often created only against a subset of the country. They might be based on subscribers to a particular MVPD, or visitors to an online website that has been matched against TVs. Although this isn’t a problem when we’re buying the campaign, it does means that when we measure it we’re missing the full picture of who we are reaching and how effective the campaign has been.
This leads us to overestimate the effectiveness of some campaigns, and underestimate others. Without a full picture across the country, we don’t know which campaigns are working and our new audiences are not better than TGI segments.
To get that full picture, we need to use data science to fuse together a patchwork of different datasets, each giving a partial view but together they make a whole. The mathematics of how we combine these datasets is a complicated, but it’s critical part of any attribution model.
So how can data science calibrate for 1000 audiences?
Traditional TV measurement representative panels work on a limited number of variables which they’ve calibrated for the TV universe. When we start to match in other target audiences, we are bringing in 1000s of new variables which we haven't controlled for, and we need to dynamically calibrate the results of each campaign accordingly.
At a high level, the new segment is going to skew the audience according to two factors: what proportion of the country is in the segment and what percentage of our universe is in the target group? As a simple example, consider a campaign to promote Netflix subscriptions that we want to measure with Smart TV viewing data. People with connected Smart TVs are significantly more likely have Netflix subscriptions than most of the population as Netflix is one of the primary drivers of connecting smart TVs. When we measure ad impressions from the Smart TV data, we therefore need to calibrate this down to make it nationally representative.
Say, for example, 90% of Smart TV owners have Netflix subscriptions, and only 30% of the country have Netflix subscriptions. In this case, we'd need to weight down our campaign effectiveness measure by a factor of three.
Adjusting for a single variable
Many of the more extensive census datasets are highly nationally representative and only need to be calibrated on a single variable, most frequently geography.
Here our data science conulting team will calculate viewership and conversions for each different segment, and then scale it up. Take an example based on gender. If we have a dataset where 60% of the audience is male, and 40% is female, we merely calculate % viewership and conversion rates for men and women separately and then adjust the male figure by 60/48 and the female by 52/40 to get a nationally representative figure.
When it comes to geography, it’s often more convenient to reduce the number of devices we include in our analysis to take only a few from each region. This ensures that the proportion from each area is nationally representative.
Either way, we can relatively quickly make nationally representative calculations when we are only calibrating on one variable. When we have multiple variables in play, it all starts to get a lot more complicated.
Handling multiple variables
If we're dealing with skews in, say, ethnicity and geography, our data science consulting work out not only how to calibrate the number of devices in each geography, but also how many devices with each ethnicity should be available in each geography. For ethnicity, we can probably source this data from census information, but if we're fusing in third-party sources such as Truck Intenders, it becomes a much more complicated problem.
Here we need to calculate what proportion in each geography should be Truck Intenders, so we have the correct number of devices in each geography and the right amount of Truck Intenders nationally.
To calculate this, we use the same statistical methods that Nielsen and the market research companies use to create panels. We are establishing the specific characteristics that we want to calibrate for, and then calculating how many devices we need with each combination. For example, if we're calibrating for Truck Intenders, geography, ethnicity, and presence of children, we'd need to calculate right down to how many devices should we have in Detroit with Hispanic ownership that have children and intend to buy a truck.
The best way to create these weighting is to use a method called Iterative Proportional fitting, and there are several useful python libraries that we use when are building these models for our clients. The input to the model is the number of people we expect nationally to have each of these characteristics, the number of devices we have with each set, and the model then creates weighting factors that we need to apply to make our results representative.
With this in place, we have created a virtual panel with our devices based on the specific characteristics we need to model. The virtual panel is nationally representative and within the margins of our sampling error, will produce the right results for any campaign we run.
What's interesting is just how different the nationally representative results can be from the raw data once you start to overlay more than one distinct variable. If you think your dataset is representative for geography, age, and ethnicity, try throwing in iPhone owners, electric car intenders and cat owners... you'll see just how badly you need to data science that representative model.