How to Interpret Attribution Data and Prove Campaign Effectiveness

September 23, 2022 -
By Scott Konopasek

As an agency marketer, every week I would spend time digging into data to produce insights and recommendations so my clients’ campaigns performed better. I’m going to share the framework I would use weekly to analyze results.

When it comes to interpreting advertising and attribution data, you need to understand the context of the overall business you’re working for as well as the marketing department. Think things like seasonality, goals, budgets, and experiments.

First we’ll cover the different layers of analytics, then the steps you can use to make data more valuable, and then finally look at specific situations in which you’ll put all this into practice.

The Four Layers of Marketing Analytics

There’s a hierarchy to how you can analyze marketing data, and as you practice each and become a more skilled analyst, the higher-level functions become easier.

Data Puking

Data Puking is the lowest level of analytics. The problem with most reporting dashboards is that they just dump out tons of information. It’s usually an overwhelming amount, and—like most people faced with too much data—you end up not doing anything because the volume prevents you from finding meaning in it all. 

Data puking is like a firehose of marketing data. You can’t use it to make decisions because there are too many variables and not enough context. The fix for data puking is to reduce the amount of data you show in reporting, and learn how to write observations.

Observation

Observation is simply stating the obvious. It may be repeating some metrics to drive home a point, but it’s mostly just summarizing data. The big difference between observation and data puking is that you’re filtering out irrelevant information and starting to focus on a few key data points. 

The reporting that gets provided to most clients or senior leaders falls into this category—it’s more relevant but often falls short of delivering meaningful insights.

Insight

Insights are conclusions drawn from metrics, which require contextualizing the data by explaining why something happened or how it relates to other events.

Insight = metrics + why

Insights is about making sense of the data, not just reporting it. Insights require more than just a set of numbers and some context—they answer questions like: How is this information relevant? Why should we care about this? What does this mean for us going forward?

If you do a good job generating insights, the highest-level of analytics, application, becomes much easier.

Application

The most important part of analysis is the application. Proper application answers questions like: So what do I do? How can I use data to make better decisions? 

The best data-driven organizations are able to take a step back and ask what insights they can extract from their data. Then, they use those insights to make decisions that drive better outcomes and ultimately impact the business in a meaningful way.

Now that you understand the 4 layers of marketing analytics, here’s how you can put them into practice.

The Interpretation Framework

Step 1: Discovery

Estimated Time: 20-45 minutes per platform

  1. Spend Time With The Data. This is a matter of diving into reports to find your standard key performance indicators—the things that you know or care about as well as what’s important to your client.These are the areas to look at first. You should also take some time to familiarize yourself with the data, as well how it’s structured. This will help you determine what questions to ask next.
  2. Look For Inconsistencies. As you take in all this information, look for inconsistencies, and search for things that stand out— anything that has changed pretty dramatically recently or is unusual. This will help you catch opportunities to exploit or issues to fix in your campaigns.
  3. Get Nuanced With Each Data Set. To get the most out of your data, you must examine each source individually. Comparing Facebook campaign data to Google AdWords campaign data will give you different results than if you compared it with programmatic. Any time that you have multiple data sets, it’s important to understand how each one of those reports was generated and if the data is siloed or deduplicated. This will make sure you’re not comparing apples to oranges.
  4. Stay Open-Minded. In order to make the most of discovery, you need to be open-minded. This can mean looking in new and different places within your reporting tools. What I typically find is that as I get into a campaign and start to review the data, there are going to be some things that strike me as interesting or odd. It’s always different data points and metrics, so approaching each discovery session like it’s a new experience helps you spot things you might otherwise miss.

Step 2: Focus your analysis

Estimated Time: 45-60 minutes

After you’ve spent time in each ad platform to familiarize yourself with all the campaign data, it’s time to focus on specific questions.

This is where you have to ask yourself, what am I trying to do with this data set? Consider things like:

  • Who will receive this report? Someone who’s a senior leader, or someone who’s hands-on-keyboard?
  • Is this data for internal or external use?
  • Are you trying to optimize the data?
  • What’s the context and intent of this report?

Understanding these questions will let you know what questions you should focus on, which drives what data you’ll look at and how you’ll express your observations.

Here are some questions you might focus on answering in a session like this:

  • What channels are driving growth?
  • What’s the ideal media mix?
  • How does X channel fit into the path to purchase?
  • If I had an extra $5,000 of ad budget, where do I put it?
  • If I had to pull $5,000 of ad budget, where would I pull it from?
  • How can I prove the value of channel X?

The last, and possibly most important question to ask is “why?” Ask it again. And again. This will lead you to find meaningful insights.

By asking these questions, you’ll be able to understand what your data says. You know whether or not you need to optimize based on existing results; if you’re behind target and need make some changes—or if things are going as planned! By focusing on your important questions, you’ll be less likely to get overwhelmed with all the data at hand.

Step 3: Build Insights

Estimated Time: Varies

Building insights is the step where things get more challenging. Stitching data together to create valuable insights requires practice. It’s easy to fall into the trap of stating observations, but thinking that you’re actually sharing insights.

“CTR increased by 10%” is not an insight. It’s just an observation.

Remember: Insights = metrics + context

You can turn that observation into an insight by adding this context: “CTR increased by 10% because we increased avg. frequency to 3.5″. Now that’s an insight!

Another example: an observation is “conversion rate doubled month over month”. This is missing the context that transforms the data into an insight: “we changed ad copy to be customer-centric and conversion rates doubled”.

As you build insights, keeping business context in mind is also important. If sales are up 20%, don’t just consider any new tests you launched in the last weeks, also consider your business’ seasonality and other unique circumstances that may be influencing buyer behavior.

Step 4: Apply the Data

Estimated Time: Varies

After you’ve familiarized yourself with all your campaign data, focused on important questions and built insights, you’re ready to apply the data and make things better. Application is by far the hardest part of this process and it’s the most valuable to leadership and/or clients.

There’s no simple formula to follow to build applications. What I’ve found works best is to ask questions around your observations and insights to reveal what can be changed. Here are some questions I like to ask:

  • Is this good/bad? Why?
  • Should I do more/less of this?
  • What is efficient?
  • What is scalable?
  • How does this relate to other ad channels?
  • What optimizations could I make as a result of this?

As you sit with your data and ask these questions, you’ll find ways to apply your insights to drive better performance.

Bonus Tip: to get the most out of your data, talk with channel specialists about what is available and how it can be interpreted. Because they have different priorities and goals, team members will view the data differently than you do. This is beneficial because you can arrive at some really interesting applications by getting a fresh set of eyes on your data.

Data Source Use Cases

Now that you understand the layers of analysis and how to practice each, let’s talk about the specific use cases in which you’ll find yourself. These use cases will differ based on what data sources you’re using, from ad platforms to advanced attribution tools.

One thing to note is that steps two and four, Focus and Application, are situationally dependent, but don’t change with data sources. Steps one and three, Discovery and Building Insights, will change based on what platform is providing the campaign data.

Ad Platforms & Google Analytics

Frequency: at least weekly, if not several times each week

The first data sources we’ll address are ad platforms. This includes Google Ads, Bing, Facebook, Instagram, TikTok, etc.

In the Discovery Phase look at the data for each individual level: campaign, ad group, ad set; creative ; audience. Also pay attention to how metrics are doing over time – week to week and month to month. You’ll want to examine any changes in Facebook and Instagram, and determine how those changes will affect your creative approach (such as what kind of content you post) or audience.

In order to understand how well platforms perform, it’s important to compare different look back windows. Comparing look back windows like 7d post click/7d post view to 7d post click/1d post view will help you understand how the platforms are attributing results, and if there’s a big variance you can build insights into why there’s a difference.

In the Build Insights Phase, you’ll want to ask questions like these:

  • Which creative elements are driving improved results?
  • What supporting metrics explain why results changed? (frequency, CTR, time on site, pages viewed etc.)
  • If conversion rate has increased, try to find where in the buyer’s journey things changed. Did more people click or make it past the landing page? Were there fewer cart abandons?

As you answer these questions, you’ll notice trends and changes and the big question to answer here is why those things are happening. If conversion rates are up, is that because more people are clicking on ads or could there be other influencing variables like a lower abandon cart rate?

Adserver

Frequency: weekly, possibly twice per month

In the Discovery Phase you’ll be focusing on display, video, and native ads separately from search and social. Begin by looking at the different ad formats. For example, is a rich-media ad format better than standard display ads? Does placing an ad on mobile perform better than desktop displays? Variables here will be things like size of creative, different channels, device type, etc.

You should also view post click and post view activity in your ad server. There is a lot of post-view activity, so you want to understand what’s the share from each—is it 90% post views? Is it 60%? You want to figure out the conversion rate for each of those.

Another data point to view is time of day and time of week, and finally paths to purchase.   It’s possible to get a report about how users engage with ads in the ad server. But it’s difficult, because you have tens of thousands of different paths for people who click on an ad: some may receive multiple clicks from one user; others might be coming through more than once per day. This might give insights around ideal frequency, but don’t get too caught up on path to purchase reports.

In the Build Insights Phase, ask questions like these:

  • What are the channels with the highest site visit rate? Conversion rate?
  • What business context helps explain the results?
  • What delivery parameters (day, time, frequency), can be optimized?

Credit-Based Attribution

Frequency: quarterly, possibly monthly

Credit-Based Attribution will be any type of Media Mix Modeling or Multi-Touch Attribution tool.

In the Discovery Phase you’ll want to compare different attribution models (both custom and standard) to discern how ad spend is turning into results. One useful practice here is to adjust your look back windows to see how credit changes in each model as well. Every credit-based attribution will have different ways of expressing media mix performance, so make sure to spend time there to understand how channels work together to produce results.

In the Build Insights Phase, ask questions like these:

  • What are the similarities and differences between attribution models?
  • How is performance changing as a result of prior optimization?

Note: we can’t be as specific here as with other data sources because there’s so much variability in every credit-based attribution platform.

Bi-Modal Attribution

Frequency: twice per month, possibly monthly

In the Discovery Phase you’ll want to start by evaluating at the channel level, then dive deeper to compare specific platforms and creative performance. The first data to understand is each channel’s shared and incremental contributions to results with metrics like reach, clicks, and conversions.

After viewing each channel’s incrementality and media mix lift, start to look for the media mixes that have the highest conversion rates and those with the most conversion events. This will let you understand what delivery parameters are driving different levels of performance.

In the Build Insights Phase, ask questions like these:

  • Where are there opportunities to build cross-channel frequency?
  • What are the average conditions for conversion, and where is there an
    opportunity to align with that?
  • What supporting metrics explain why results changed? (frequency, CTR, time on site, pages viewed etc.)

Interpreting Attribution Data

Diving into data regularly to interpret and uncover insights about your campaign is a great idea for any agency marketer. Here are some things to remember:

  1. Conduct ‘data discovery’ before you try to analyze.
  2. Have a clear sense of what you’re trying to do and who will be receiving the data.
  3. Generate insights, don’t data puke. Add context to a metric, explaining why or how.
  4. It’s hard, but ask ‘what are the implications?’ and apply the data; answer “so what do I do about this?”
  5. Follow the frameworks specific to each type of data set.
  6. Keep an open mind, ask “why?”