New channel launch - evaluating performance

Updated by Litong Chen

Steps to take when considering launching a new channel or performing a lift testing.

Want a quick checklist to run through before launching with a new partner to ensure it's tracked in Rockerbox? You can view instructions here.

Before you launch a new channel - the prep work:

  1. Determine your test and control time periods 
    1. Control period = time period before any heavy up. (Ideally this is a time period without other significant shifts in marketing mix, promos, or heavy ups)
    2. Test period = equal time period to control during which you will heavy up channel spend. (This time period should also avoid other significant changes to marketing mix or spend heavy up across other channels as this will impact your ability to identify impact of spend changes)

Ideally this should be at least a 3 week test based on where we have seen success with other customers 

  1. Decide on % heavy up for test period 
    1. This % should be determined by your team based on current spend, budget, and conversations with your internal team
    2. It is important that the heavy up is significant enough to have stat sig read on performance changes.

After your channel is launched:

1. Immediately Post-Launch: Monitor Performance

What to expect: immediately after launching your channel test, users who have been exposed to your ads are likely still in the consideration period. This means

  • Spend: in most cases, spend will be available the day after launch. You can see spend in the Cross-Channel Report.
  • Conversions + Revenue: it may take a few days for the channel to drive any primary (ex. Purchase) conversion events.

How to measure initial performance:

  • Dive into an upper funnel conversion events (ex. View Product). How does performance compare to your other channels at a similar position in the funnel?
2. A few days or weeks into the test: Execute In-Channel Optimizations

What to expect: performance (CPA/ROAS) may start to normalize as users exposed to your ads begin to convert

How to measure performance:

  • Check your de-duplicated CPA/ROAS. How does it compare to other channels? Are there in-channel optimization opportunities you might start to execute?
3. A few weeks into the test: Scale, Maintain, or Cut Spend

What to expect: performance is normalized and users who will convert off of this channel or vendor have converted

How to measure performance:

  • Check your de-duplicated CPA/ROAS. Does it over or under perform compared to channels meant to play a similar role in the funnel? Begin to budget accordingly.
  • Check for channel overlap. Do users who have this channel on their path to conversion also have other channels meant to play the same role in the funnel? (Ex. overlap of multiple display retargeting vendors or overlap of TikTok and Linear TV.)
  • Understand your time to convert. Based on the channel's average time to convert, were users taking more vs less time to convert and interacting with different marketing touchpoints? Do you expect that users who have been exposed to an ad from early launch to have converted? If time to convert is longer than average, users may not have converted yet -- meaning you might wait longer for performance to normalize before cutting spend.

Once you completed your channel testing:

Use “Time period Comparison” feature in the Cross-Channel Report, to view performance between 'control period' vs. 'test period' to understand the overall effectiveness of your campaign for CPA/ROAS.

How did we do?