How to Build a Testing Roadmap

Updated by Will Burghes

Planning a thoughtful, strategic approach to testing is critical to getting reliable insights. This guide walks through how to create a learning agenda, prioritize your tests, and plan a calendar that avoids disruptions—so your results are clean, actionable, and business-relevant.

✅ How to Prioritize Your Tests

When deciding which channels and strategies to test first, consider the following:

  1. High Spend Levels
    Channels with larger or growing budgets should be prioritized—small inaccuracies here can lead to big consequences.
  2. Suspected Low Incrementality
    Tactics like remarketing, brand search, and affiliate often appear to perform well but may not drive net-new conversions. These should be tested to validate their true impact.
  3. No Other Measurement Available
    For hard-to-measure channels like print or influencer, testing may be the best way to evaluate performance in the absence of attribution or MMM.
  4. Conflicting Measurement Results
    If attribution and MMM results don’t align for a channel, testing can help uncover the reason for the discrepancy.

📆 When (and When Not) to Run Tests

To ensure your tests are as clean and unbiased as possible:

  • Only test during peak promotional periods like BFCM, Prime Day, product launches, sponsorships, or major distribution events if you specifically want to understand the impact of media during these events.
  • Avoid known competitor product launchs or promotional events if you have insights into their calendar, since these moments can decrease the impact of your media.
  • Ideally, aim for normal periods in your sales cycle—between promos or seasonal spikes, if you want to understand the normal impact of media.

Regardless of when tests take place. they should be carefully scoped to avoid major revenue impact.

🔁 When to Retest

You should consider retesting if:

  • Your brand awareness or brand strength have changed significantly
  • The media available on the platform being tested has changed
  • Your buying strategy on the platform (i.e. buying objective or audience) has evolved
  • Your competitive landscape has shifted
  • The amount you spend on the channel has increased significantly
  • You want to get deeper insights into the channel, such as the impact of increased (instead of business-as-usual) spend

These changes can all impact how media performs, and retesting ensures your insights stay accurate.

How did we do?