MMM - Guidelines for Feature Selection
Rockerbox's Marketing Mix Modeling (MMM) measures the impact of your marketing on your target KPI—typically revenue or conversions (signups, subscriptions, and so on). We break down your marketing into features, which can be channels, platforms, strategies, or combinations of the above.
During onboarding, our team collaborates with you to identify the features to include in the model based on the insights you seek. For example, if you want to understand the impact of Facebook Retargeting compared to Facebook Prospecting, we will strive to incorporate that level of detail into your model. However, effective modeling requires specific conditions to generate meaningful insights, necessitating trade-offs between detailed breakdowns and strong signals. This page reviews guidelines we use when determining which features to include in the model.
General Guidelines
- Significant Spend Representation
- Criteria: Represents at least 5% of total spend throughout the training period.
- Rationale: Spend less than 5% likely impacts your KPI within the model's error rate, making it difficult to distinguish from noise.
- Sufficient Activity Duration
- Criteria: Must be active for at least half of the training period. Typically, the training period spans 2+ years.
- Rationale: Limited data points hinder the model's learning ability and reduce the reliability of insights.
- Low Correlation with Other Features
- Criteria: Must not be highly correlated with another feature.
- We aim for feature correlation to be less than 60%. If two or more features' spend is 70+% correlated, those features will often need to be combined. Features where spend is 60-70% correlated are at risk of needing to be combined.
- Rationale: The model has difficulty differentiating the impact on the KPI if multiple channels move in the same direction. In such cases, correlated features are often grouped together.
- Criteria: Must not be highly correlated with another feature.
- Low Correlation with Key Performance Indicators (KPIs)
- Criteria: Must not be highly correlated with your KPI (e.g., Revenue, Conversions).
- Rationale: Including highly correlated features can skew the model by attributing too much credit to them, thus underestimating the impact of other features. This often happens with channels like Branded Search and Affiliates paid on a CPA.
- Branded Search spend typically occurs just before purchase, making it appear as if it directly causes conversions, when instead it's a step on the path to conversion.
- Affiliate CPA is highly correlated with conversions because it is recorded at the time of purchase, but it is not necessarily causal.
- Excluding these features ensures a more accurate and balanced attribution across all marketing channels.
By adhering to these guidelines, Rockerbox ensures that your MMM provides actionable and accurate insights into your marketing performance.