MMM - Guidelines for Feature Selection
Rockerbox's Marketing Mix Modeling (MMM) measures the impact of your marketing on your target KPI—typically revenue or conversions (signups, subscriptions, and so on). We break down your marketing into features, which can be channels, platforms, strategies, or combinations of the above.
During onboarding, our team collaborates with you to identify the features to include in the model based on the insights you seek. For example, if you want to understand the impact of Facebook Retargeting compared to Facebook Prospecting, we will strive to incorporate that level of detail into your model. However, effective modeling requires specific conditions to generate meaningful insights, necessitating trade-offs between detailed breakdowns and strong signals. This page reviews guidelines we use when determining which features to include in the model.
General Guidelines
- Significant Spend Representation
- Criteria: Represents at least 5% of total spend throughout the training period.
- Rationale: Spend less than 5% likely impacts your KPI within the model's error rate, making it difficult to distinguish from noise.
- Sufficient Activity Duration
- Criteria: Must be active for at least one quarter (25%) of the training period. Typically, the training period spans 2+ years.
- Rationale: Limited data points hinder the model's learning ability and reduce the reliability of insights.
- Low Correlation with Other Features
- Criteria: Must not be highly correlated with another feature.
- We aim for feature correlation to be less than 75%. If two or more features' spend is 70+% correlated, those features will often need to be combined. Features where spend is 75+% correlated are at risk of needing to be combined.
- Rationale: The model has difficulty differentiating the impact on the KPI if multiple channels move in the same direction. In such cases, correlated features are often grouped together.
- Criteria: Must not be highly correlated with another feature.
- Spend Not Driven by the KPI
- Criteria: Feature spend must not be determined by, or mechanically tied to, the conversions or revenue you’re measuring.
- Rationale: When spend follows conversions rather than leading them, the model can’t separate the feature’s true impact from the KPI itself. It credits the feature with conversions it merely tracked, which understates the contribution of the channels that actually drove demand. Two common cases:
- Branded Search: Spend and clicks rise when people are already looking for your brand, which is usually the result of other marketing or existing demand. The channel captures intent that was created elsewhere, so the model overstates its impact and understates the upstream channels that created that intent.
- Affiliates paid on a CPA basis: Spend is calculated from conversions (conversions × payout), so it’s a direct function of the KPI. Any relationship the model finds is built in by the payout structure, not evidence of incremental impact.
- Outcome: Excluding these features keeps the model from crediting channels for demand they captured or conversions they were paid on, so credit goes to the channels that drove incremental results.
By adhering to these guidelines, Rockerbox ensures that your MMM provides actionable and accurate insights into your marketing performance.