Model Fit for MMM
This documentation is designed to help you understand the expectations and estimate KPIs regarding the fit of your MMM model.
Understanding Model Fit
1. Purposeful Simplicity
Our MMM models are intentionally designed to be simple, prioritizing interpretability to offer better insights. While more complex models might yield higher predictive accuracy, they often become harder to interpret, limiting their usefulness in deriving actionable insights.
2. Data Input Limitations
It's essential to note that our models are trained using high-level spend data. While this is a crucial factor in predicting marketing effectiveness, it cannot capture all the nuances of consumer purchasing behavior. This limitation is intrinsic to the nature of MMM and influences the achievable model accuracy.
Key Performance Indicators (KPIs)
1. R^2 (R-Squared)(Coefficient of Determination)
R^2 measures the proportion of the variance in the dependent variable (revenue) that is predictable from the independent variables (marketing spend).
The achievable R^2 values will vary customer by customer based on the quality of data inputs and the impact of external factors. Rockerbox aims for a target around 70% or higher, but this is a flexible target that may vary.
2. MAPE (Mean Absolute Percentage Error)
MAPE represents the average percentage difference between the predicted and actual values. In the context of MMM, a lower MAPE indicates better accuracy. However, considering the model's simplicity and data limitations, achieving extremely low MAPE values may be challenging.
In general, Rockerbox aims for a MAPE of 25% or lower. This will vary from customer to customer due to the uniqueness of each marketing scenario. We believe that each model should be evaluated based on its ability to provide actionable insights rather than rigid adherence to predefined benchmarks.