Understanding Geo Lift Test Results
Rockerbox's Test Results UI offers the summary-level outcomes and insights needed to interpret and act on test results delivered by Rockerbox's Incrementality Testing.
Currently, the UI supports geo-lift test types only and serves as a historical reference displaying any tests run through Rockerbox. Results will be seen in the UI upon conclusion of each test as results are analyzed and insights are derived.
🎨 Test Design Summary
- See a snapshot of test design details, putting your results in context
- Flip between test cells (heavy up, holdout) to view results from each test treatment
📈 Test Results Overview
- Measure incremental ROAS and CPA of the tested channel
- Compare key testing outcomes across all testing KPIs
- Understand statistical significance of each test result, building the confidence needed to act on test results
No Lift Observed: a test may conclude that no lift is observed in the below scenarios, meaning that the test had less of a result than expected. In these case Lift %, Confidence, and Incremental KPI will not be reported.
- Hold Out Result > Predicted
- Heavy Up Result < Predicted
- Hold Out Result > Predicted
- Heavy Up Result < Predicted
If the confidence level is < 0.8, confidence and incremental KPI will not be shown as results are not statistically significant. This is often seen if lift is positive but a small %.
🧮 Results Deep Dive
- Compare test outcomes (eg. revenue in test markets for holdout or heavy up) to the predicted result to both
- Understand the pre-test correlation of test markets to the counterfactual (should be highly correlated to ensure test validity)
- Analyze divergence during the test, indicative of lift over time
💡 Insights and Next Steps
- See test conclusions and learnings, analyzed by Rockerbox's professional services experts
- Reference Rockerbox's recommended next steps to put the test learnings into action in your media plan
