--- title: Best Practices for Incrementality Testing in Cassandra description: Incrementality testing is essential for measuring the true impact of marketing activities beyond natural trends and external influences. This guide outlines best practices for running reliable experiments, interpreting results, and applying findings. --- [Skip to content](https://docs.cassandra.app/migration/best-practices-for-incrementality-testing-in-cassandra#main-content) English Show submenu for translations [Open a Support Ticket](https://docs.cassandra.app/migration/kb-tickets/new?hsLang=en) ![64e670ef6e6ad43bbf323db2\_logo\_large-1-p-500.png\]](https://docs.cassandra.app/hs-fs/hubfs/Mktg%20Asset/64e670ef6e6ad43bbf323db2_logo_large-1-p-500.png?height=35&name=64e670ef6e6ad43bbf323db2_logo_large-1-p-500.png) Open main navigation Close main navigation - English Show submenu for translations - [Open a Support Ticket](https://docs.cassandra.app/migration/kb-tickets/new) - [Go to cassandra.app](https://www.cassandra.app/) [Go to cassandra.app](https://www.cassandra.app/?hsLang=en) Welcome. How can we help you? - There are no suggestions because the search field is empty. 1. [Knowledge base](https://docs.cassandra.app/migration?hsLang=en) 2. [Experimentation & Validation](https://docs.cassandra.app/migration/experimentation-validation?hsLang=en) # Best Practices for Incrementality Testing in Cassandra ## Incrementality testing is essential for measuring the true impact of marketing activities beyond natural trends and external influences. This guide outlines best practices for running reliable experiments, interpreting results, and applying findings. #### **1. Understanding Incrementality Testing** ### **What is Incrementality Testing?** Incrementality testing isolates the impact of marketing spend by **comparing a test group (exposed to marketing) with a control group (not exposed)** under similar conditions. The difference in outcomes between the two groups represents the **incremental effect** of marketing efforts. ### **Why is Incrementality Testing Important?** - **Validates model assumptions** about channel performance. - **Measures true impact** of marketing beyond platform-reported metrics. - **Calibrates MMM models** with real-world experimental data. - **Optimizes budget allocation** based on empirical evidence. - **Challenges or confirms platform-reported attribution.** --- #### **2. Best Practices for Running Experiments** ### **Types of Incrementality Experiments** 1. **Geo-Experiments (Geographic Holdouts)** - Divide markets into **test** and **control** regions. - Adjust marketing spend in test regions while keeping control regions unchanged. - Measure impact by comparing performance across regions. 2. **Conversion Lift Studies (Platform-Based)** - Conducted within platforms like Meta, Google, and TikTok. - Test and control groups are defined by the ad platform’s attribution system. - Measures the incremental impact of ads based on conversion behavior. 3. **Difference-in-Differences (DID) Experiments** - Used when external factors might influence both test and control groups. - Compares performance **before and after** marketing changes. 4. **Scale-Up Tests** - Increase spending in test regions and analyze whether incremental revenue aligns with expected ROI. - Helps define **saturation points** for scaling budgets. 5. **New Channel Introduction** - Launch a new channel or tactic in selected test regions. - Compare performance to control regions where the channel is absent. --- #### **3. Data & Experiment Design** ### **Data Requirements** To ensure accuracy, gather: - **Date**: Daily or weekly timestamps. - **Outcome KPI**: Total revenue, orders, or conversions. - **Geo-Granularity**: City, state, or DMA-level data. ### **Historical Data Requirements** - Minimum **4-5× the test duration** (e.g., for a 2-week test, use 8-10 weeks of historical data). - Optimal scenario: **1 year of historical data** when available. ### **Geographic Granularity Considerations** - Avoid **zip-code-level granularity** in large markets (too fragmented). - Optimal granularity: **State or region level** for stable test-control comparisons. - Ensure at least **100-250 locations** for robust statistical significance. ### **Designing a Reliable Experiment** - **Duration**: At least **14-21 days** to capture full impact. - **Budget**: Align with expected lift and statistical significance. - **Geography Selection**: Use Cassandra’s algorithm to select **comparable test/control groups**. - **Lag Effects**: Consider delayed conversions (e.g., upper-funnel campaigns may take weeks to show results). --- #### **4. Running the Experiment** ### **Implementation Steps** 1. **Select test regions** using Cassandra’s recommendations. 2. **Adjust campaign targeting or budgets** accordingly. 3. **Monitor campaign execution** to ensure accurate spend and reach. 4. **Maintain consistent tracking** for total conversions (not just ad-attributed ones). 5. **Allow additional time** for lag effects before final analysis. ### **Best Practices** - Avoid **major changes** to other marketing channels during the test period. - Consider **seasonal factors and promotions** that may skew results. - Document **external factors** that could impact results (economic events, competitor actions). - Maintain test conditions for the **entire duration** without premature optimizations. --- #### **5. Interpreting & Applying Results** ### **Key Metrics for Analysis** - **Measured Lift**: Percentage difference between test and control groups. - **Statistical Significance**: Confidence level (aim for 95%). - **Incremental ROAS (iROAS)**: Revenue lift divided by additional spend. - **Total Incremental Revenue**: Attributed revenue beyond baseline. - **Confidence Intervals**: Range of estimated impact, indicating certainty. ### **Understanding Your Results** - If **measured lift is significant**, the tested channel or tactic has a proven impact. - If **confidence intervals are wide**, results may be inconclusive (consider longer tests or more data). - If **measured lift is lower than expected**, reconsider assumptions about channel ROI. ### **Using Results for Model Calibration** 1. **Input findings into Cassandra’s Calibration Panel**. 2. **Adjust channel ROI assumptions based on experimental results**. 3. **Re-run MMM models with updated priors** to refine future predictions. 4. **Use results to refine budget allocation and optimization strategies**. --- #### **6. Common Pitfalls & How to Avoid Them** | **Pitfall** | **Solution** | | --- | --- | | Test/control groups are imbalanced | Use algorithmic matching based on historical performance. | | External factors influence results | Document and adjust for promotions, economic changes, etc. | | Sample size is too small | Expand the number of test locations or extend test duration. | | Measuring too soon | Consider **delayed effects**, especially for brand campaigns. | | Platform-reported data is misleading | Compare results with **total business outcomes** (not just platform attribution). | --- #### **7. Summary & Next Steps** - **Use geo-experiments or conversion lift studies** to measure true incremental impact. - **Ensure strong test-control matching** to eliminate bias. - **Interpret results carefully**, considering confidence intervals and statistical significance. - **Use results to calibrate Cassandra models** and refine future budget allocation. - **Avoid common pitfalls** like small sample sizes, external disruptions, and incorrect attribution. - [Getting started with Cassandra](https://docs.cassandra.app/migration/getting-started-with-cassandra?hsLang=en) - [Data preparation & Model Calibration](https://docs.cassandra.app/migration/data-preparation-model-calibration?hsLang=en#main-content) - [Connectors - How to?](https://docs.cassandra.app/migration/data-preparation-model-calibration?hsLang=en#connectors-how-to) - [Optimization & Budgeting](https://docs.cassandra.app/migration/optimization-budgeting?hsLang=en) - [Experimentation & Validation](https://docs.cassandra.app/migration/experimentation-validation?hsLang=en) - [Forecasting & Strategic Planning](https://docs.cassandra.app/migration/forecasting-strategic-planning?hsLang=en) - [Troubleshooting](https://docs.cassandra.app/migration/troubleshooting?hsLang=en) # Cassandra cassandra.app Help Center Copyright © 2026, Cassandra