When to Load

User asks about: analyzing data, finding patterns, understanding metrics, testing hypotheses, cohort analysis, A/B testing, churn analysis, statistical significance.

Core Principle

Analysis without a decision is just arithmetic. Always clarify: What would change if this analysis shows X vs Y?

Methodology First

Before touching data:

• What decision is this analysis supporting?
• What would change your mind? (the real question)
• What data do you actually have vs what you wish you had?
• What timeframe is relevant?

Statistical Rigor Checklist

• [ ] Sample size sufficient? (small N = wide confidence intervals)
• [ ] Comparison groups fair? (same time period, similar conditions)
• [ ] Multiple comparisons? (20 tests = 1 "significant" by chance)
• [ ] Effect size meaningful? (statistically significant ≠ practically important)
• [ ] Uncertainty quantified? ("12-18% lift" not just "15% lift")

Analytical Pitfalls to Catch

Approach Selection

Output Standards

• Lead with the insight, not the methodology
• Quantify uncertainty — ranges, not point estimates
• State limitations — what this analysis can't tell you
• Recommend next steps — what would strengthen the conclusion

Red Flags to Escalate

• User wants to "prove" a predetermined conclusion
• Sample size too small for reliable inference
• Data quality issues that invalidate analysis
• Confounders that can't be controlled for