What is statistical significance?

The probability that your observed difference is not due to random chance. A p-value of 0.05 means there's a 5% chance you'd see this result (or more extreme) if there was actually no difference between A and B. Most teams use 95% confidence (p < 0.05) as the minimum bar before declaring a winner.

When can I stop the test?

Not just when you hit p < 0.05. Stopping early inflates false positives. Set a target sample size up front (based on expected effect size and MDE), wait until it's reached, then check the result. If you peek and stop mid-test whenever p drops below 0.05, you'll be wrong ~30% of the time in practice.

What does 'lift' mean?

The relative change in conversion rate. If control converts at 5% and variant at 6%, absolute lift is +1 percentage point, but relative lift is +20% ((6-5)/5). Relative lift is the number teams usually report — it scales with baseline and matters more for projecting revenue.

A/B Test Significance Calculator

SEO & Marketing· 10 tools

Free A/B test statistical significance calculator. Enter your control and variant traffic and conversion numbers, and get a two-tailed z-test result with p-value, confidence level, observed lift, and a 95% confidence interval on the difference. Uses the standard two-proportion z-test that Google Optimize, Optimizely, and VWO are built on.

A — Control

Visitors

Conversions

Conversion rate: 5.00%

B — Variant

Visitors

Conversions

Conversion rate: 6.00%

Verdict (2-tailed, α=0.05)

B wins

Confidence: 97.2%

Relative lift

+20.0%

Absolute: +1.00%

z-score

2.193

p-value

0.0283

95% CI (low)

0.11%

95% CI (high)

1.89%

Uses a two-proportion z-test (two-tailed) with pooled standard error. Don't stop the test early just because p drops below 0.05 — plan the sample size before running and wait until it's reached to avoid false positives.

Frequently Asked Questions

Related Tools

UTM Builder (Campaign URL Generator)
Build Google Analytics UTM-tagged URLs. Compose source, medium, campaign, term, and content parameters with proper URL encoding.
HTML Meta Tag Generator (SEO, OG, Twitter)
Generate a complete set of HTML meta tags: title, description, canonical, Open Graph, Twitter Card, and robots. Copy the full <head> snippet.
Keyword Density Checker (1-word, 2-word, 3-word)
Analyze keyword frequency and density in your content. See top 1-, 2-, and 3-word phrases with counts and percentage of total words.