n per variant ≈ [1.96√(2p̄(1−p̄)) + zpower√(pA(1−pA)+pB(1−pB))]² ÷ (pB−pA)²How to use this calculator
- Enter an expected baseline conversion rate between zero and 100%.
- Choose the smallest relative lift or percentage-point increase worth detecting, and select 80% or 90% power.
- Review the estimated participants per group. Optionally enter total eligible visitors per day for a traffic-only duration estimate.
Worked example
An illustrative baseline converts 5% of eligible participants. A 20% relative lift means detecting 6%, an increase of one percentage point. With equal allocation, two-sided alpha 0.05 and 80% power, this approximation requires about 8,158 participants per group, or 16,316 total. At 1,000 eligible participants per day, the traffic-only estimate rounds up to 17 days.
Minimum detectable effect is a planning input
The effect is the increase you want the design to have the stated power to detect. It is not a prediction that your treatment will achieve that lift. A 20% relative lift on a 5% baseline means a 6% target; a 20-percentage-point increase would instead mean 25%. Switching the definition preserves the entered effect when the baseline is valid.
Power and significance answer different questions
The design fixes the two-sided significance level at 5%. Power is the planned chance of detecting the specified alternative under the model assumptions. Increasing power from 80% to 90% generally needs more traffic. Neither setting guarantees that a completed test will produce a significant result.
A sample count does not define the entire test
Use independent randomized participants and a binary outcome, with one outcome per participant. The form assumes exactly two groups with a 50/50 split and a fixed analysis point. Conversion delay, weekday patterns, clustering and measurement quality still need a plan. It does not adjust for many variants, repeated peeking or revenue metrics.
Reference: statsmodels: two-proportion sample-size approximation.
Common questions
Is the result per variant or for the whole test?
The headline is participants per variant, rounded up. The total is twice that count for the two equally allocated groups. Daily eligible traffic is the total across both groups, not traffic for each group separately.
Which sample-size method is used?
A closed-form normal approximation with pooled variance under the null and unpooled variance under the alternative, without continuity correction. It uses alpha/2 for a two-sided design and neglects the far tail in the power approximation. It is not an exact-binomial calculation.
Why can the estimate be unavailable?
The target must be higher than the baseline and both rates must be strictly between zero and 100%. The tool withholds estimates when expected converters or non-converters are below 10 per group, or the count exceeds its numeric limit. Sparse outcomes need an appropriate alternative design.
Can I stop as soon as I reach the traffic count?
The count is only one design requirement. Allow outcomes to mature and cover the business cycles specified before starting. Repeatedly checking for significance and stopping opportunistically changes the interpretation of a fixed-sample analysis. Use the separate A/B calculator at the planned analysis point.