Over half of shipped features are never used. Make every launch a testable hypothesis: a two-proportion z-test on control vs variant, judged against a real significance bar. A win only ships if it clears the bar — and an underpowered test returns inconclusive, never a false winner.
Honest scope. A deterministic significance test — a pooled two-proportion z-test (Acklam inverse-normal for the critical value); it decides ship/keep/inconclusive from counts you supply. It does not design the experiment, assign traffic, or attribute revenue (your product does that). It's the measurement half of the loop the rollout kernel then ramps. FAIL-CLOSED — samples below the floor, conversions > trials, zero variance, or a confidence outside (0,1) return inconclusive; significance uses the unrounded z. Source: flowdesk-experiment-kernel.js.