Rule of three calculator: zero failures in n trials

Zero failures in n trials does not mean a zero failure rate. The rule of three gives a rough 95% upper bound on the rate that is still consistent with a clean run. Enter the number of clean trials to see the bound, or set the failure rate you want to rule out to see how many trials that takes.

What it computes#

After $n$ independent, representative trials with no failures,

$$p_{\text{fail}} \lesssim \frac{3}{n}$$

is a rough 95% upper bound. The exact one-sided bound is $1 - 0.05^{1/n}$, which the calculator shows alongside. Ruling out a failure rate of one in ten thousand this way takes about thirty thousand clean trials.

Cautions#

• The trials must be independent and drawn from the population the claim is about. Clustered or correlated trials make the bound optimistic.
• Seeing even one failure at rate $p$ takes about $1/p$ trials under naive sampling, so sampling alone is expensive evidence about rare failures.
• The Bayesian form updates a uniform prior to $\mathrm{Beta}(1, n + 1)$, with 95% upper bound $1 - 0.05^{1/(n+1)}$, and extends to $s$ failures as $\mathrm{Beta}(1 + s, 1 + n - s)$.

The chapter Rare-event assurance covers the bound and the methods that need fewer trials.