When a Product Is Pulled: Is the Defect Rate Really Too High?

A product disappears from the shelves, and the headline says the defect rate was too high. But how high is too high, and how do you know from a handful of checked items? Quality control is one of the oldest uses of statistics in business, and the logic behind it is the same one that decides a recall.

One sample against a standard

Suppose a specification allows at most 1% defective units, and inspectors check 400 items and find 9 defects, a sample rate of 2.25%. Is that bad luck or a real problem? If the true rate were exactly 1%, the binomial distribution calculator shows that 9 or more defects would appear only about 2.1% of the time.

The one-sample proportion test agrees. Its z-statistic is about 2.5, with a one-sided p-value near 0.006, so the data are hard to reconcile with a 1% defect rate. The confidence interval for a single proportion runs from roughly 0.8% to 3.7%, a wide range that still includes values above the limit.

Why the sample size matters

Now imagine a larger check: 14 defects in 2,000 items, or 0.7%. That is below the limit and is entirely consistent with it, since the chance of 14 or more at a true 1% rate is about 93%. A small sample can raise alarms or miss real problems, and a large one is much more informative.

Deciding about a whole batch

Factories often use acceptance sampling, where a lot is accepted if a sample contains no more than c defects. A single sampling plan is defined by the pair (n, c), and the plan's operating characteristic curve shows the chance of accepting lots with different true defect rates (NIST handbook).

For a plan that checks 200 items and accepts a lot with 2 or fewer defects, the acceptance probability is about 68% when the true rate is 1%, 24% at 2% and 6% at 3%. The Poisson distribution calculator approximates the same numbers for rare defects. No plan is perfect: good lots are sometimes rejected and bad lots sometimes accepted.

What sampling cannot do

Acceptance sampling extrapolates from the units measured to the units not measured, which assumes the sample is random and representative (Quality Digest). It also decides about lots rather than improving the process that makes them. A pattern of failures points to causes upstream.

The two kinds of error

Any decision rule can fail in two ways. A recall that was not needed is a false alarm that wastes money and trust, while a missed recall is a false reassurance that can harm customers. Making the rule stricter reduces one error but usually raises the other, unless the sample grows. That is why quality plans specify both a defect level the producer finds acceptable and one the customer will not tolerate, and the operating characteristic curve shows how the plan treats each.

For safety-critical products, companies often inspect every unit or test continuously, because the cost of a missed defect is too high to rely on sampling alone.

Statistics cannot set that threshold. Ethics, law and cost do.

In a real recall, many other factors matter: safety severity, whether defects are random or clustered in one production run, and the cost of being wrong in each direction. Statistics frames the decision but does not make it.