One-Tailed or Two-Tailed Test? When Is It Allowed?

A result lands at p = 0.084. Annoyingly close. Then someone notices that a one-tailed test would halve it to 0.042, and suddenly the finding is "significant." Few choices in statistics are as tempting, and as easy to abuse, as picking the tail.

What the tail means

A two-tailed test asks whether the true value differs from the null value in either direction. A one-tailed test asks only about one direction, such as "greater than." Because it puts the whole rejection region on one side, the one-tailed p-value for a result in the predicted direction is half the two-tailed value.

The same data, two answers

Suppose a sample of 40 has mean 104.2 and standard deviation 15, and the null value is 100. The t-statistic is about 1.77. The one-sample t-test calculator gives a two-tailed p ≈ 0.084 and a one-tailed p ≈ 0.042. The z-test calculator for known variance behaves the same way when the standard deviation is known.

Now flip the story. Had the sample mean been 95.8 instead, the one-tailed test for "greater than 100" would give p ≈ 0.96, and nothing could be concluded, however large the drop. That is the price of looking in only one direction.

The rule that justifies a one-tailed test

A paper in Methods in Ecology and Evolution suggests two questions. Can you explain why you care about an effect in one direction only, and would you treat a large difference in the unexpected direction no differently from a non-significant one? (Ruxton and Neuhäuser, 2010). If the surprising direction would change your action, the test should be two-tailed.

Reviews of published papers have been harsh. Andy Field's discussion of the topic cites one survey in which only 1 of 17 papers using one-tailed tests was justified, and another concluding that all one-tailed uses in the journals surveyed seemed invalid (Field, SAGE Research Methods).

Decide before you see the data

A directional hypothesis has to be stated in advance, ideally in a written plan. Choosing the tail after seeing which way the data point is equivalent to doubling the false-positive rate, since you would have used the other tail had the sign been reversed.

What about confidence intervals?

A one-sided test pairs with a one-sided bound. In our example, the usual two-sided 95% interval runs from 99.4 to 109.0, which includes 100, while a one-sided 95% lower bound is about 100.2. The two descriptions are equally valid, but they answer different questions, and readers expect the two-sided interval by default.

A worked decision

Imagine a manufacturer testing whether a new process raises the defect rate. Only an increase would trigger action, and a decrease would be welcome but would change nothing. With the direction declared in advance, a one-tailed test can be defended. Compare a medical trial of a new treatment against a placebo. Harm is as important as benefit, so the unexpected direction matters, and a two-tailed test is the norm.

When in doubt, use two tails. The benefit is that nobody can accuse you of choosing the tail to rescue a result.

In practice, report the two-sided result unless there is a compelling reason, such as a safety rule in which only an increase would trigger action. Our A/B testing article shows how directional claims interact with early peeking, and the one-sample proportion test allows the choice of alternative as well.