t-Test or Mann-Whitney? How to Choose for Two Groups

You have two groups and one number per person: recovery days, test scores, waiting times. Which test do you run? Many guides give a one-line rule: t-test if the data are normal, Mann-Whitney if not. The truth is more interesting, because the two tests do not answer exactly the same question.

What each test actually asks

The t-test, in its Welch version, compares the two group means. The Mann-Whitney test, also called the Wilcoxon rank-sum test, ranks all observations together and asks whether values from one group tend to be larger than values from the other. It is often described as a test of medians, but that is only true when the two distributions have the same shape (Hart, 2001).

A worked example with an outlier

Imagine recovery times in days for two treatments. Group A: 3, 4, 4, 5, 5, 6, 6, 7, 8, 9, 12 and 30. Group B: 5, 6, 7, 7, 8, 9, 10, 11, 12, 13, 15 and 20. Group A has mean 8.25 and median 6, while Group B has mean 10.25 and median 9.5.

The independent samples t-test calculator gives p ≈ 0.42, mostly because the single 30-day value inflates A's standard deviation to 7.3. The Mann-Whitney calculator works on ranks, so that outlier counts only as "the largest value," and it returns p ≈ 0.049.

Same data, different verdicts. Neither test is cheating. They answer different questions, and the rank test is simply less sensitive to one extreme case.

When the t-test is the better choice

A paper in BMC Medical Research Methodology described a paradox: large studies increasingly use non-parametric tests, yet with big samples the t-test and its confidence interval work well even for skewed data, while the Wilcoxon-Mann-Whitney test can reject when means and medians are equal but spreads differ (Fagerland, 2012).

Rank tests shine in small studies. With hundreds of observations per group and a question about averages, such as revenue per customer, the Welch t-test with an interval for the difference in means is usually the more informative answer.

A quick decision guide

Lean toward Mann-Whitney when samples are small, the data are strongly skewed or contain outliers, or the scores are ordinal ratings. Lean toward the Welch t-test when samples are large and the mean is what matters. Plot the data first with the descriptive statistics calculator, and choose the test before you see the p-value, because running both and keeping the better result inflates false positives.

Reporting a rank test properly

A bare p-value from Mann-Whitney is hard to interpret. A friendlier summary is the probability of superiority: the chance that a randomly chosen value from one group exceeds one from the other. In our example, a random patient from Group B recovers more slowly than one from Group A about 74% of the time. That tells readers how large the difference is, which a p-value of 0.049 cannot.

Ordinal data, such as 1-to-5 satisfaction ratings, are a natural home for rank tests, since the gaps between points are not guaranteed to be equal. With three or more groups, the same reasoning leads to the Kruskal-Wallis test.

Remember that the same logic changes for paired data. For before-and-after measurements, the choices become the paired t-test or the Wilcoxon signed-rank test, as explained in our article on before-and-after data.