ANOVA or Kruskal-Wallis? Choosing a Test for Three or More Groups

You have three or more groups, one measurement, and a simple question: do the groups differ? Two tests dominate the answer, one-way ANOVA and the Kruskal-Wallis test. Many guides say to use ANOVA if the data are normal and Kruskal-Wallis if not. The reality is more subtle, because the tests ask different questions.

Different questions

One-way ANOVA compares group means. Kruskal-Wallis, sometimes called one-way ANOVA on ranks, asks whether values in some groups tend to be larger than in others. Its null hypothesis is often described as equal medians, but that is true only if the groups share the same distribution shape (Handbook of Biological Statistics).

An example where they disagree

Take three groups of six values. Group A: 12, 14, 15, 15, 16, 18. Group B: 13, 15, 17, 18, 19, 21. Group C: 16, 18, 19, 21, 22, 40. The means are 15.0, 17.2 and 22.7, and the medians 15, 17.5 and 20.

The one-way ANOVA calculator returns F ≈ 3.17 and p ≈ 0.071. The Kruskal-Wallis calculator returns H ≈ 7.40 and p ≈ 0.025. The outlier of 40 inflates Group C's standard deviation to 8.8, which makes ANOVA cautious.

Now replace 40 with 24. ANOVA's p-value falls to about 0.016, but Kruskal-Wallis stays at 0.025, because only the ranks matter and the ranks did not change. That is the rank test's robustness, and also its limit.

ANOVA listens to how extreme the data are. Kruskal-Wallis listens only to their order.

A recommendation that surprises people

The Handbook of Biological Statistics cautions against treating Kruskal-Wallis as the default cure for non-normality. ANOVA is not very sensitive to deviations from normality, ranks lose information, and the handbook reserves Kruskal-Wallis for data that are genuinely ranked.

When group variances differ, Welch's ANOVA is the better choice for the same reasons Welch's t-test is, and one research team has argued for it as the default (Delacre et al., 2019). Our piece on Welch versus Student explains the logic.

After a significant result

A significant omnibus test says that some groups differ, not which ones. Follow-up comparisons need adjustment for multiple testing. With three groups there are three pairwise tests, and the chance of at least one false positive at the 5% level is about 14%. With six groups there are 15 pairs, and that chance rises to about 54%. The Mann-Whitney calculator and t-test calculator handle each comparison, but the adjustment is on you.

Whatever you choose, report an effect size too. Eta squared, the share of variance that lies between groups, tells readers how much the grouping matters, which a p-value alone cannot.

When the same people are in every group

Both tests assume independent groups. When the same participants are measured under three or more conditions, use repeated-measures ANOVA or the Friedman test calculator. For exactly two groups, see our article on t-test versus Mann-Whitney.

A quick decision guide

Use one-way ANOVA for roughly normal data with similar spreads. Use Welch's ANOVA when spreads differ. Use Kruskal-Wallis for ordinal data, small samples with strong outliers, or when the rank question is what you want to ask. Always plot first, for example with the descriptive statistics calculator, and report group sizes, medians and means, and an effect size, not just the p-value.