Normality, and what to do when it fails

What the assumption actually means, how it is checked, and why failing it is not a disaster.

Many tests assume the data — strictly, the sampling distribution or the residuals — follow a roughly bell-shaped curve. It is checked with the Shapiro–Wilk test, with a Q-Q plot, and by looking at skewness and kurtosis. A Shapiro–Wilk p-value below 0.05 suggests the data depart from normal.

When it fails, a non-parametric test applies instead: Mann–Whitney U in place of the independent t-test, Wilcoxon signed-rank in place of the paired t-test, Kruskal–Wallis in place of one-way ANOVA.

Why it matters

Using a parametric test on badly non-normal data can give a p-value that means nothing. But the reverse is also a mistake: reaching for a non-parametric test out of caution when the assumption holds throws away power for no reason.

An example

Forty-two VAS pain scores. Shapiro–Wilk gives W = 0.972, p = 0.38 — no evidence against normality, so a t-test is appropriate.

Length of hospital stay in the same patients. Shapiro–Wilk gives W = 0.821, p < .001 — strongly skewed by a few long stays, as length-of-stay data usually is. Mann–Whitney U applies, and the median with the interquartile range is the right summary rather than the mean.

Common mistakes

• Testing normality on the raw outcome when the assumption is about each group, or about the residuals.
• Reporting a mean and standard deviation for data you have just shown to be skewed. Use the median and IQR.
• Treating a significant Shapiro–Wilk in a very large sample as fatal. With n in the hundreds it detects departures too small to matter — look at the Q-Q plot as well.

Read next

• How the right statistical test is chosen — Four questions decide it, and none of them require you to know statistics first.
• Describing your data before you test anything — Mean and SD, or median and IQR — and how to tell which.