What a p-value is, and what it is not
The single most misreported number in postgraduate research.
A p-value is the probability of seeing a difference at least as large as the one you observed, if there were truly no difference at all. A small p-value means your data would be surprising in a world where nothing is going on.
It is not the probability that your hypothesis is true. It is not the probability that the result was chance. It does not tell you how large the effect is, or whether it matters.
Why it matters
Three claims cost marks in almost every viva. 'p > 0.05 proves there is no difference' — no; absence of evidence is not evidence of absence, especially in an underpowered study. 'p < 0.05 proves our hypothesis' — no; a significance test provides evidence against a null hypothesis, never proof. 'p = 0.000' — no; a p-value is never exactly zero. Report p < .001.
An example
Correct: 'Mean pain reduction was greater in the supervised group (4.2 ± 1.6) than the home group (2.9 ± 1.8); this difference was statistically significant (t(40) = 2.47, p = .018, Cohen's d = 0.76).'
Incorrect: 'The supervised group was significantly better (p = 0.000), which proves that supervision works.'
Common mistakes
- 'Approaching significance' or 'a trend towards significance' for p = 0.07. It is either below your threshold or it is not.
- Reporting p without the test statistic, the degrees of freedom and the effect size.
- Accepting the null hypothesis rather than failing to reject it.
Read next
- Effect size — how big, not just whether — Significance tells you something is there. Effect size tells you whether it matters.
- Confidence intervals, and why they say more than a p-value — A range of values your data are consistent with.
Chavery Research Companion applies this to your own study: it asks the questions in plain language, checks the assumptions against your data, recommends the test, and writes the sentence that reports it. Start free — planning and the master chart cost nothing.