Use of statistics

In health research, researchers typically use statistics to determine two things:

• Could the results have happened by chance? (statistical significance)
• How large was the effect found in the study? (effect size)

Statistical significance

Random chance can affect any study, and any measurement a researcher takes will be affected by some degree of chance. It is very important that researchers are not led by random chance into making false conclusions.

Imagine researchers are comparing two weight loss treatments, A and B.  Their measurements suggest that more participants lose weight with Treatment A than with Treatment B. Assuming there is no bias in their study, there are two possible explanations for this:

• The difference measured was due to chance, and Treatment A may not be any more effective than Treatment B
• The difference measured was not due to chance, and Treatment A really is more effective than Treatment B

Researchers use statistical tests to decide how likely it is that results are due to chance. If a result is probably not due to chance, the result is described as statistically significant. If the result is statistically significant, researchers may conclude that Treatment A really is more effective than Treatment B in similar situations to their trial.

Effect size

As well as determining whether results were due to chance or not, researchers use statistics to measure the size of the effect seen in their findings.

As well as knowing that Treatment A is more effective than Treatment B, it is important to measure exactly how much better it is. This information is important for making decisions. For example, it could be that Treatment A has more side effects than Treatment B, so doctors and patients might require Treatment A to be much more effective than Treatment B for them to consider using it.

There are many different ways to measure, describe and interpret statistical significance and effect size, and even experienced researchers can find it challenging to understand statistics in health research.

Some pointers to consider:

• Is the effect size expressed as a relative or absolute measure?
• How much uncertainty is there around the effect size?
• How important is the effect to a patient?

Good use of statistics is very important, and bad use of statistics can produce misleading results. If you have experience in statistics, you can consider whether the use of statistics in the paper is appropriate or not. If you do not have experience with statistics, and are interested in learning more, these resources can introduce you to the fundamentals:

Ask for Evidence - Do the statistics back up the claim?

Sense about Science - Making Sense of Statistics

Testing Treatments - Taking account of the play of chance

Trisha Greenhalgh - How to read a paper: Statistics for the non-statistician. I: Different types of data need different statistical tests (British Medical Journal)