Not all evidence is created equal. Therefore, systematic reviews always consider the quality of evidence in each research study before summarizing the information in a review. Some terms that you may see in a systematic review are the 'strength of the evidence' and the 'size of the effect'.
How certain can you be that the stated evidence is a true measure of the benefits and harms of treatment?
Strength of the evidence
The quality of the evidence is determined by the methods used to minimise bias within a study design. Cochrane reviews are systematic assessments of all the relevant randomised controlled trials (RCTs), which give the highest level of evidence. Statistical precision is the degree of certainty about the existence of a true measured effect.
The second level of evidence is provided by at least one randomised controlled trial on the effectiveness of an intervention, then a pseudo-randomised trial e.g. a trial that assigns participants to a treatment group by alternating between groups as they present or by date of admission.
Relevance of the evidence
How appropriate is the outcome measure for the healthcare problem, and how useful is it in measuring the benefits (or harms) of the treatment?
Size of effect
How high is the likelihood that the effect of the treatment will achieve clinically relevant benefits (or harms)?
Lower levels of evidence are provided by:
non-randomised studies of groups of people where a control group has run concurrently with the group receiving the intervention being assessed;
non-randomised studies of groups of people where intervention effects are compared with previous or historical information;
single case studies.
Confidence interval (CI):
Even studies that are perfectly designed and carried out may show variable results because of the play of chance. CI covers the likely range of the true effect. For example, the result of a study may be that 40% (95% CI 30% to 50%) of people are helped by a treatment. That means that we can be 95% certain the true effect is between 30% and 50%.
(From: Smart Health Choices How to make informed health decisions by Judy Irwig, Les Irwig and Melissa Sweet, Allen and Unwin 1999)
See our training resources to learn more.