What consumers can and cannot get from systematic reviews

Systematic reviews ask a very specific research question about a particular intervention in a clearly defined group of people who have a health condition or problem. Reviews provide powerful information on the state of knowledge about a healthcare intervention and whether that intervention is an effective treatment of a healthcare condition.

Reviews are dependent on the availability of studies and the information these studies contain.

Healthcare studies differ dramatically in what they look for and how well they are carried out and, therefore, how much weight one can put on their conclusions. Part of the reason for performing systematic reviews is to reduce the effects of these shortcomings. Issues of conflict of interest and corporate funding of healthcare studies are also important considerations in drawing conclusions from any study.

Reviews more often assess benefits rather than harms.

Healthcare studies are generally designed to assess the benefits, rather than the harms, of an intervention. Studies generally have a relatively short designated time period. Any possible harms of an intervention may be expected to occur less frequently and over a longer period of time than the studies cover.

Randomised controlled trials are expensive to run. They are very time consuming and multiple factors may limit how many participants are involved, the outcomes measured and the length of the trial. Therefore, many studies are conducted to identify benefits that may be financially rewarding.

Participants of studies are carefully selected to reduce the risk of other problems interfering with the effectiveness of an intervention. Therefore, the selective nature of this process needs to be carefully considered when assessing a systematic review.

Systematic reviews:

  • follow stringent guidelines as to what types of studies are included and how healthcare measures of effectiveness can be expressed and combined;

  • cannot offer a guideline for treatment, especially if a person differs from those defined in the review. Individuals may have accompanying health problems, be in a different healthcare setting, or receive more than one intervention, for example;

  • may consider outcomes other than the one you are interested in and may not look at long term effects of an intervention;

  • may only find studies that are limited in a single healthcare setting, in which they take place;

  • may provide limited conclusions because of the specific question asked and/or the studies that were found.