A systematic review is a summary of clinical literature which, through explicit reproducible methods, performs a comprehensive and systematic literature search and critical appraisal of individual studies. Systematic reviews may incorporate statistical techniques with which they combine valid and similarly designed studies together to produce an average estimate of effect based on all reviewed studies – this is what we refer to as a meta-analysis.
Hierarchy of Evidence
How To Perform Systematic Reviews & Meta Analysis
- Step 1 – develop a question
- Step 2 – define the criteria for inclusion of studies
- Step 3 – perform a systematic search through available literature to find eligible studies (fitting all PICO elements)
- Step 4 – review methods and results of the selected studies
- Step 5 – assess heterogeneity (variation) between studies
- Step 6 – apply statistical methods to produce a summary result
Critically Appraising a Review
- check whether the author/s specified their clinical query
- check whether each key element of the question (PICO) is clear and specific
- scrutinise the search strategy that was used – were boolean operators used? Was the author/s’ search extensive and comprehensive enough? If not, the study should be revised and retested. The search should include a good keyword combination, good databases, grey literature (so as to prevent publication bias), limiters, and more than one person in the search (prevents bias – sometimes it’s difficult to assess whether or not a study should be used or not, hence 2 or more should be included in this process (if consensus is not met)
- check whether the author/s applied any quality criteria in study selection
Publication Bias
In order to avoid publication bias, authors of reviews should look into books, grey literature, and unpublished material eg. conference proceedings, dissertations etc. This is because one should keep in mind that positive and statistically significant results are more likely to be published and be included in scientific journals easily found in electronic databases.
Critically Appraising a Meta Analysis
Heterogeneity testing
- test for clinical and statistical heterogeneity
- use various tests such as chi square test, and give p value. If p value is large, difference between results is not significant
statistical calculations
- use fixed or random effect models
- random effect models are used to make up for heterogeneity, usually needing a large sample but providing conservative results
- NEED to qualitatively examine the potential cause of heterogeneity prior to deciding to pool results
sensitivity analysis
- check on the impact of low quality papers on the overall result
- 1st analysis requires the exclusion of dubious studies
- 2nd analysis requires the re-inclusion of dubious studies to see if they affect the overall result
PRISMA Diagram
When researchers choose to perform a systematic review, they are required to present the whole search process within a PRISMA diagram. Any grey literature or manually retrieved literature should also be included in the PRISMA diagram (sample PRISMA flow diagram can be found below…)
PRISMA, a.k.a. Preferred Reporting Items for Systematic Reviews and Meta-Analyses, is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses (sample PRISMA checklist can be found below…)
CASP Tool for Systematic Reviews
CASP tool for systematic reviews can be found here.
To view blogpost featuring Cochrane videos on all types of studies please click here.
Did you find the above nursing information useful? Follow us on Facebook and fill in your email address below to receive new blogposts in your inbox as soon as they’re published 🙂
