A Complete Guide To Systematic Review

There are different types of reviews, such as systematic and scoping reviews. 

Systematic reviews play a pivotal role when it comes to making evidence-based decisions in health, social sciences, and many other fields. By collating and summarising the findings of individual studies, systematic reviews help stakeholders make informed decisions. This guide discusses how to conduct a systematic review.

What Is A Systematic Review?

Key components of a systematic review include:

Clearly Defined Question

The review starts with a clearly formulated research question, often framed in terms of the Population, Intervention, Comparison, and Outcome (PICO) framework.

Protocol Development

Before starting the review, a protocol is developed that details the specific methods that will be used. This protocol ensures transparency and can be registered on platforms like PROSPERO.

Search Strategy

A comprehensive and systematic search of multiple databases (e.g., PubMed, Cochrane Library, Embase) is conducted to identify all relevant studies. Search terms and strategies are meticulously designed to capture as much relevant literature as possible.

Study Selection

Using predefined inclusion and exclusion criteria, studies are selected for inclusion in the review. This process is typically done by at least two reviewers independently to minimise bias.

Data Extraction

Relevant data are extracted from the included studies using standardised forms or software. This can include details on study design, population, interventions, outcomes, and results.

Quality Assessment

The individual studies’ quality or risk of bias is assessed using standardised tools. This helps in determining the overall strength of the evidence.

Data Synthesis

The data from individual studies are synthesised. If studies are sufficiently homogenous in terms of design and outcomes, a meta-analysis may be conducted to statistically pool results.

Interpretation and Conclusion

The findings are interpreted in the context of the overall evidence, and conclusions are drawn about the research question.

Reporting

The review is reported following guidelines such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) to ensure transparent and complete reporting.

Why Are Systematic Reviews Important?

Systematic reviews are a type of research meta synthesis that plays a crucial role in evidence-based practice and decision-making in various fields, especially in healthcare. Here are some reasons why systematic reviews are important:

Comprehensive Overview

Systematic reviews comprehensively summarise the available evidence on a particular topic or question. By aggregating results from multiple studies, they offer a more complete picture than individual studies can provide.

Reduced Bias

The systematic approach involves a structured and predefined methodology, which helps minimise biases that can occur in traditional narrative reviews. This includes the explicit selection criteria, comprehensive literature search, and standardised evidence appraisal.

Increased Reliability

Because of the rigorous methodology, the findings of systematic reviews are often considered more reliable than other types of reviews. They can help identify the consistency or variability in research findings.

Identification of Knowledge Gaps

Systematic reviews can highlight areas where evidence is lacking or inconsistent, guiding future research endeavours.

Informing Decision-making

Policymakers, clinicians, and other stakeholders can use systematic reviews to make informed decisions, as they present a synthesised view of the best available evidence.

Time and Resource Efficiency

Given that systematic reviews summarise vast amounts of research, they can save time for practitioners and researchers trying to understand the state of knowledge on a particular topic.

Improved Generalisability

By pooling data from various studies, systematic reviews can provide insights that are more generalisable across different populations and settings than individual studies.

Statistical Power

When systematic reviews include a meta-analysis, they combine quantitative results from multiple studies, which can increase the statistical power to detect effects that might be missed in individual studies.

Consensus and Guideline Development

They play a pivotal role in the development of clinical guidelines and consensus statements, ensuring that recommendations are grounded in a thorough review of the evidence.

Public Policy Impact

Beyond healthcare, systematic reviews can influence public policy decisions by providing consolidated evidence on the effectiveness and implications of interventions and policies.

Educational Tool

For students and trainees in various fields, systematic reviews can serve as educational resources, giving a distilled overview of what is known and not known about a topic.

The Steps To Conducting A Systematic Review

Systematic reviews are a rigorous and transparent form of literature review designed to answer a specific research question. They provide a complete, exhaustive summary of current literature relevant to a research question. Here is a comprehensive guide to conducting a systematic review:

Step 1. Defining the Research Question

Your research question should be clear and specific. Consider the PICO format:

• P: Patient/Population – Who is the patient or target population?
• I: Intervention – What is the primary intervention or exposure?
• C: Comparison – What is the comparator or alternative to the intervention?
• O: Outcome – What are the expected outcomes?

Step 2. Developing a Protocol

Before you start your search, develop a protocol that outlines your planned methods. This should include:

• The research question
• Eligibility criteria for studies
• Databases and other evaluating sources
• Search terms and strategy
• The process for selecting studies
• Data extraction methods
• Plans for assessing the quality of studies

Step 3. Literature Search

Begin a comprehensive search using electronic databases like PubMed, Scopus, Web of Science, and specialised databases relevant to your field. Employ controlled vocabulary and free-text terms. Document all search strategies to ensure transparency.

Step 4. Study Selection

After retrieving studies, eliminate duplicates. Screen titles and abstracts for relevance, followed by full-text reviews. Document reasons for exclusion to ensure transparency and reproducibility.

Step 5. Data Extraction

For each included study, extract relevant data using standardised forms. This may include:

• Study design
• Population characteristics
• Interventions or exposures
• Outcome measures
• Results

Step 6. Assessment of Study Quality

Evaluate the quality or risk of bias in each study. Tools like the Cochrane Risk of Bias tool or the Newcastle-Ottawa Scale can be employed depending on the study design.

Step 7. Data Synthesis

Combine and summarise the findings. If the studies are sufficiently homogeneous, consider a meta-analysis. This statistical method combines results from multiple studies to derive an overall effect size. If a meta-analysis isn’t feasible, provide a narrative synthesis.

Step 8. Interpretation and Report Writing

Discuss the findings in context, consider the strengths and limitations of the included studies, and draw conclusions. Remember to:

• Highlight the implications of the findings.
• Suggest areas for future research.
• Follow reporting guidelines like PRISMA for systematic reviews.

Step 9. Peer Review and Publication

Submit the systematic review to a relevant journal for peer review. Feedback from experts can greatly enhance the quality and validity of your findings.

Step 10. Stay updated

• Systematic reviews can become outdated. Set up alerts to notify you of new studies that meet your inclusion criteria.
• Consider updating the review after a certain number of new relevant studies are published.

Challenges In Conducting Systematic Reviews

Systematic reviews are essential for summarising the best available evidence on a particular topic, guiding clinical decisions, and identifying gaps in current research. However, conducting systematic reviews is a complex process that can encounter multiple challenges. Some of these challenges include:

Literature Search Complexity

It is crucial to develop a comprehensive and unbiased search strategy. Missing relevant studies or including irrelevant ones can skew results.

Publication Bias

Positive or significant findings are more likely to be published than negative or non-significant findings. This bias can lead to an overestimation of an intervention’s effect.

Data Extraction Difficulties

Extracting data from studies can be laborious and sometimes challenging if the data are presented in a non-standardised way or if necessary details are missing.

Heterogeneity

Differences between studies in terms of their populations, interventions, outcomes, and methodologies can make combining data problematic.

Assessment of Study Quality

Deciding how to evaluate the quality and risk of bias in the included studies can be challenging.

Synthesis Issues

Deciding whether to conduct a meta-analysis or a qualitative synthesis can be difficult, depending on the heterogeneity of the studies.

Reporting Biases within Individual Studies

Authors might not always report all outcomes, or selectively report only the positive findings.

Updates

The literature on a given topic continuously evolves, making it challenging to keep systematic reviews up-to-date.

Time and Resource Intensive

Systematic reviews require a significant amount of time, managing sources, and expertise, making them expensive and sometimes prohibitive for some researchers.

Access to Full Texts

Sometimes, full-text articles may be behind paywalls or not available online at all, making it challenging to assess and extract data.

Language and Regional Limitations

Important studies may be published in languages other than English or in regional databases that are not commonly searched, which could lead to inadvertent exclusions.

Standardisation of Outcomes

Different studies may measure outcomes differently or use different scales, complicating synthesis efforts.

Grey Literature

This includes non-peer-reviewed articles, reports, theses, etc. Searching, appraising, and incorporating grey literature can be challenging but essential to reduce publication bias.

Stakeholder Engagement

Involving all relevant stakeholders can be time-consuming, but is crucial for ensuring the review is comprehensive and relevant.

Interpreting and Applying Results

Results from systematic reviews may not always be directly applicable to all populations or settings due to variability in the included studies.

Ethical Considerations

Especially when primary data are used or when patient-sensitive data are involved.

Tips For A Successful Systematic Review

Here are some tips to ensure the success of a systematic review:

Preparation and Planning

• Formulate a Clear Research Question: Use the PICO format (Population, Intervention, Comparison, Outcome) or another appropriate framework to clearly define your question.
• Assemble a Multidisciplinary Team: Include experts from relevant fields, a librarian or information specialist, and statisticians if meta-analysis is planned.

Protocol Development

• Register Your Review: Platforms like PROSPERO allow researchers to register their systematic reviews, which ensures transparency and reduces the likelihood of duplication.
• Define Eligibility Criteria: Clearly state which studies will be included or excluded based on factors like study design, population, intervention, and outcomes.

Comprehensive Search Strategy

• Consult a Librarian or Information Specialist: They can help in devising a comprehensive search strategy across multiple databases.
• Search Multiple Databases: Don’t limit yourself to just one. Common databases include PubMed, EMBASE, Cochrane Library, and Web of Science.
• Include Grey Literature: Look for conference abstracts, theses, and other unpublished materials.
• Handsearch Relevant Journals: This can help catch articles missed in database searches.
• Reference List Scanning: Review the references of included articles for additional relevant studies.

Study Selection and Data Extraction

• Use Screeners: Employ at least two independent screeners to review titles, abstracts, and full-text articles to minimise errors and biases.
• Piloted Data Extraction Forms: Ensure that data extraction forms are clear and capture all necessary information.
• Document Exclusions: Clearly note why specific studies were excluded during the full-text screening phase.

Assess Quality and Risk of Bias

Using established tools like the Cochrane Collaboration’s risk of bias tool or the Newcastle-Ottawa Scale can help in evaluating the quality of included studies.

Data Synthesis and Analysis

• Consider Heterogeneity: Assess clinical, methodological, and statistical heterogeneity before combining study results.
• Choose Appropriate Meta-analysis Models: For instance, fixed-effect or random-effects models based on heterogeneity.

Transparent Reporting

• Adhere to PRISMA Guidelines: The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist ensures a systematic review is reported transparently.
• Clearly Report Limitations: Every systematic review will have limitations. Acknowledge them transparently.

Stakeholder Engagement

Engage clinicians, patients, or policymakers to ensure the review remains relevant to its intended audience.

Stay Updated

Developing a research strategy is continuously evolving. Depending on the topic, you may need to update the review periodically.

Peer Review and Feedback

Before finalising, get input from experts or peers who haven’t been involved in the review process.

Dissemination

Tailor the message as different audiences (clinicians, patients, policymakers) require different presentations of the findings.

Frequently Asked Questions