A Complete Guide To Systematic Review
There are different types of reviews, such as systematic and scoping reviews.
Systematic reviews play a pivotal role when it comes to making evidence-based decisions in health, social sciences, and many other fields. By collating and summarising the findings of individual studies, systematic reviews help stakeholders make informed decisions. This guide discusses how to conduct a systematic review.
What Is A Systematic Review?
Key components of a systematic review include:
Clearly Defined Question
The review starts with a clearly formulated research question, often framed in terms of the Population, Intervention, Comparison, and Outcome (PICO) framework.
Before starting the review, a protocol is developed that details the specific methods that will be used. This protocol ensures transparency and can be registered on platforms like PROSPERO.
A comprehensive and systematic search of multiple databases (e.g., PubMed, Cochrane Library, Embase) is conducted to identify all relevant studies. Search terms and strategies are meticulously designed to capture as much relevant literature as possible.
Using predefined inclusion and exclusion criteria, studies are selected for inclusion in the review. This process is typically done by at least two reviewers independently to minimise bias.
Relevant data are extracted from the included studies using standardised forms or software. This can include details on study design, population, interventions, outcomes, and results.
The individual studies’ quality or risk of bias is assessed using standardised tools. This helps in determining the overall strength of the evidence.
The data from individual studies are synthesised. If studies are sufficiently homogenous in terms of design and outcomes, a meta-analysis may be conducted to statistically pool results.
Interpretation and Conclusion
The findings are interpreted in the context of the overall evidence, and conclusions are drawn about the research question.
The review is reported following guidelines such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) to ensure transparent and complete reporting.
Why Are Systematic Reviews Important?
Systematic reviews are a type of research meta synthesis that plays a crucial role in evidence-based practice and decision-making in various fields, especially in healthcare. Here are some reasons why systematic reviews are important:
Systematic reviews comprehensively summarise the available evidence on a particular topic or question. By aggregating results from multiple studies, they offer a more complete picture than individual studies can provide.
The systematic approach involves a structured and predefined methodology, which helps minimise biases that can occur in traditional narrative reviews. This includes the explicit selection criteria, comprehensive literature search, and standardised evidence appraisal.
Because of the rigorous methodology, the findings of systematic reviews are often considered more reliable than other types of reviews. They can help identify the consistency or variability in research findings.
Identification of Knowledge Gaps
Systematic reviews can highlight areas where evidence is lacking or inconsistent, guiding future research endeavours.
Policymakers, clinicians, and other stakeholders can use systematic reviews to make informed decisions, as they present a synthesised view of the best available evidence.
Time and Resource Efficiency
Given that systematic reviews summarise vast amounts of research, they can save time for practitioners and researchers trying to understand the state of knowledge on a particular topic.
By pooling data from various studies, systematic reviews can provide insights that are more generalisable across different populations and settings than individual studies.
When systematic reviews include a meta-analysis, they combine quantitative results from multiple studies, which can increase the statistical power to detect effects that might be missed in individual studies.
Consensus and Guideline Development
They play a pivotal role in the development of clinical guidelines and consensus statements, ensuring that recommendations are grounded in a thorough review of the evidence.
Public Policy Impact
Beyond healthcare, systematic reviews can influence public policy decisions by providing consolidated evidence on the effectiveness and implications of interventions and policies.
For students and trainees in various fields, systematic reviews can serve as educational resources, giving a distilled overview of what is known and not known about a topic.
The Steps To Conducting A Systematic Review
Systematic reviews are a rigorous and transparent form of literature review designed to answer a specific research question. They provide a complete, exhaustive summary of current literature relevant to a research question. Here is a comprehensive guide to conducting a systematic review:
Step 1. Defining the Research Question
Your research question should be clear and specific. Consider the PICO format:
- P: Patient/Population – Who is the patient or target population?
- I: Intervention – What is the primary intervention or exposure?
- C: Comparison – What is the comparator or alternative to the intervention?
- O: Outcome – What are the expected outcomes?
Step 2. Developing a Protocol
Before you start your search, develop a protocol that outlines your planned methods. This should include:
- The research question
- Eligibility criteria for studies
- Databases and other evaluating sources
- Search terms and strategy
- The process for selecting studies
- Data extraction methods
- Plans for assessing the quality of studies
Step 3. Literature Search
Begin a comprehensive search using electronic databases like PubMed, Scopus, Web of Science, and specialised databases relevant to your field. Employ controlled vocabulary and free-text terms. Document all search strategies to ensure transparency.
Step 4. Study Selection
After retrieving studies, eliminate duplicates. Screen titles and abstracts for relevance, followed by full-text reviews. Document reasons for exclusion to ensure transparency and reproducibility.
Step 5. Data Extraction
For each included study, extract relevant data using standardised forms. This may include:
- Study design
- Population characteristics
- Interventions or exposures
- Outcome measures
Step 6. Assessment of Study Quality
Evaluate the quality or risk of bias in each study. Tools like the Cochrane Risk of Bias tool or the Newcastle-Ottawa Scale can be employed depending on the study design.
Step 7. Data Synthesis
Combine and summarise the findings. If the studies are sufficiently homogeneous, consider a meta-analysis. This statistical method combines results from multiple studies to derive an overall effect size. If a meta-analysis isn’t feasible, provide a narrative synthesis.
Step 8. Interpretation and Report Writing
Discuss the findings in context, consider the strengths and limitations of the included studies, and draw conclusions. Remember to:
- Highlight the implications of the findings.
- Suggest areas for future research.
- Follow reporting guidelines like PRISMA for systematic reviews.
Step 9. Peer Review and Publication
Submit the systematic review to a relevant journal for peer review. Feedback from experts can greatly enhance the quality and validity of your findings.
Step 10. Stay updated
- Systematic reviews can become outdated. Set up alerts to notify you of new studies that meet your inclusion criteria.
- Consider updating the review after a certain number of new relevant studies are published.
Challenges In Conducting Systematic Reviews
Systematic reviews are essential for summarising the best available evidence on a particular topic, guiding clinical decisions, and identifying gaps in current research. However, conducting systematic reviews is a complex process that can encounter multiple challenges. Some of these challenges include:
Literature Search Complexity
It is crucial to develop a comprehensive and unbiased search strategy. Missing relevant studies or including irrelevant ones can skew results.
Positive or significant findings are more likely to be published than negative or non-significant findings. This bias can lead to an overestimation of an intervention’s effect.
Data Extraction Difficulties
Extracting data from studies can be laborious and sometimes challenging if the data are presented in a non-standardised way or if necessary details are missing.
Differences between studies in terms of their populations, interventions, outcomes, and methodologies can make combining data problematic.
Assessment of Study Quality
Deciding how to evaluate the quality and risk of bias in the included studies can be challenging.
Deciding whether to conduct a meta-analysis or a qualitative synthesis can be difficult, depending on the heterogeneity of the studies.
Reporting Biases within Individual Studies
Authors might not always report all outcomes, or selectively report only the positive findings.
The literature on a given topic continuously evolves, making it challenging to keep systematic reviews up-to-date.
Time and Resource Intensive
Systematic reviews require a significant amount of time, managing sources, and expertise, making them expensive and sometimes prohibitive for some researchers.
Access to Full Texts
Sometimes, full-text articles may be behind paywalls or not available online at all, making it challenging to assess and extract data.
Language and Regional Limitations
Important studies may be published in languages other than English or in regional databases that are not commonly searched, which could lead to inadvertent exclusions.
Standardisation of Outcomes
Different studies may measure outcomes differently or use different scales, complicating synthesis efforts.
This includes non-peer-reviewed articles, reports, theses, etc. Searching, appraising, and incorporating grey literature can be challenging but essential to reduce publication bias.
Involving all relevant stakeholders can be time-consuming, but is crucial for ensuring the review is comprehensive and relevant.
Interpreting and Applying Results
Results from systematic reviews may not always be directly applicable to all populations or settings due to variability in the included studies.
Especially when primary data are used or when patient-sensitive data are involved.
Tips For A Successful Systematic Review
Here are some tips to ensure the success of a systematic review:
Preparation and Planning
- Formulate a Clear Research Question: Use the PICO format (Population, Intervention, Comparison, Outcome) or another appropriate framework to clearly define your question.
- Assemble a Multidisciplinary Team: Include experts from relevant fields, a librarian or information specialist, and statisticians if meta-analysis is planned.
- Register Your Review: Platforms like PROSPERO allow researchers to register their systematic reviews, which ensures transparency and reduces the likelihood of duplication.
- Define Eligibility Criteria: Clearly state which studies will be included or excluded based on factors like study design, population, intervention, and outcomes.
Comprehensive Search Strategy
- Consult a Librarian or Information Specialist: They can help in devising a comprehensive search strategy across multiple databases.
- Search Multiple Databases: Don’t limit yourself to just one. Common databases include PubMed, EMBASE, Cochrane Library, and Web of Science.
- Include Grey Literature: Look for conference abstracts, theses, and other unpublished materials.
- Handsearch Relevant Journals: This can help catch articles missed in database searches.
- Reference List Scanning: Review the references of included articles for additional relevant studies.
Study Selection and Data Extraction
- Use Screeners: Employ at least two independent screeners to review titles, abstracts, and full-text articles to minimise errors and biases.
- Piloted Data Extraction Forms: Ensure that data extraction forms are clear and capture all necessary information.
- Document Exclusions: Clearly note why specific studies were excluded during the full-text screening phase.
Assess Quality and Risk of Bias
Using established tools like the Cochrane Collaboration’s risk of bias tool or the Newcastle-Ottawa Scale can help in evaluating the quality of included studies.
Data Synthesis and Analysis
- Consider Heterogeneity: Assess clinical, methodological, and statistical heterogeneity before combining study results.
- Choose Appropriate Meta-analysis Models: For instance, fixed-effect or random-effects models based on heterogeneity.
- Adhere to PRISMA Guidelines: The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist ensures a systematic review is reported transparently.
- Clearly Report Limitations: Every systematic review will have limitations. Acknowledge them transparently.
Engage clinicians, patients, or policymakers to ensure the review remains relevant to its intended audience.
Developing a research strategy is continuously evolving. Depending on the topic, you may need to update the review periodically.
Peer Review and Feedback
Before finalising, get input from experts or peers who haven’t been involved in the review process.
Tailor the message as different audiences (clinicians, patients, policymakers) require different presentations of the findings.
Frequently Asked Questions
A systematic review is a rigorous synthesis of research studies on a specific topic. Researchers follow a structured process to identify, appraise, and summarise all available evidence on a question. This helps to provide a comprehensive understanding and ensures conclusions are based on the best available evidence, minimising biases.
- Define a clear research question.
- Develop a protocol with inclusion/exclusion criteria.
- Search databases systematically for relevant studies.
- Screen studies using predefined criteria.
- Extract data from included studies.
- Assess study quality.
- Synthesise findings, often statistically (meta-analysis).
- Report results comprehensively.
- Introduction: Present the rationale and research question.
- Methods: Detail search strategy, selection criteria, data extraction, and quality assessment.
- Results: Summarise included studies, their characteristics, and findings.
- Discussion: Interpret synthesised results to address limitations.
- Conclusion: Highlight the main findings and implications.
- References: Include references.
Example: A systematic review on the efficacy of face masks in preventing respiratory virus transmission. Researchers search databases for studies comparing mask usage to non-usage. After screening, relevant studies are analysed. Findings might suggest that wearing masks significantly reduces the risk of viral spread, with implications for public health recommendations.