Systematic Reviews: Step 6: Assess Quality of Included Studies

In step 6 you will evaluate the articles you included in your review for quality and bias. To do so, you will:

1. Use quality assessment tools to grade each article.
2. Create a summary of the quality of literature included in your review.

This page has links to quality assessment tools you can use to evaluate different study types. Librarians can help you find widely used tools to evaluate the articles in your review.

Reporting your review with PRISMA

Covidence includes the Cochrane Risk of Bias 2.0 quality assessment template, but you can also create your own custom quality assessment template.

• What the quality assessment or risk of bias stage of the review entails
• How to choose an appropriate quality assessment tool
• Best practices for reporting quality assessment results in your review

• Cochrane Risk of Bias (ROB) 2.0 Tool
  Templates are tailored to randomized parallel-group trials, cluster-randomized parallel-group trails (including stepped-wedge designs), and randomized cross-over trails and other matched designs.
• CASP- Randomized Controlled Trial Appraisal Tool
  A checklist for RCTs created by the Critical Appraisal Skills Program (CASP)
• The Jadad Scale
  A scale that assesses the quality of published clinical trials based methods relevant to random assignment, double blinding, and the flow of patients
• CEBM-RCT
  A critical appraisal tool for RCTs from the Centre for Evidence Based Medicine (CEBM)
• Checklist for Randomized Controlled Trials (JBI)
  A critical appraisal checklist from the Joanna Briggs Institute (JBI)
• Scottish Intercollegiate Guidelines Network (SIGN)
  Checklists for quality assessment
• LEGEND Evidence Evaluation Tools
  A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.

• CASP- Cohort Studies
  A checklist created by the Critical Appraisal Skills Programme (CASP) to assess key criteria relevant to cohort studies
• Checklist for Cohort Studies (JBI)
  A checklist for cohort studies from the Joanna Briggs Institute
• The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses
  A validated tool for assessing case-control and cohort studies
• Scottish Intercollegiate Guidelines Network (SIGN)
  Checklists for quality assessment
• STROBE Checklist
  A checklist for quality assessment of case-control, cohort, and cross-sectional studies
• LEGEND Evidence Evaluation Tools
  A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.

• CASP- Case Control Study
  A checklist created by the Critical Appraisal Skills Programme (CASP) to assess key criteria relevant to case-control studies
• STROBE Checklist
  A checklist for quality assessment of case-control, cohort, and cross-sectional studies
• Tool to Assess Risk of Bias in Case Control Studies by the CLARITY Group at McMaster University
  A quality assessment tool for case-control studies from the CLARITY Group at McMaster University
• Scottish Intercollegiate Guidelines Network (SIGN)
  Checklists for quality assessment
• The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses
  A validated tool for assessing case-control and cohort studies
• LEGEND Evidence Evaluation Tools
  A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
• Checklist for Case-Control Studies
  A checklist created by the Joanna Briggs Institute

• STROBE Checklist
  A checklist for quality assessment of case-control, cohort, and cross-sectional studies
• LEGEND Evidence Evaluation Tools
  A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.

• CASP- Diagnostic Studies
  A checklist for diagnostic studies created by the Critical Appraisal Skills Program (CASP)
• QUADAS-2
  A quality assessment tool developed by a team at the Bristol Medical School: Population Health Sciences at the University of Bristol
• Critical Appraisal Checklist for Diagnostic Test Accuracy Studies (JBI)
  A checklist for quality assessment of diagnostic studies developed by the Joanna Briggs Institute
• Scottish Intercollegiate Guidelines Network (SIGN)
  Checklists for quality assessment

• Consensus Health Economic Criteria (CHEC) List
  19 yes-or-no questions, one for each category to assess economic evaluations
• CASP- Economic Evaluation
  A checklist for quality assessment of economic studies by the Critical Appraisal Skills Programme
• Scottish Intercollegiate Guidelines Network (SIGN)
  Checklists for quality assessment
• LEGEND Evidence Evaluation Tools
  A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.

• McGill Mixed Methods Appraisal Tool (MMAT) 2018 User Guide
  See full site for additional information, including FAQ's, references and resources, earlier versions, and more
• LEGEND Evidence Evaluation Tools
  A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.

• CASP- Qualitative Studies
  10 questions to help assess qualitative research from the Critical Appraisal Skills Programme
• LEGEND Evidence Evaluation Tools
  A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.

• Critical Appraisal Checklist for Systematic Reviews and Research Syntheses
  An 11-item checklist for evaluating systematic reviews
• AMSTAR Checklist
  A 16-question measurement tool to assess systematic reviews
• AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews
  A guide to selecting eligibility criteria, searching the literature, extracting data, assessing quality, and completing other steps in the creation of a systematic review
• CASP - Systematic Review
  A checklist for quality assessment of systematic review from the Critical Appraisal Skills Programme
• Scottish Intercollegiate Guidelines Network (SIGN)
  Checklists for quality assessment
• LEGEND Evidence Evaluation Tools
  A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.

• National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Instrument
  A 15-item instrument using a scale of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trust worth guidelines
• AGREE-II Appraisal of Guidelines for Research and Evaluation
  The Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument evaluates the process of practice guideline development and the quality of reporting

• NTACT Quality Checklists
  Quality indicator checklists for correlational studies, group experimental studies, single case research studies, and qualitative studies developed by the National Technical Assistance Center on Transition (NTACT). (Users must make an account.)

The Cochrane Risk of Bias 2.0 tool asks questions about five types of potential bias for individually randomized trials:

• bias arising from the randomization process
• bias due to differences between actual and intended interventions
• bias due to missing outcome data
• bias in measurement of the outcome
• bias in selection of the reported result

The Newcastle-Ottawa scale assesses the quality of nonrandomized studies based on three broad perspectives:

• the selection of the study groups
• the comparability of the groups
• the ascertainment of either the exposure or outcome of interest for case-control or cohort studies respectively

These quality assessment checklists ask 11 or 12 questions each to help you identify

• Is the basic study design valid?
• Was the study methodologically sound?
• What are the results?
• Will the results help locally?

Available study designs include randomized controlled trials, systematic reviews, qualitative studies, cohort studies, diagnostic studies, case control studies, economic evaluations, and clinical prediction rules.

These evidence evaluation tools ask questions each to help you examine

• validity
• reliability
• applicability

across the clinical question domains of intervention, diagnosis & assessment, prognosis, etiology & risk factors, incidence, prevalence, and meaning.

Available study designs include systematic review / meta analysis, meta-synthesis, randomized controlled trials, controlled clinical trials, psychometric studies, cohort-prospective / retrospective, case control, longitudinal, cross sectional, descriptive / epidemiology / case series, qualitative study, quality improvement, mixed methods, decision analysis / economic analysis / computer simulation, case report / n-of-1 study, published expert opinion, bench studies, and guidelines.

Use Covidence for quality assessment

Covidence uses Cochrane Risk of Bias (which is designed for rating RCTs and cannot be used for other study types) as the default tool for quality assessment of included studies. You can opt to manually customize the quality assessment template and use a different tool better suited to your review. More information about quality assessment using Covidence, including how to customize the quality assessment template, can be found below. If you decide to customize the quality assessment template, you cannot switch back to using the Cochrane Risk of Bias template.