Is there a clear review question or aim?
The question or questions the review is addressing should be clearly stated.
Was the literature search strategy stated?
The review should list the search terms, databases searched, years searched, and other identification strategies used.
Were there explicit inclusion/exclusion criteria reported relating to selection of the primary studies?
The inclusion/exclusion criteria should cover study design, populations, interventions, and outcomes of interest.
Details should be reported relating to the process of decision-making (i.e., how many reviewers were involved, whether the studies were examined independently, and how disagreements between reviewers were resolved). Some measure of interrater reliability (e.g., kappa, percent agreement) is important if only a sample of abstracts or studies were dual-reviewed. Additionally, is there any evidence that the process was biased or inadequately implemented?
Is there evidence of a substantial effort to search for all relevant research?
In addition to details of the search terms and databases, descriptions of hand-searching, attempts to identify unpublished material, and any contact with authors, industry, and research institutes should be provided. The appropriateness of the database(s) searched by the authors should also be considered (e.g., if only MEDLINE is searched for a review looking at health education, then it is unlikely that all relevant studies will have been located).
Is there a transparent system to evaluate the quality of individual studies?
A systematic assessment of the quality of primary studies should include an explanation of the criteria used (e.g., method of randomization, whether outcome assessment was blinded, whether analysis was on an intention-to-treat basis). The process of the assessment should be explained (i.e., the number of reviewers who assessed each study, whether the assessment was independent, and how discrepancies between reviewers were resolved).
Is sufficient detail of the individual studies presented?
The review should demonstrate that the studies included are suitable to answer the question posed and that a judgement on the appropriateness of the authors' conclusions can be made. If a review includes a table giving information on the design and results of the individual studies, or includes a narrative description of the studies within the text, this criterion is usually fulfilled. If relevant, the tables or text should include information on study design, sample size in each study group, patient characteristics, description of interventions, settings, outcome measures, followup, drop-out rate (withdrawals), effectiveness, results, and adverse events.
Are the primary studies summarized appropriately?
The authors should attempt to synthesize the results from individual studies. In all cases, there should be a narrative summary of results, which may or may not be accompanied by a quantitative summary (meta-analysis). For reviews that use a meta-analysis, heterogeneity between studies should be assessed using statistical techniques. If heterogeneity is present, the possible reasons (including chance) should be investigated. In addition, the individual evaluations should be weighted in some way (e.g., according to sample size, or inverse of the variance) so that studies that are considered to provide the most reliable data have greater impact on the summary statistic.
Is there a transparent system to evaluate the quality of body of evidence? Were the authors’ conclusions supported by the evidence they presented?
Optimally, a reviewer will also appraise the body of literature as a whole, pointing out strengths and weaknesses, and offering an assessment of their confidence in the conclusions.
What was the funding source and role of funder?