Methods for Synthesising Evidence at the Systematic Review Level: An Introduction to Commonly Encountered Scenarios in ‘Overviews of Systematic Reviews’

Jan 7, 2019 - 12:00pm to 1:00pm
700-828 W. 10 Ave., VGH Research Pavilion
Carole Lunny

Navigating the expanding body of research literature is an increasing problem for health practitioners, researchers and decision-makers. Global research output is growing rapidly, and it is estimated that every nine years, publications in Web of Science double. Similarly, the number of published systematic reviews (SRs) being produced yearly is also expanding. In 2010, Bastian and colleagues highlighted that 11 SRs were published per day. ‘Overviews of systematic reviews’ (also called overviews, umbrella reviews, meta-reviews, reviews of reviews) have evolved in response to the growing number of SRs, and have grown in number. Overviews summarise the results of multiple SRs, and help make sense of potentially conflicting, discrepant and overlapping information across similar SRs.

While the methods for conducting SRs are well developed, the methods for overviews are still being developed. Overview authors often encounter unique methodological challenges for which there are no obvious solutions or clear guidance, and current practice is driven largely by personal experience and trial and error. For example: discrepancy in data reported across SRs; dealing with the inclusion of the same primary study in multiple SRs (‘overlap’); and dealing with discordant results across SRs. While many methods have been proposed to address some of these unique challenges, there is often a range of different methods that could be applied, but without guidance on which approach might be optimal. For example, a common method for dealing with multiple SRs with overlapping primary studies is to specify eligibility criteria to include only one SR. However, alternative methods exist such as using statistical approaches to deal with overlap. We developed a systematic review to catalogue the full range of overview methods, called the MOoR Framework. The objectives of this presentation are:

  1. Introduce, define and describe ‘overview of systematic reviews’
  2. Describe eight scenarios which present unique challenges to overview authors
  3. Summarise methods used to handle or resolve the unique challenges
  4. Provide examples of methods used to handle the scenarios in real world overviews of reviews