Skip to main content
Effective Health Care Program

Systematizing the Use of Value of Information Analysis in Prioritizing Systematic Reviews

Research Report

People using assistive technology may not be able to fully access information in these files. For additional assistance, please contact us.

Structured Abstract

Background

As a means to inform clinical and health policy decisions, systematic reviews are inexpensive relative to clinical trials and other (observational) studies but are costly enough that not all possible systematic reviews can be performed. Value of information (VOI) analysis has been considered as a tool to help prioritize topics for systematic review. Since VOI analysis typically involves constructing a complex decision-analytic model of the disease and its treatment to fully characterize the uncertainty in the health outcomes and costs of the treatments or other health interventions being studied, the standard approach to VOI may be prohibitively costly for use in prioritizing systematic reviews. As alternatives to typical full modeling VOI, three newer approaches to analyzing the value of information can be identified that are less burdensome: (1) In a conceptual approach to VOI, information is used about multiplicative elements of VOI, which include comprehensive outcome measures and the implementation and durability of review findings, to provide informative bounds on value of research without formally quantifying this through modeling. (2) In a minimal modeling approach to VOI, which is possible when data on comprehensive outcome measures, such as quality-adjusted life-years or net benefit, are readily available from existing research, VOI can be estimated without constructing a complex model. (3) In a maximal modeling approach to VOI, a single comprehensive model may be constructed to simultaneously inform priorities concerning multiple clinical questions. The presence of these lower cost VOI methods creates the possibility for VOI analysis to be practically applied in priority-setting process for systematic reviews, and raises questions about how the use of VOI can be systematized.

Methods

This study (1) reviews VOI methods and the approaches currently used to inform priorities for systematic reviews in the literature and in practice; (2) describes an algorithm for selecting an effective and efficient approach to VOI for a given clinical question; and (3) applies this algorithm to assess its potential utility in prioritizing topics nominated to the Agency for Healthcare Research and Quality (AHRQ) and Quality Evidence-based Practice Centers (EPCs) for systematic review.

Results

Our review of past research identified a substantial number of VOI studies but found that VOI had not been used to set priorities for systematic reviews. We did find that many of the elements that are used to estimate VOI are often considered in prioritizing systematic reviews, but they are rarely quantified or combined in an explicit manner that would be consistent with VOI principles. We propose an algorithm that describes a multistage process for identifying an effective and efficient approach to VOI for a given clinical question. This process begins with conceptual VOI to provide informative bounds on VOI, followed by the clustering of review topics and consideration of the use of maximal modeling VOI, and then minimal modeling using comprehensive outcome measures of the benefits of the alternative treatments or health interventions under study. In applying our algorithm to topics nominated to AHRQ's EPCs for systematic review, we found the algorithm useful in selecting the appropriate VOI methods to inform priorities for systematic reviews and found examples in which each of the lower cost VOI approaches might be valuable. Although full modeling VOI may aid in the planning and design of research, we find limited conditions for its use in prioritizing systematic reviews.

Conclusions

We conclude that consideration of VOI principles and methods may have a useful role in informing priorities for systematic reviews. VOI can help decision makers and others involved in priority-setting for systematic reviews to explicate criteria and quantify measures of expected value that can be used to prioritize reviews. Systematic application of VOI using an algorithm that guides the choice among conceptual VOI, maximal modeling, minimal and full modeling may aid in minimizing the costs and burden to the practical application of VOI. We propose future work that would (1) incorporate VOI into the process by which systematic review topics proposed to AHRQ's EPCs are prioritized, and (2) assess whether that process is found to be useful by decision makers and others.