Skip to main content
Effective Health Care Program

Correcting for Publication Bias in the Presence of Covariates

Research Report

People using assistive technology may not be able to fully access information in this file. For additional assistance, please contact us.

Structured Abstract

Objectives

To date, there are no established methods for assessing publication bias when study characteristics induce heterogeneity in the effects. The "trim and fill" method was developed to adjust for censored (i.e., missing) studies in a meta-analysis, assumed due to publication bias. We sought to modify this algorithm for use in the context where study characteristics induce heterogeneity in the effects.

Methods

An iterative algorithm based on the original trim and fill algorithm was developed. We performed Monte Carlo simulations with 5,000 iterations per instance of the adapted trim and fill algorithm. In each instance we set six parameters, both to alter the structure of the randomly generated data, and to manipulate the algorithm itself. We assessed the average performance (type 1 error, power, bias) of the algorithm, in the context of inference regarding the metaregression parameters. We also applied the method to data from 19 randomized studies examining the hypothesis that teachers' expectations influence students' IQ intelligence test scores, the covariate of interest being the dichotomized length of teacher-student contact prior to the study. We developed user-friendly software in R, for one covariate at this stage, with future versions to incorporate several covariates.

Results

Meaningful, albeit incomplete, reduction in the bias of estimated metaregression model parameters was achieved. Bias and coverage probability improved as the number of studies increased. The R estimator outperformed both L and Q from the original trim and fill method. Performance declined in the presence of large heterogeneity, but substantial bias reduction was still obtained. Two algorithm variants were developed, with the simpler one-dimensional version performing slightly better than the two-dimensional.

Conclusions

This new method provides a generalized trim and fill algorithm that is applicable to metaregression, that is, where covariates are available. The new algorithm should be seen as a sensitivity analysis to the influence of covariates on funnel plot asymmetry.