This report is available in PDF (1.1 MB) only. People using assistive technology may not be able to fully access information in these files. For additional assistance, please contact us.
Structured Abstract
Background
Despite a significant number of published systematic reviews of implantable medical devices, no empirical evaluation of the reviews has been performed. We conducted a critical appraisal of the quality of reporting in systematic reviews of implantable medical devices to understand the methodologies used; identify current strengths, limitations, deficiencies, and unique challenges; and make recommendations to improve future conduct and reporting.
Methods
A Technical Expert Panel (TEP) of private and public payers, industry, and U.S. Food and Drug Administration representatives helped to refine the scope of the project, identified device categories, and provided feedback on the methodological approach. Five device categories were reviewed: cardiac implantable devices (e.g., defibrillators), vascular interventional devices (e.g., stents), orthopedic implants (e.g., prosthetic discs), skin-replacement grafts (e.g., wound care products), and neurostimulators (e.g., deep brain neurostimulators). Searches were conducted in MEDLINE® and the Cochrane Database of Systematic Reviews for systematic reviews published from January 2009 to December 2010. The search was repeated to cover January 2004 to January 2009 to identify additional eligible reviews for skin-replacement grafts and cardiac implantable devices. We addressed 30 items about quality of reporting in systematic reviews from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Meta-analysis of Observational Studies in Epidemiology (MOOSE) guidelines. With input from the TEP, we developed eight device- and procedure-specific items relevant to the evaluation of implantable medical devices.
Results
Our searches yielded 467 citations, of which 181 met the eligibility criteria: 19 (10 percent) evaluated cardiac implantable devices, 124 (69 percent) evaluated vascular interventional devices, 16 (9 percent) evaluated orthopedic implants, 8 (4 percent) evaluated skin-replacement grafts, and 14 (8 percent) evaluated neurostimulators. Of the 181 systematic reviews included, 123 (68 percent) involved meta-analyses and the remaining 58 (32 percent) involved no meta-analyses; 66 (36 percent) reviewed randomized trials only, 51 (28 percent) reviewed nonrandomized studies only, 56 (31 percent) reviewed both, and 8 (4 percent) did not explicitly report the study designs included. Twenty of 30 PRISMA and MOOSE items we ascertained were commonly reported in greater than 50 percent of the reviews. Device-specific information was less commonly reported in reviews--differences in characteristics across devices (47 percent), differences in characteristics within a device (36 percent), and evolution of technology and its potential effects (21 percent). Operator-specific information was rarely reported in reviews--including training of providers (1 percent), ramp-up in provider technique or learning curve (7 percent), evaluation of team expertise (9 percent), practitioner variability (10 percent), and volume at each study site (8 percent).
Conclusions
Our evaluation of 181 systematic reviews on implantable medical devices reveals a lack of reporting of some important generic items applicable to any systematic review as well as device- and operator-specific information. We identified eight device- or operator-specific items that might be of value in reporting on systematic reviews of implantable devices and could be incorporated into reporting guidelines.
