Skip to main content
Effective Health Care Program

Diabetes Medications for Adults with Type 2 Diabetes: An Update Focused on Monotherapy and Add-on Therapy to Metformin

Research Protocol
Download PDF861.3 KB

Background and Objectives for the Systematic Review

Public Health Burden of Type 2 Diabetes

Type 2 diabetes currently affects 9.3% of the US population (29.1 million people) and is growing in prevalence.1 Diabetes and its complications impose a substantial public health burden as they contribute significantly to mortality, morbidity, and health care costs.1-3 Diabetes-related complications was the seventh leading cause of death in the US in 2010. Diabetes increases the risk of cardiovascular-related death nearly two-fold and is the leading cause of new-onset blindness and new-onset end-stage renal disease in adults in the U.S.4 Costs related to diabetes were approximately $245 billion in 2012.1

Medication Management of Type 2 Diabetes in 2014

Diabetes medications can effectively reduce morbidity and mortality associated with diabetes,5,6 yet there is uncertainty about the comparative effectiveness and safety of the different medications as monotherapies and when used in combination (especially regarding long-term outcomes and safety).

Including insulin, there are 10 medication classes with approval by the U.S. Food and Drug Administration (FDA) for treatment of type 2 diabetes. These medications vary in their effectiveness at reducing hemoglobin A1c (HbA1c) and their propensities to cause side effects and serious adverse events; and not all are approved for monotherapy. These medications are typically FDA-approved based on evidence from a small number of randomized clinical trials. Additional evidence is usually incorporated into the labels. If needed, warnings are issued when safety signals become apparent through case reports and post-approval studies. While the FDA has become stricter regarding pre- and post-approval evaluation of cardiovascular risk for diabetes medications,7 serious safety concerns about these medications continue to arise.8,9

Metformin has strong evidence to support its use as the initial pharmacologic treatment for most patients with type 2 diabetes.4 However, the evidence base regarding the best drug to add to metformin continues to evolve. Additionally, the evidence regarding the comparative effects and safety of other monotherapies among patients who cannot initiate or who cannot tolerate metformin remains unclear.

Evolving Evidence on the Comparative Effectiveness of Medications for Type 2 Diabetes

Effective Health Care (EHC) Program systematic reviews, published in 2007 and 2011, compared monotherapies and medication combinations for adults with type 2 diabetes.10,11 Data on the newly-approved medication classes (e.g., dipeptidyl-peptidase 4 (DPP-4) inhibitors) were sparse, and data on long-term outcomes for both older and newer medications were lacking.12,13

Since January 2010, one new medication class [the sodium-glucose co-transporter 2 (SGLT-2) inhibitors, with three new medications] and several new DPP-4 inhibitors and glucagon-like peptide-1 (GLP-1) agonists have been approved by the FDA. Additional data on the earlier-approved medications have also emerged since 2010 which may change the balance of benefit and risk attributable to these drugs or may alter the strength of evidence on some of the drug comparisons that we evaluated.9,14-16 For instance, in 2010, the FDA restricted rosiglitazone prescription sales based on evidence of increased heart attack and stroke. However, in 2013, a re-analysis of the pivotal trial substantially reduced the FDA’s concern leading to removal of the sales restrictions.17

Given the continued evidence being generated about new and established type 2 diabetes medications, an updated systematic review evaluating the effects of the newer and older medications on intermediate and long-term effectiveness and safety outcomes will be especially useful to clinicians, patients, investigators, guideline developers, and payers. In this era of intensive direct-to-consumer marketing of new drugs, clinicians need a trustworthy source of comprehensive information about the comparative effectiveness and safety of medications. This review will be useful to diverse clinicians who need to discuss treatment options with their patients, including family practitioners, general internists, nurse practitioners, physician assistants, nurses, pharmacists, endocrinologists, cardiologists, nephrologists, and others. Patients and patient advocates also will find the information valuable when making decisions about treatment options. Finally, investigators will be able to use the review to identify gaps in the literature and formulate original research questions to fill these knowledge gaps.

Key Questions

This review will update the 2011 review on oral diabetes medications for adults with type 2 diabetes.11 This review will differ from the 2011 review in the following ways:

  • A focus will be placed on priority head-to-head drug comparisons, identified a priori as clinically relevant comparisons for which there are evidence gaps (see Table 1).
  • The inclusion of a new FDA-approved class of oral diabetes medications: SGLT-2 inhibitors, including empagliflozin, dapagliflozin, and canagliflozin.
  • The inclusion of new DPP-4 inhibitors: linagliptin and alogliptin.
  • The inclusion of new GLP-1 agonists: albiglutide and dulaglutide.
  • The inclusion of the safety outcomes of impaired renal function, urinary tract infections, genital mycotic infections, volume depletion, and hip and non-hip fractures for studies with a comparison including SGLT-2 inhibitors. We will not review these outcomes for any medication classes or comparisons except those including SGLT-2 inhibitors.1,2
  • The inclusion of systolic blood pressure and heart rate as intermediate outcomes for studies with a comparison including either SGLT-2 inhibitors or GLP-1 agonists.
  • The exclusion of meglitinides as an intervention of interest.3
  • The exclusion of lipid concentrations as an intermediate outcome.4

1Strength of evidence on fractures was high in our previous report and indicated that the risk of fracture was limited to thiazolidinediones (and not the other classes evaluated in that report).11 Data on SGLT-2 inhibitors are less clear for this outcome.
2A meta-analysis published in 2013,18 and data from pivotal trials19,20 have suggested that renal impairment, urinary tract infections, genital mycotic infections, volume depletion, and fractures are potential risks of SGLT-2 inhibitors.
3Meglitinides have been FDA-approved since 1997 and are not commonly used in current clinical practice (used <1% of the time) as evidenced by two recent national pharmacoepidemiology studies.21,22 We evaluated meglitinides in our first systematic review10 and in our 2011 update11 and found that they have similar effects on HbA1c and similar rates of hypoglycemia as sulfonylureas. The 2011 update did not add any relevant new information for clinicians or patients related to this medication versus its comparators. We are not aware of important new evidence for this class of medications which would be expected to change our findings.
4LDL targets are no longer universally the primary factor guiding the use of cholesterol-lowering therapy. Current guidelines suggest that 10-year global CVD risk be used to determine statin usage and intensity and this global risk score does not include LDL cholesterol. Furthermore, triglycerides and HDL are also not usual targets of cholesterol therapy, and statin usage is recommended for all patients age 40 and over with diabetes in the US.23

The proposed key questions are:

Key Question 1a

In adults age 18 or older with type 2 diabetes mellitus, what is the comparative effectiveness of the specified monotherapy FDA-approved diabetes medications (see Table 1) for the intermediate outcomes of hemoglobin A1c, weight, systolic blood pressure (for comparisons including SGLT-2 inhibitors or GLP-1 agonists), and heart rate (for comparisons including SGLT-2 inhibitors or GLP-1 agonists)?

Key Question 1b

In adults age 18 or older with type 2 diabetes mellitus, what is the comparative effectiveness of the specified metformin-based combinations of FDA-approved diabetes medications (see Table 1) for the intermediate outcomes of hemoglobin A1c, weight, systolic blood pressure (for comparisons including SGLT-2 inhibitors or GLP-1 agonists), and heart rate (for comparisons including SGLT-2 inhibitors or GLP-1 agonists)?

Key Question 2a

In adults age 18 or older with type 2 diabetes mellitus, what is the comparative effectiveness of the specified monotherapy FDA-approved diabetes medications (see Table 1) for the long-term clinical outcomes of all-cause mortality, cardiovascular and cerebrovascular morbidity and mortality, retinopathy, nephropathy, and neuropathy?

Key Question 2b

In adults age 18 or older with type 2 diabetes mellitus, what is the comparative effectiveness of the specified metformin-based combinations of FDA-approved diabetes medications (see Table 1) for the long-term clinical outcomes of all-cause mortality, cardiovascular and cerebrovascular morbidity and mortality, retinopathy, nephropathy, and neuropathy?

Key Question 3a

In adults age 18 or older with type 2 diabetes mellitus, what is the comparative safety of the specified monotherapy FDA-approved diabetes medications (see Table 1) regarding liver injury, lactic acidosis, pancreatitis, hypoglycemia, congestive heart failure, cancer, severe allergic reactions, macular edema or decreased vision, and gastrointestinal side effects; and for comparisons including SGLT-2 inhibitors, urinary tract infections, impaired renal function, genital mycotic infections, fracture, and volume depletion?

Key Question 3b

In adults age 18 or older with type 2 diabetes mellitus, what is the comparative safety of the specified metformin-based combinations of FDA-approved diabetes medications (see Table 1) regarding liver injury, lactic acidosis, pancreatitis, hypoglycemia, congestive heart failure, cancer, severe allergic reactions, macular edema or decreased vision, and gastrointestinal side effects; and for comparisons including SGLT-2 inhibitors, urinary tract infections, impaired renal function, genital mycotic infections, fracture, and volume depletion?

Key Question 4

Do the comparative safety and effectiveness of these treatments differ across subgroups defined by the age, sex, race/ethnicity, and body mass index (BMI) of adults with type 2 diabetes?

PICOTS

Population(s)

  • The population of interest is non-pregnant adults with type 2 diabetes mellitus.
  • We will not include studies where everyone included has at least one of the following comorbid conditions:
    • End-stage liver disease or cirrhosis
    • End-stage renal disease (i.e., stage 5 chronic kidney disease or dialysis)
    • Cancer
    • New onset diabetes after an organ transplant
    • Cardiovascular event within the past 3 months [e.g., acute coronary syndrome, acute myocardial infarction, coronary artery bypass grafting or percutaneous intervention (angioplasty or stent placement)]

Interventions

  • We will include evaluations of these FDA-approved therapies:
    • Biguanides: metformin
    • Thiazolidinediones (TZDs): rosiglitazone, pioglitazone
    • Second-generation sulfonylureas: glyburide, glibenclamide, glipizide, glimepiride
    • DPP-4 inhibitors: sitagliptin, saxagliptin, linagliptin, alogliptin
    • SGLT-2 inhibitors: dapagliflozin, canagliflozin, empagliflozin
    • GLP-1 agonists: exenatide, liraglutide, albiglutide, dulaglutide
    • Combination of metformin and a TZD
    • Combination of metformin and a sulfonylurea
    • Combination of metformin and a DPP-4 inhibitor
    • Combination of metformin and a SGLT-2 inhibitor
    • Combination of metformin and a GLP-1 agonist
    • Combination of metformin and a basal insulin (insulin glargine, insulin detemir, neutral protamine Hagedorn (NPH) insulin)
    • Combination of metformin and a premixed insulin (NPH/regular 50/50, NPH/regular 70/30, insulin lispro 50/50, insulin lispro 75/25, insulin aspart 70/30)
  • We will exclude meglitinides,1 acarbose,6 colesevelam, and bromocriptine due to infrequent use in the U.S. and the expectation of little relevant new evidence since the 2011 update which would change prior findings.11

Comparators

See Table 1 for monotherapy and combination therapy comparisons we will include.

Outcomes for each question

  • The intermediate outcomes included in KQ1 will be:
    • Hemoglobin A1c (will not evaluate metformin vs. sulfonylureas, evidence is high grade from the prior report)
    • Weight (will not evaluate for metformin vs. thiazolidinediones or metformin vs. sulfonylureas; evidence is high grade from the prior report)
    • Systolic blood pressure (will evaluate for included studies of SGLT-2 inhibitors and GLP-1 agonists only)
    • Heart rate (will evaluate for included studies of SGLT-2 inhibitors and GLP-1 agonists only)
  • The long-term clinical outcomes included in KQ2 will be:
    • All-cause mortality
    • Cardiovascular and cerebrovascular morbidity and mortality
    • Retinopathy
    • Nephropathy
    • Neuropathy
  • The safety outcomes included in KQ3 will be (will not evaluate any of these for metformin vs. sulfonylureas except cancer; evidence is high grade from the prior report):
    • Liver injury
    • Lactic acidosis
    • Pancreatitis
    • Hypoglycemia
    • Congestive heart failure
    • Cancer
    • Severe allergic reactions
    • Macular edema or decreased vision
    • Gastrointestinal side effects
    • Urinary tract infections for comparisons that include SGLT-2 inhibitors
    • Impaired renal function comparisons that include SGLT-2 inhibitors
    • Genital mycotic infections for comparisons that include SGLT-2 inhibitors
    • Fracture for comparisons that include SGLT-2 inhibitors
    • Volume depletion for comparisons that include SGLT-2 inhibitors
  • KQ4 will consider any of the outcomes.

Timing

We will include studies if participants are on the medications for at least 3 months, 12 weeks, or 90 days.

Settings

We will include all study settings.

Table 1. Priority Medication Comparisons Included for Each Key Question
Type of Therapy Main Intervention Comparisons
DPP-4 inhibitor = dipeptidyl peptidase-4 inhibitor; GLP-1 agonist = glucagon-like peptide-1 receptor agonist; SGLT-2 inhibitor = sodium-glucose co-transporter 2
* For studies comparing thiazolidinediones with metformin, we will review only the outcomes of HbA1c, long-term outcomes, and select safety outcomes given the high strength of evidence from our prior evidence report for other outcomes (specifically fracture and weight).11
For studies comparing sulfonylureas with metformin, we will review only the long-term outcomes and cancer given the high strength of evidence on the other outcomes from our prior CER.11
Monotherapy as main intervention Metformin
  • Thiazolidinediones*
  • Sulfonylureas
  • DPP-4 inhibitors
  • SGLT-2 inhibitors
  • GLP-1 agonists
  • Combination of metformin plus thiazolidinedione
  • Combination of metformin plus sulfonylurea
  • Combination of metformin plus DPP-4 inhibitor
  • Combination of metformin plus SGLT-2 inhibitor
  • Combination of metformin plus GLP-1 agonist
Thiazolidinedione
  • Sulfonylureas
  • DPP-4 inhibitors
  • SGLT-2 inhibitors
  • GLP-1 agonists
Sulfonylurea
  • DPP-4 inhibitors
  • SGLT-2 inhibitors
  • GLP-1 agonists
DPP-4 inhibitor
  • SGLT-2 inhibitors
  • GLP-1 agonists
SGLT-2 inhibitor
  • GLP-1 agonists
Combination therapy as main intervention Combination of metformin plus (thiazolidinedione or sulfonylurea or DPP-4 inhibitor or SGLT-2 inhibitor or GLP-1 agonist or basal insulin) Combination of metformin plus (sulfonylurea or DPP-4 inhibitor or SGLT-2 inhibitor or GLP-1 agonist or basal insulin or premixed insulin)

Analytic Framework

Figure 1. Conceptual model

Figure 1 displays the conceptual model. Patients with type 2 diabetes for whom diet and lifestyle management have failed are treated with monotherapy. Monotherapies include biguanides, thiazolidinediones, sulfonylureas, DPP-4 inhibitors, SGLT-2 inhibitors, and GLP-1 agonists. Some patients are treated with combination therapy, which could include the addition of a second oral agent, insulin, or a GLP-1 agonist. Outcomes were categorized as intermeiate outcomes (addressed in KQ1a and 1b), long-term outcomes (addressed in KQ2a and 2b), safety/adverse events (addressed in KQ3a and 3b). Intermediate outcomes include hemoglobin A1c, weight, systolic blood pressure, and heart rate. Long-term outcomes inlcude macrovascular complications (cardiovascular or cerebrovascular disease), microvascular complications (retinopathy, nephropathy, and neuropathy), and mortality. Safety and adverse events include hypoglycemia, congestive heart failure, pancreatitis, fractures, cancer, and genital mycotic infections.

BMI = body mass index; DPP-4 = dipeptidyl peptidase-4; GLP1 = glucagon-like peptide-1; HbA1c = hemoglobin A1c; KQ=key question; NPH = neutral protamine Hagedorn; SGLT-2 inhibitor = sodium-glucose co-transporter 2

Methods

Criteria for Inclusion/Exclusion of Studies in the Review

The inclusion and exclusion criteria are listed in Table 2.

Table 2. Inclusion and exclusion criteria
PICOTS Inclusion criteria Exclusion criteria
ESLD = end-stage liver disease; ESRD = end-stage renal disease; KQ = Key Question; RCT = randomized controlled trial
* Of note, some outcomes could be classified as either safety or long-term clinical outcomes (e.g., myocardial infarction and cancer).
^ We will not evaluate this outcome for metformin vs. sulfonylurea comparisons as the evidence was high from the prior report.
We will not evaluate this outcome for metformin vs. thiazolidinedione or metformin vs. sulfonylurea comparisons as the evidence was high from the prior report.
We will evaluate this outcome only for comparisons that include a GLP-1 agonist or a SGLT-2 inhibitor.
§ We will evaluate this outcome only for comparisons that include a SGLT-2 inhibitor.
|For crossover randomized trials, we will abstract data on all outcomes at the end of the first period prior to the crossover. If data are not presented at the end of the first period, then we will exclude the article for the following outcomes where we would be unable to draw conclusions about causality: long-term outcomes (KQ2); fractures; cancer; intermediate outcomes in studies where there was a washout period of less than 3 months; and safety outcomes besides hypoglycemia, gastrointestinal side effects, and liver injury in studies where the washout period was less than a month.
We have decided to include non-English language articles through the full text article review phase of the updated search and assess the volume and content of these articles along with workload to determine if abstracting data from these articles will add value to the review.
Population We will include studies of adult humans with type 2 diabetes, non-insulin dependent diabetes mellitus, or adult-onset diabetes.
  • We will exclude studies of patients with type 1 diabetes, impaired glucose tolerance, metabolic syndrome, maturity onset diabetes of youth, and gestational diabetes.
  • We will exclude studies if they included only pregnant women or subjects less than or equal to 17 years of age.
  • We will exclude studies where everyone is required to have at least one of the following comorbid conditions: ESLD, ESRD, cancer, new onset diabetes after organ transplant, or a recent cardiovascular event.
Interventions We will include studies that evaluate a diabetes medication of interest or drug combination of interest (see list under Interventions). We will exclude studies that did not specify the adjunctive medications, such as those stating use of “any oral hypoglycemic” or if the study listed several possible medications without stratification of the results by treatment.
Comparisons We will include studies that evaluate a comparison of interest (see Table 1).
  • We will exclude studies that do not have a comparison group or that use a placebo comparison or non-pharmacological comparison.
  • We will exclude intraclass head-to-head comparisons.
Outcomes*
  • We will include studies addressing the following intermediate outcomes for KQ1:
    • Hemoglobin A1c^
    • Weight
    • Systolic blood pressure
    • Heart rate
  • We will include studies addressing the following long-term clinical outcomes for KQ2:
    • All-cause mortality
    • Cardiovascular and cerebrovascular morbidity and mortality
    • Retinopathy
    • Nephropathy
    • Neuropathy
  • We will include studies addressing the following safety outcomes for KQ3:
    • Liver injury^
    • Impaired renal function§
    • Lactic acidosis^
    • Pancreatitis^
    • Hypoglycemia^
    • Gastrointestinal side effects^
    • Congestive heart failure^
    • Cancer
    • Macular edema or decreased vision^
    • Fractures§
    • Urinary tract infections§
    • Genital mycotic infections§
    • Volume depletion§
  • KQ4 will include studies considering any of the above outcomes.
 
Type of study
  • For KQ1, we will include only RCTs.
  • For KQ2 and KQ3, we will include RCTs, non-randomized experimental studies with a comparison group, and high-quality observational studies with a comparison group.
  • We will include randomized trials utilizing a crossover design with some exceptions.|
  • We anticipate excluding studies not written in English and will exclude articles with no original data.
  • We will exclude meeting abstracts.
Timing and setting   We will exclude studies in which the observed intervention or exposure period is less than 3 months, 12 weeks, or 90 days.

Searching for the Evidence

The 2011 review searched the following databases for the dates: MEDLINE® (1966 to April 2010), Embase™ (1974 to April 2010), and the Cochrane Central Register of Controlled Trials (CENTRAL). Per AHRQ’s guidance, we will include an overlap of at least 1 year in the search dates.24 We will run the search strategy developed for the 2011 review with date restrictions of April 2009 through July 2014 (see Appendix).

An additional expanded search will include medical subject headings (MeSH) and text words for the new medications included in the update (e.g., linagliptin). The expanded search will not have any date restrictions.

The searches will be updated during the peer review process. We will handsearch the reference lists of all newly included articles and relevant systematic reviews. Additionally, the team will search ClinicalTrials.gov to identify relevant registered trials. We will review any Scientific Information Packets provided by the manufacturers. We will also review the FDA website for any unpublished additional studies relevant to this topic.

Two independent reviewers will conduct title scans. For a title to be eliminated at this level, both reviewers will need to indicate that the study was ineligible. If the reviewers disagree, the article will be advanced to the next level, which is abstract review.

The abstract review phase will be designed to identify studies reporting the effectiveness or safety of the medications and combinations of interest. Abstracts will be reviewed independently by two investigators and will be excluded if both investigators agree that the article meets one or more of the exclusion criteria (see the inclusion and exclusion criteria listed in Table 2). Differences between investigators regarding the inclusion or exclusion of abstracts will be tracked and resolved through consensus adjudication.

Articles promoted on the basis of the abstract review will undergo another independent parallel review to determine if they should be included in the final qualitative and quantitative systematic review and meta-analysis. The differences regarding article inclusion will be tracked and resolved through consensus adjudication.

Data Abstraction and Data Management

We will use a systematic approach to extract all data to minimize the risk of bias in this process. We will use standardized forms from the previous reviews as templates for data extraction and pilot test them for the new medications and outcomes. By creating standardized forms for data extraction, we seek to maximize consistency in identifying all pertinent data available for synthesis.

Each article will undergo double review by the study investigators for data abstraction. The second reviewer will confirm the first reviewer’s abstracted data for completeness and accuracy. Reviewer pairs will be formed to include personnel with both clinical and methodological expertise. A third reviewer will audit a random sample of articles to ensure consistency in the data abstraction of the articles. Reviewers will not be masked to the authors of the articles, their respective institutions, nor the journals in which their articles were published.

For all articles, the reviewers will extract information on general study characteristics (e.g., study design, study period, and followup), study participants (e.g., age, sex, race, weight/body mass index, hemoglobin A1c levels, and duration of diabetes), interventions (e.g., initial, maximum, and mean doses, frequency of use, duration of use, and permissibility of treatment intensification with additional therapies), comparisons, the method of ascertainment of outcomes, and the outcome results, including measures of variability. We will also collect data on outcomes for the subgroups of interest, including age, sex, race/ethnicity, and BMI.

All information from the article review process will be entered into a DistillerSR database (Evidence Partners Inc., Ottawa, Canada) by the reviewer. Reviewers will enter comments into the system whenever applicable. The DistillerSR database will be used to maintain the data and to create detailed evidence tables and summary tables. We may contact the authors of the included studies for additional data, if necessary. Data will later be uploaded into the Systematic Review Data Repository.

Assessment of Methodological Risk of Bias of Individual Studies

Two independent reviewers will assess study quality. We will assess the risk of bias in individual RCTs using the Jadad criteria consistent with the prior report.25 We will use the Downs and Black tool for assessment of internal validity for non-randomized trials and observational studies.26 Given that observational studies that have a high risk of bias add little value to a systematic review of effectiveness,27 we will include only high-quality observational studies as determined by assessment of each study’s risk of bias. For inclusion, we will require that observational studies adjust for the following confounders in their analysis: age, sex, and co-morbid conditions (defined by using a co-morbidity scale or index; by including other medical conditions or medications used by the patient; or with a propensity score or other method(s) to adjust for confounding by indication). We will also require that included observational studies have accounted for losses to follow up in the analysis (such as by using a time-to-event analysis), state that the losses to follow up were less than 20 percent, or state that the individuals included in the analysis were similar to those lost to follow up or in the original cohort. If the study meets both the confounding and losses to follow up criteria and most of the other Downs and Black internal validity criteria, the observational study will be considered eligible for the review. For case-control studies, in particular, we will also require that cases and controls were recruited from the same population and during the same time period to be eligible. We will record reasons for exclusions of observational studies as we will for all excluded studies. The Downs and Black tool and other inclusion criteria for nonrandomized trials and observational studies will be applied to newly identified studies from the planned update and on non-randomized trials and observational studies included in the prior report.11

Data Synthesis

For each Key Question, we will create a set of detailed evidence tables containing all information extracted from eligible studies, including those from the prior CERs. We will conduct meta-analyses when there are sufficient data (at least three trials) and studies are sufficiently homogenous with respect to key variables (population characteristics, study duration, and drug dose). We will use the results of individual studies included in the prior reports as well as those from newly-identified studies in this report as described below.

Since we anticipate that most molecules will have similar physiologic effects within class, we will combine studies of unique medications within classes when reporting outcomes except where known differences exist (such as the effects of pioglitazone and rosiglitazone on cardiovascular outcomes). If we see substantial heterogeneity (I-squared >50%) in pooled estimates for any outcome, we will explore whether this is due to pooling studies of unique medications. We will then stratify studies by medication and repeat the pooled analyses and measures of heterogeneity.

For continuous outcomes, we will extract the mean difference between groups along with its measure of dispersion. If this is not reported, we will calculate the point estimate using the mean difference from baseline for each group. If the mean difference from baseline is not reported, we will calculate this from the baseline and final values for each group.28 If there are no measures of dispersion for the mean difference from baseline for each group, we will calculate the variance using the standard deviation of the baseline and final values, assuming a correlation between baseline and final values of 0.5. For trials that have more than one dosing arm, we will choose the arm that is most consistent with dosing in the other trials. When more than one followup interval is reported, we will use the data from the followup most similar to the other trials. We will report the rest of the results descriptively.

Heterogeneity among the trials for each outcome we consider appropriate for quantitative pooling will be tested using a standard chi-squared test using a significance level of alpha less than or equal to 0.10. We also will examine heterogeneity among studies with an I-squared statistic, which describes the variability in effect estimates that is due to heterogeneity rather than random chance. A value greater than 50 percent will be considered to indicate substantial heterogeneity.29

We will pool the mean difference between groups using a random-effects model with the DerSimonian and Laird formula in settings of low heterogeneity30 or with appropriate analyses when there is higher heterogeneity.31 When data are not sufficient to combine in a meta-analysis, we will summarize the outcomes by reporting the ranges of values for mean differences from baseline or mean differences between groups (when possible).

For the outcome of hypoglycemia, we will conduct separate analyses for: (a) severe hypoglycemia and (b) mild or moderate hypoglycemia. The categories will be based on the definitions of hypoglycemia provided in the studies. For hypoglycemia and all other dichotomous outcomes, we will calculate pooled odds ratios using a random-effects model with the DerSimonian and Laird formula in settings of low heterogeneity,30 or with appropriate analyses for higher heterogeneity.31

We will attempt to determine reasons for heterogeneity by evaluating study-level characteristics such as baseline values of the outcome and duration of diabetes using metaregression techniques or stratification of meta-analyses. We will conduct sensitivity analyses by omitting one study at a time to assess the influence of any single study on the pooled estimates.

Publication and reporting biases will be assessed in the following ways32 in the included randomized controlled trials:

  1. Publication bias will be evaluated by:
    1. Visually assessing the symmetry of funnel plots
    2. Using the Begg and Mazumdar33 and the Egger34 test to quantitatively assess for publication bias. If publication bias is present, we will use the trim and fill technique35 to assess the impact of publication bias on the point estimate and confidence interval for any pooled analyses.
    3. Comparing ClinicalTrials.gov entries and actual publications
    4. Comparing FDA medical and statistical reviews and actual publications
  2. Selective Outcomes Reporting bias (i.e., did the publications report on the outcomes that they pre-specified) will be evaluated by comparing differences in reporting of outcomes of HbA1c, hypoglycemia, and all-cause mortality in the actual publications to the ClinicalTrials.gov entries, to the published study protocols referenced in the actual publication, to the methods sections of included publications, and to the FDA medical and statistical reviews.
  3. Selective Analysis Reporting bias will be evaluated by:
    1. Assessing if subgroups of interest (i.e., age, sex, race/ethnicity, and BMI) were pre-specified in the analysis plan a priori.
    2. Assessing the precision of outcome data reporting by determining the number and percent of studies which report on an outcome of interest (e.g., HbA1c) but do not report a precise measure of dispersion completely or at all.

Grading the Strength of Evidence (SOE) for Major Comparisons and Outcomes

At the completion of our review, we will grade the quantity, quality and consistency of the best available evidence addressing the Key Questions by adapting an evidence grading scheme recommended by the Guide for Conducting Comparative Effectiveness Reviews.36 We will apply evidence grades to the bodies of evidence about each intervention comparison for each intermediate outcome, long-term outcome, and for hypoglycemia. Additionally, we will grade the strength of evidence for adverse events that are most relevant for a particular intervention comparison (e.g., volume depletion for comparisons including SGLT-2 inhibitors). We will assess the quality and consistency of the best available evidence, including assessment of limitations to individual study quality (using individual risk of bias assessments), consistency, directness, precision, reporting bias, and the magnitude of the effect.

We will classify evidence pertaining to the Key Questions into four categories: (1) “high” grade (indicating high confidence that the evidence reflects the true effect and further research is very unlikely to change our confidence in the estimate of the effect); (2) “moderate” grade (indicating moderate confidence that the evidence reflects the true effect but further research could change our confidence in the estimate of the effect and may change the estimate); (3) “low” grade (indicating low confidence that the evidence reflects the true effect and further research is likely to change our confidence in the estimate of the effect and is likely to change the estimate); and (4) “insufficient” grade (indicating evidence is unavailable or the body of evidence has unacceptable deficiencies, precluding reaching a conclusion).

Assessing Applicability

We will discuss the applicability of studies in terms of the degree to which the study population (e.g., age, sex, race/ethnicity,37 and co-morbid conditions), interventions (e.g., dose, frequency, rescue therapy, duration of exposure), outcomes (e.g., outcome definition and reporting), and settings are typical of the treatment of individuals with type 2 diabetes who are receiving treatment in a usual care setting (conceived as outpatient treatment by internists, family physicians, and endocrinologists).

References

  1. Centers for Disease Control and Prevention. National Diabetes Statistics Report: Estimates of Diabetes and Its Burden in the United States, 2014. Atlanta, GA: U.S. Department of Health and Human Services; 2014 [cited 2014 July 9]; Available from: https://www.cdc.gov/diabetes/pdfs/data/2014-report-estimates-of-diabetes-and-its-burden-in-the-united-states.pdf.
  2. Centers for Disease Control and Prevention. National Diabetes Fact Sheet: National Estimates and General Information on Diabetes and Prediabetes in the United States, 2011. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention; 2011 [cited 2014 July 9]; Available from: http://www.cdc.gov/diabetes/pubs/pdf/ndfs_2011.pdf.
  3. Economic costs of diabetes in the U.S. In 2007. Diabetes Care. 2008 Mar;31(3):596-615.
  4. Standards of medical care in diabetes--2014. Diabetes Care. 2014 Jan;37 Suppl 1:S14-80.
  5. Stratton IM, Adler AI, Neil HA, Matthews DR, Manley SE, Cull CA, et al. Association of glycaemia with macrovascular and microvascular complications of type 2 diabetes (UKPDS 35): prospective observational study. BMJ. 2000 Aug 12;321(7258):405-12.
  6. Holman RR, Paul SK, Bethel MA, Matthews DR, Neil HA. 10-year follow-up of intensive glucose control in type 2 diabetes. N Engl J Med. 2008 Oct 9;359(15):1577-89.
  7. Center for Drug Evaluation and Research. Guidance for Industry. Diabetes Mellitus - Evaluating Cardiovascular Risk in New Antidiabetic Therapies to Treat Type 2 Diabetes. Silver Spring, MD: U.S. Department of Health and Human Services; 2008 [cited 2014 July 13]; Available from: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM071627. pdf?utm_campaign=Google2utm_source=fdaSearchutm_medium=websiteutm_term=guidance.
  8. Jin SM, Song SO, Jung CH, Chang JS, Suh S, Kang SM, et al. Risk of bladder cancer among patients with diabetes treated with a 15 mg pioglitazone dose in Korea: a multi-center retrospective cohort study. J Korean Med Sci. 2014 Feb;29(2):238-42.
  9. Colhoun HM, Livingstone SJ, Looker HC, Morris AD, Wild SH, Lindsay RS, et al. Hospitalised hip fracture risk with rosiglitazone and pioglitazone use compared with other glucose-lowering drugs. Diabetologia. 2012 Nov;55(11):2929-37.
  10. Bolen S, Wilson L, Vassy J, Feldman L, Yeh J, Marinopoulos S, et al. Comparative Effectiveness and Safety of Oral Diabetes Medications for Adults With Type 2 Diabetes. Comparative Effectiveness Review No. 8. (Prepared by Johns Hopkins Evidence-based Practice Center under Contract No. 290-02-0018). Rockville, MD: Agency for Healthcare Research and Quality. July 2007.
  11. Bennett WL, Wilson LM, Bolen S, Maruthur N, Singh S, Chatterjee R, et al. Oral Diabetes Medications for Adults With Type 2 Diabetes: An Update. Comparative Effectiveness Review. (Prepared by Johns Hopkins Evidence-based Practice Center under Contract No. 290-02-0018.) AHRQ Publication No. 11-EHC038-EF. Rockville, MD: Agency for Healthcare Research and Quality. March 2011.
  12. Bolen S, Feldman L, Vassy J, Wilson L, Yeh HC, Marinopoulos S, et al. Systematic review: comparative effectiveness and safety of oral medications for type 2 diabetes mellitus. Ann Intern Med. 2007 Sep 18;147(6):386-99.
  13. Bennett WL, Maruthur NM, Singh S, Segal JB, Wilson LM, Chatterjee R, et al. Comparative effectiveness and safety of medications for type 2 diabetes: an update including new drugs and 2-drug combinations. Ann Intern Med. 2011 May 3;154(9):602-13.
  14. Lu CJ, Sun Y, Muo CH, Chen RC, Chen PC, Hsu CY. Risk of stroke with thiazolidinediones: a ten-year nationwide population-based cohort study. Cerebrovasc Dis. 2013;36(2):145-51.
  15. Mahaffey KW, Hafley G, Dickerson S, Burns S, Tourt-Uhlig S, White J, et al. Results of a reevaluation of cardiovascular outcomes in the RECORD trial. Am Heart J. 2013 Aug;166(2):240-9 e1.
  16. Mamtani R, Haynes K, Bilker WB, Vaughn DJ, Strom BL, Glanz K, et al. Association between longer therapy with thiazolidinediones and risk of bladder cancer: a cohort study. J Natl Cancer Inst. 2012 Sep 19;104(18):1411-21.
  17. U.S. Food and Drug Administration. FDA Drug Safety Communication: FDA requires removal of some prescribing and dispensing restrictions for rosiglitazone-containing diabetes medicines. 2013 [cited 2014 July 9]; Available from: http://www.fda.gov/Drugs/DrugSafety/ucm376389.htm.
  18. Vasilakou D, Karagiannis T, Athanasiadou E, Mainou M, Liakos A, Bekiari E, et al. Sodium-glucose cotransporter 2 inhibitors for type 2 diabetes: a systematic review and meta-analysis. Ann Intern Med. 2013 Aug 20;159(4):262-74.
  19. Highlights of Prescribing Information - Invokana (canagliflozin) tablets, for oral use. 2013 [cited 2014 July 9]; Available from: https://www.invokanahcp.com/prescribing-information.pdf.
  20. Highlights of Prescribing Information - Farxiga (dapagliflozin) tablets, for oral use. 2014 [cited 2014 July 9]; Available from: http://www1.astrazeneca-us.com/pi/pi_farxiga.pdf#page=1.
  21. Turner LW, Nartey D, Stafford RS, Singh S, Alexander GC. Ambulatory treatment of type 2 diabetes in the U.S., 1997-2012. Diabetes Care. 2014 Apr;37(4):985-92.
  22. Raebel MA, Xu S, Goodrich GK, Schroeder EB, Schmittdiel JA, Segal JB, et al. Initial antihyperglycemic drug therapy among 241 327 adults with newly identified diabetes from 2005 through 2010: a surveillance, prevention, and management of diabetes mellitus (SUPREME-DM) study. Ann Pharmacother. 2013 Oct;47(10):1280-91.
  23. Stone NJ, Robinson JG, Lichtenstein AH, Bairey Merz CN, Blum CB, Eckel RH, et al. 2013 ACC/AHA Guideline on the Treatment of Blood Cholesterol to Reduce Atherosclerotic Cardiovascular Risk in Adults: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014 Jun 24;129(25 Suppl 2):S1-S45.
  24. Tsertsvadze A, Maglione M, Chou R, Garritty C, Coleman C, Lux L, et al. Updating comparative effectiveness reviews: current efforts in AHRQ's Effective Health Care Program. J Clin Epidemiol. 2011 Nov;64(11):1208-15.
  25. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996 Feb;17(1):1-12.
  26. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998 Jun;52(6):377-84.
  27. Institute of Medicine. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press; 2011 [cited 2014 July 9]; Available from: books.nap.edu/openbook.php?record_id=13059&page=81.
  28. Higgins JPT, S. G. Cochrane handbook for systemic reviews of interventions Version 5.1.0. 2011 [cited Oxford, England]; Available from: http://handbook.cochrane.org/.
  29. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003 Sep 6;327(7414):557-60.
  30. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986 Sep;7(3):177-88.
  31. Cornell JE, Mulrow CD, Localio R, Stack CB, Meibohm AR, Guallar E, et al. Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change. Ann Intern Med. 2014 Feb 18;160(4):267-70.
  32. Balshem H, Stevens A, Ansari M, Norris S, Kansagara D, Shamliyan T, et al. Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville MD2008.
  33. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994 Dec;50(4):1088-101.
  34. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997 Sep 13;315(7109):629-34.
  35. Duval S, Tweedie R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000 Jun;56(2):455-63.
  36. Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions--agency for healthcare research and quality and the effective health-care program. J Clin Epidemiol. 2010 May;63(5):513-23.
  37. Saydah SH, Eberhardt MS, Loria CM, Brancati FL. Age and the burden of death attributable to diabetes in the United States. Am J Epidemiol. 2002 Oct 15;156(8):714-9.

Definition of Terms

DPP-4 inhibitor = dipeptidyl peptidase-4 inhibitor
FDA = U.S. Food and Drug Administration
GLP-1 agonist = glucagon-like peptide-1 receptor agonist
HbA1c = hemoglobin A1c
RCT = randomized controlled trial
SGLT-2 inhibitor = sodium-glucose co-transporter 2 inhibitor

Summary of Protocol Amendments

If we need to amend this protocol, we will give the date of each amendment, describe the change and give the rationale in this section. Changes will not be incorporated into the protocol.

Review of Key Questions

AHRQ posted the key questions on the Effective Health Care Website for public comment. The EPC refined and finalized the key questions after review of the public comments, and input from Key Informants and the Technical Expert Panel (TEP). This input is intended to ensure that the key questions are specific and relevant.

Technical Experts

Technical experts constitute a multi-disciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, or outcomes and identify particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicting opinions are common and perceived as health scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design, and methodological approaches do not necessarily represent the views of individual technical and content experts. Technical experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical experts do not do analysis of any kind nor do they contribute to the writing of the report. They have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.

Technical experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as technical experts and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer Reviewers

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. The EPC considers all peer review comments on the draft report in preparation of the final report. Peer reviewers do not participate in writing or editing of the final report or other products. The final report does not necessarily represent the views of individual reviewers. The EPC will complete a disposition of all peer review comments. The disposition of comments for systematic reviews and technical briefs will be published three months after the publication of the evidence report.

Potential peer reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited peer reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

EPC Team Disclosures

EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Related financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify EPC core team investigators.

Role of the Funder

This project was funded under Contract No. 290-201-20007-I from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

Appendix

Search Strategy

PubMed
Type of Search Terms Returns
Orignal search (“diabetes mellitus, type 2”[mh] or (diabet*[tiab] and (“non-insulin dependent”[tiab] or type-2[tiab] or “type II”[tiab] or “type 2”[tiab]))) AND (“metformin”[mh] or “thiazolidinediones”[mh] or “glipizide”[mh] or “glyburide”[mh] or “Dipeptidyl-Peptidase IV Inhibitors”[mh] or “Glucagon-Like Peptide 1”[mh] or biguanide*[tiab] or metformin[tiab] or thiazolidinedione*[tiab] or pioglitazone[tiab] or rosiglitazone[tiab] or sulfonylurea*[tiab] or sulphonylurea*[tiab] or glipizide[tiab] or glyburide[tiab] or glimepiride[tiab] or glibenclamide[tiab] or “insulin secretagogues”[tiab] or sitagliptin*[tiab] or saxagliptin*[tiab] or dpp-4[tiab] or dpp-iv[tiab] or liraglutide[tiab] or exenatide[tiab]) NOT (animal[mh] NOT human[mh]) NOT (letter[pt] or comment[pt] or editorial[pt]) AND (("2009/04/01"[edat] : "2014/07/11"[edat])) 5397
Expanded search (“diabetes mellitus, type 2”[mh] or (diabet*[tiab] and (“non-insulin dependent”[tiab] or type-2[tiab] or “type II”[tiab] or “type 2”[tiab]))) AND (linagliptin*[tiab] or alogliptin*[tiab] or albiglutide*[tiab] or dulaglutide*[tiab] or "sodium-glucose co-transporter 2 inhibitors”[tiab] or “sodium-glucose co-transporter 2 inhibitor” [tiab] or “SGLT-2” [tiab] or “canagliflozin”[tiab] or “dapagliflozin”[tiab] or empagliflozin*[tiab]) NOT (animal[mh] NOT human[mh]) NOT (letter[pt] or comment[pt] or editorial[pt]) 545
EMBASE Strategy
Type of Search Terms Returns
Original search ('non insulin dependent diabetes mellitus'/exp OR 'non insulin dependent diabetes mellitus' or (diabet*:ti,ab and (‘non-insulin dependent’:ti,ab or type-2:ti,ab or ‘type II’:ti,ab or ‘type 2’:ti,ab))) AND ('thiazolidinedione'/exp or 'rosiglitazone'/exp or 'pioglitazone'/exp or 'glipizide'/exp or 'glyburide'/exp or ‘glimepiride’/exp or 'metformin'/exp or ‘sitagliptin’/exp or thiazolidinedione*:ti,ab or pioglitazone:ti,ab or rosiglitazone:ti,ab or sulfonylurea*:ti,ab or sulphonylurea*:ti,ab or glipizide:ti,ab or glyburide:ti,ab or glimepiride:ti,ab or glibenclamide:ti,ab or biguanide*:ti,ab or metformin:ti,ab or ‘insulin secretagogues’:ti,ab or ‘Dipeptidyl-Peptidase IV Inhibitor’/de or saxagliptin/exp or saxagliptin*:ti,ab or sitagliptin/exp or sitagliptin*:ti,ab or dpp-4:ti,ab or dpp-iv:ti,ab or exenatide/exp or exenatide:ti,ab or liraglutide/exp or liraglutide:ti,ab) NOT ([animals]/lim NOT [humans]/lim) NOT (letter:it or comment:it or editorial:it) AND [2009-2014]/py 15183
Expanded search ('non insulin dependent diabetes mellitus'/exp OR 'non insulin dependent diabetes mellitus' or (diabet*:ti,ab and (‘non-insulin dependent’:ti,ab or type-2:ti,ab or ‘type II’:ti,ab or ‘type 2’:ti,ab))) AND (linagliptin/exp or linagliptin*:ti,ab or alogliptin/exp or alogliptin*:ti,ab or albiglutide/exp or albiglutide*:ti,ab or dulaglutide/exp or dulaglutide*:ti,ab or ‘sodium glucose cotransporter 2 inhibitor’/de or ‘sodium-glucose co-transporter 2 inhibitors’:ti,ab or ‘sodium-glucose co-transporter 2 inhibitor’:ti,ab or ‘sodium glucose cotransporter 2 inhibitors’:ti,ab or ‘sodium glucose cotransporter 2 inhibitor’:ti,ab or ‘SGLT-2”:ti,ab or canagliflozin/exp or canagliflozin:ti,ab or dapagliflozin/exp or dapagliflozin:ti,ab or empagliflozin/exp or empagliflozin*:ti,ab) NOT ([animals]/lim NOT [humans]/lim) NOT (letter:it or comment:it or editorial:it) 1979
The Cochrane Central Register of Controlled Trials (CENTRAL)
Type of Search Terms Returns
Original search ((diabetes near type-2):ti,ab,kw or (diabet*:ti,ab,kw and (“non-insulin dependent”:ti,ab,kw or type-2:ti,ab,kw or “type II”:ti,ab,kw or “type 2”:ti,ab,kw))) AND (thiazolidinedione*:ti,ab,kw or pioglitazone:ti,ab,kw or rosiglitazone:ti,ab,kw or sulfonylurea*:ti,ab,kw or sulphonylurea*:ti,ab,kw or glipizide:ti,ab,kw or glyburide:ti,ab,kw or glimepiride:ti,ab,kw or glibenclamide:ti,ab,kw or biguanide*:ti,ab,kw or metformin:ti,ab,kw or “insulin secretagogues”:ti,ab,kw or “Dipeptidyl-Peptidase IV Inhibitors”:ti,ab,kw or saxagliptin*:ti,ab,kw or sitagliptin*:ti,ab,kw or liraglutide:ti,ab,kw or exenatide:ti,ab,kw)
Publication Year from 2009 to 2014
1535
Expanded search ((diabetes near type-2):ti,ab,kw or (diabet*:ti,ab,kw and (“non-insulin dependent”:ti,ab,kw or type-2:ti,ab,kw or “type II”:ti,ab,kw or “type 2”:ti,ab,kw))) AND (linagliptin*:ti,ab,kw or alogliptin*:ti,ab,kw or albiglutide*:ti,ab,kw or dulaglutide*:ti,ab,kw or ‘sodium-glucose co-transporter 2 inhibitors’:ti,ab,kw or ‘sodium-glucose co-transporter 2 inhibitor’:ti,ab,kw or ‘sodium glucose cotransporter 2 inhibitors’:ti,ab,kw or ‘sodium glucose cotransporter 2 inhibitor’:ti,ab,kw or ‘SGLT-2’:ti,ab,kw or canagliflozin:ti,ab,kw or dapagliflozin:ti,ab,kw or empagliflozin*:ti,ab,kw) 216