Skip Navigation
AHRQ--Agency for Healthcare Research and Quality: Advancing Excellence in Health Care
  • Home
  • Search for Research Summaries, Reviews, and Reports
 
 

EHC Component

  • EPC Project

Related Products for this Topic



Research Protocol – Feb. 9, 2017

Obesity Prevention and Control

Formats

Table of Contents

I. Background and Objectives for the Systematic Review

The prevalence of overweight and obesity in adults and children has increased dramatically,1,2 with recent estimates that 17% of U.S. children and adolescents are obese.3,4 The reasons for the obesity epidemic are complex, involving interrelated, multilevel and dynamic forces in many societal sectors that influence patterns of eating and physical activity.5 Furthermore, racial and socioeconomic disparities in obesity may be worsening.6 Most obesity prevention and weight loss interventions focus on individual-level behavioral changes in dietary habits and physical activity.7 To eliminate health disparities and address the social determinants of obesity, we need effective population-level programs and government policies. A wide range of programs and policies (see Appendix A for definitions of terms) have been implemented in diverse settings, ranging from limiting new fast food chains in Los Angeles,8 reducing access to sugar sweetened beverages in schools,9 occupational programs to reduce sedentary time,10 changes in the built environment,11 and population-oriented interventions in health system settings.12,13

Although we are beginning to understand the role of social determinants in the obesity epidemic, programs and policies targeting social determinants have been difficult to implement and evaluate. Innovative and efficient solutions are needed that take advantage of existing data sources and enable researchers to design larger scale studies linked with clinical data, such as electronic medical records (EMRs). One example of an opportunity to link data to enable an evaluation of program effectiveness is the Park Rx program, which was highlighted by the Office of the National Coordinator for Health Information Technology (ONC). The Park Rx program is a collaborative program between the National Park Services' (NPS) Healthy Parks Healthy People Initiative and pediatric providers in Washington, DC. The program enables providers to search the NPS database while in the EMR and then "prescribe a nearby park" for high risk, sedentary patients. This program illustrates how the linkage of data sources enables the implementation of a program. Despite the program's potential for evaluation using these linked data sources, to date, no studies have evaluated its effect on outcomes such as physical activity or obesity.14 The Moving to Opportunity (MTO) study represents the largest and most impactful natural experiment, implemented by the Department of Housing and Urban Development (HUD). Families living in low-income or subsidized housing participated in a randomized lottery to receive a voucher that enabled them to move to a higher income neighborhood. Compared with the control group, the group with the voucher had a lower prevalence of extreme obesity and diabetes.15 The MTO study provides a model for evaluation that combined HUD data with in-person data collection for key outcomes thereby providing an infrastructure for linkages of data.

Unanswered questions remain regarding the effectiveness of obesity prevention and control policies and programs, best practices for evaluations that link existing data to enhance efficiency and rigor, and the strengths and limitations of various approaches. We plan to conduct a systematic review to improve understanding of the population-based data sources that have been used to conduct and evaluate population-oriented programs and policies focused on obesity prevention and control, and to identify methodological/analytic advances that would help strengthen efforts to evaluate the effect of programs and policies on obesity prevention and control. This project will help inform the Pathways to Prevention Workshop: Methods for Evaluating Natural Experiments in Obesity.

Few reviews have identified the effect of natural experiments like policies and programs on obesity outcomes. A recent systematic review examined the use of natural experiments to evaluate the efficacy of policy and built environment changes on body mass index (BMI), diet, or physical activity.16 The review identified 37 studies: 18 studies evaluated impacts on nutrition/diet, 17 on physical activity, and 3 on BMI. Despite the review's focus on built-environment policies (not programs and not nutritional policies like sugar sweetened beverage taxes), as well as population-level (not individual level) interventions, they identified a broad range of studies that addressed physical activity and nutritional outcomes relevant to obesity prevention and control.16 Another systematic review and meta-analysis identified 19 studies focused on the effect of menu labeling on nutritional outcomes.17 None of these studies reported BMI/weight or total daily caloric intake as outcomes. Most reported the behavioral outcome of change in caloric intake during a single meal, which was the outcome used in the meta-analysis.17

As demonstrated in the review,16 few studies included BMI or weight as a primary outcome. To address the limitations of the evidence base, this review will include both BMI and weight outcomes, as well as the more proximal, weight related behavioral outcomes. We recognize that defining the behavioral outcomes of physical activity and dietary change will be challenging because many self-reported instruments have been developed and reported in the literature,18 with varying degrees of comprehensiveness, validity and reliability. We will include studies that used validated instruments to capture multiple domains of individual dietary and physical activity behaviors (versus food environment), as well as specific behaviors associated with obesity, namely intake of sugar sweetened beverages, fruits/vegetables, and fast foods. Our reference list of validated instruments will be guided by The National Collaborative on Childhood Obesity Research (NCCOR) Measures Registry19 as well as the literature, as new measures are always being tested.20 For self-reported measures, we will capture which instruments were used and the validation reference, and when objective measures (e.g. activity trackers) were used, we will describe these methods.21 Because the field of obesity control and prevention has advanced to having a large number of natural experiment studies that address weight, dietary and physical activity behaviors,15,22-30 we will exclude studies that only report environmental or structural outcomes, such as measures of access to healthy food (e.g., stocking of shelves) or "walkability." When articles report neighborhood outcomes that meet the inclusion criteria, we will record such outcomes (e.g., urban renewal) in addition to the primary outcomes.

II. The Key Questions

This project topic was nominated by the National Institutes of Health (NIH) P2P program. The Key Questions (KQ) were derived from the P2P topic submission form and approved by the NIH working group associated with this topic, but did not undergo external review by key informants and were not released for public comment. Detailed PICOTS descriptions are available in Table 1, which describes the inclusion criteria for studies that may provide or identify data sources and data systems for obesity prevention and control interventions. See Appendix A for definitions of terms.

KQ1

What population-based data sources have been used in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

KQ2

What methods have been used to link different population-based data sources?

KQ3

What obesity measures, dietary and physical behaviors, and other outcomes have been assessed in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control?

KQ4

Which experimental and non-experimental methods have been used in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

KQ5

What are the risks of bias in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

KQ6

What methodological/analytic advances (e.g., data system features, approaches to linking data sources, or analytic methods) would help to strengthen efforts to estimate the effect of programs, policies or built environment changes on obesity prevention and control?

III. Analytic Framework

The following figure depicts the key questions within the context of the PICOTS described in the previous section (see Table 1 and Figure 1). The figure illustrates how data sources, collected from the general population (no age restrictions), from programs and policies focused on obesity prevention and control, including individual/family, local/neighborhood, City, State, and National level, have been evaluated and can be linked. Additionally, it illustrates what study designs and analytic methods have been used to evaluate the effect of these programs and policies, and points to the need for future research.

Figure 1: Preliminary analytic framework for obesity prevention and control

Figure 1. Figure 1 illustrates the preliminary analytic framework used for this project. Population is defined as all ages, general population with the following subgroups investigated: age, gender, race/ethnicity, socioeconomic status, urban/rural residency, type 2 diabetes mellitus, pre-diabetes, and elevated cardiovascular risk. Interventions are defined as policies, programs, and built environmental changes. Key questions 1 and 2 (data sources and linkage methods) and key questions 4 and 5 (experimental and non-experimental methods and risk of bias link from populations and interventions to key question 3, outcomes. Key question 3 details the outcomes: obesity-related health behaviors (required outcome), individual dietary and physical activity behaviors; obesity measures (required outcome), body weight and body mass index; and co-outcomes, e.g., purchases and commuting behavior. All of the above elements link to key question 6 which relates to gaps in the literature.

KQ=Key Question

Table 1: PICOTS framework for all key questions
PICOTS Inclusions Exclusions
Population(s)
  • All ages, general population; and sub-populations of obese and overweight individuals
  • Consider stratifying by subgroups: age, race/ethnicity, gender, rural/urban, or clinical conditions (e.g., type 2 diabetes, pre-diabetes, elevated cardiovascular risk).
  • Studies within specific clinical populations only, other than obese/overweight populations (e.g., severe mental illness, people with genetic predispositions for obesity
  • Studies with less than 100 subjects to achieve the focus on population-based programs and policies.
Intervention(s) U.S. and non-U.S. policies, programs and built environment changes targeting a population (See Appendix A, definition of terms). Programs or policies targeting the individual level (not a system or population level), such as Weight Watchers; If the program or policy includes multiple levels of intervention (e.g., population level and individual level), it would be considered for inclusion).
Comparison(s)
  • Studies with a clearly defined comparison group either prior to the policy, or a defined group without exposure to the policy or program. Comparison group does not need to be concurrent.
  • Observational or cohort studies that use regression models to assess the association of a policy or program on an exposed versus unexposed group.
Studies without a comparison or unexposed group.
Outcome(s) Outcomes of interest (need one or more of these: See Appendix B, list of outcome measures):
  • Body weight
  • Body mass index
  • Individual physical activity behavior assessed using a validated questionnaire that assesses both quantity and type of activity, or measures physical activity objectively (e.g. step counts).
  • Individual dietary behavior assessed using a validated questionnaire, measuring one or more of the following:
  • Total daily caloric intake,
  • Specific dietary macronutrients related to obesity: vegetable, fruit, or fiber intake.
  • Specific eating behaviors associated with obesity: sugar sweetened beverage intake, or fast food frequency.
Co-outcomes to be considered ONLY if at least one required outcome is also reported: Food-environment, physical activity environment, commuting behavior, purchasing behavior, or urban renewal (See Appendix B; list of key measures).
  • Studies without reference for validation or use of a validated instrument (see Appendix B for list of commonly used validated instruments) to measure diet or physical activity.
  • Studies measuring a specific macronutrient not described above (e.g., protein intake) or specific vitamin or mineral (e.g., calcium intake).
  • Studies assessing other dietary or physical activity behaviors not described above (e.g. overeating).
  • Studies without an English-language data dictionary (such studies will be identified, but we will not attempt to extract information about the data in the studies).
Timing Programs and policies enacted or implemented in 2000 or later. The U.S. Surgeon General's Call To Action To Prevent and Decrease Overweight and Obesity was published in 2001 and marked a turning point to raise public health awareness about obesity.
Setting U.S. and non-U.S. setting at all levels (e.g. national, state, community/neighborhood). Studies in specific settings that would not be generalizable to a free-living population or community (e.g. prison, nursing home). Note: we will include those that are deemed generalizable (e.g., college campuses, employers).

IV. Methods

Due to the complexity of this systematic review, which focuses on data systems and study design and analytic methods used in obesity prevention and control programs and policies, the review process includes a number of methods and additional data abstraction and synthesis steps for each KQ.

Search Methodologies

We will conduct a systematic review of the published literature to identify studies focusing on programs and policies implemented for obesity prevention and control. The results of this phase will inform Key Questions 1 through 5.

To identify unpublished (gray) literature on policy evaluations, we will search the websites of selected governmental agencies and organizations for unpublished literature.

Systematic Literature Review

Search Engines (Literature Search and Policy Search Strategies)

We will search PubMed, CINAHL, PsycINFO, and EconLit. We will explore the availability and usefulness of additional sources that have articles on the built environment and transportation. A search strategy will be developed for PubMed, and will be used as a guide to develop search strategies in the other search engines (see Appendix C). Index articles will be used during the search development to ensure that an appropriate set of studies is captured by the search strategy.15,22-30

We will search for articles authored in English and published between 2000 to present. The search will be supplemented with a hand search to identify references in other published relevant systematic and narrative reviews. Articles will be selected through independent screening by two screeners based on the inclusion and exclusion criteria described in PICOTS Table 1.

Grey Literature Search to Identify Unpublished Evaluations

We have compiled a list of agencies and organizations that conduct evaluations of policies and programs aimed at preventing or controlling obesity (Appendix D). We will search the websites of these organizations to identify white papers or unpublished evaluations.

Search Criteria (Inclusion and Exclusion)

The principal inclusion and exclusion criteria for the systematic literature review will be derived from the identified PICOTS framework (see Table 1). We will search for all studies on programs or policies targeting obesity prevention and control in people of any age to identify potential data sources. We will include programs or policies implemented in any country. We will exclude smaller studies with less than 100 subjects because of the review's goal of identifying population-based studies.

We created a list of commonly used validated-instruments to assess obesity-related individual-level behavioral outcomes in Appendix B. This list will be a starting point to determine inclusion of the articles that include dietary or physical activity behavioral outcomes, but no weight or BMI measures. When we identify instruments not listed in Appendix B, we will assess whether the study provides a reference for the validation of the instrument, and if so, we will add the instrument to our list and include the article. If no validated instrument is used, we will exclude the study from full data abstraction. We will maintain a list of these excluded articles that used non-validated instruments for diet and physical activity assessments.

Studies will not be included if published prior to 2000. In 2001, the U.S. Surgeon General released a report entitled, "The Surgeon General's Call To Action To Prevent and Decrease Overweight and Obesity," to call attention to the major public health problem of obesity.31 The publication of this report marked a shift toward directing public health funding and policies toward the prevention and control of obesity, and sparked new research in this area.

We will include studies that report on one of the following outcomes of interest: obesity measures (either body weight or BMI), and obesity-related individual health behaviors (dietary and physical activity behaviors). Studies will be excluded if they do not report any of these outcomes (see Table 1 and Appendix B).

Data Abstraction and Data Management

We will use Distiller SR (Evidence Partners, Ottawa, Canada) to manage the screening and abstraction process. Distiller SR is a web-based data management program that manages all levels of the review process. Data from applicable articles will be abstracted and uploaded to the Systematic Review Data Repository TM (SRDR), a web-based data repository. This source serves as both an archive and a data abstraction tool. Data will be exported from SRDR into a project-specific database to serve as archived or backup copies and to create detailed evidence and summary tables.

Data Abstraction Overview and Process

Two trained research assistants will abstract data. We will abstract data about the study characteristics (e.g., year of publication, country), study design, participant and population characteristics, details about the policy and programs (year of enactment, type of legislation/policy, location of policy [e.g. state, country, locality], original goal of policy/program [diet, physical activity, both, other]) data sources and linkages, and analytic methods. See below for additional data abstraction elements by KQ.

We will assess how each of the outcomes was measured and the validated reference for instruments not listed in Appendix B (see Appendix B for list of commonly used dietary and physical activity instruments).

After data have been abstracted, an independent data abstraction expert will review a random sample for quality assurance. Inconsistencies in data abstraction will be resolved by a consensus approach involving the research assistants and the expert reviewer. If consensus is not attainable for a specific case, it will be discussed among the rest of the review team and resolved by a majority vote. The same process will be used for all data abstraction activities throughout this review project.

Data abstraction by Key Question

Key Question 1

What population-based data sources have been used in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

We will abstract and create a list of all data sources reported in included studies. We will first create a list of data sources by study.

We will then create a complete list of the identified data sources (U.S. and non-U.S.), with all duplicate data sources removed, and provide a count of the number of studies that used each data source. We will use this list of data sources to apply a set of criteria (Table 2) to determine if a data source meets the definition of a population-based data system. These criteria were developed for a previous project our team conducted for the P2P program focused on data sources used to evaluate suicide prevention programs.32 We have decided to apply these criteria to data sources identified in KQ1 in order to inform obesity researchers about which data sources have been used, their key characteristics that make them useful for future research, and whether they meet the highest standard of being considered a data system (i.e., maintained, able to be acquired and accessed, and having variables of interest).

Table 2. Criteria to determine if a data source meets the criteria of a data system related to obesity
# Criteria
1 Data source is still in existence
2 Data is available and accessible in digital format (e.g., datasets are downloadable from a current website)
3 Data is sharable and can be acquired by others for research purposes (e.g., it has a public or transferable license that allows the data to be used for research purposes)
4 Data system collects/contains at least one of the outcomes of interest

For U.S. data systems, we will further code these population based data systems according to data quality issues when they can be assessed from the article. We have limited the data quality assessments to U.S. data systems to ensure consistent access and understanding of language across the quality systems. The coding and classification schema for the quality of data systems was adapted and modified from a framework previously developed to evaluate the quality of community-based data sources.33 The items in the coding schema can be found in Appendix E. These data quality issues include: data relatedness (related to obesity) and availability; data granularity (population or individual level); denominator coverage; data sampling; data scalability; data interoperability; data governance; data uses and functions; and data linkage mechanisms (see KQ2). The quality assessment coding process will focus on the information that can be abstracted from the articles. We will attempt to locate and access publicly available data dictionaries after we have obtained as much information as possible from the articles, when it is feasible and has high potential to enhance our understanding of the quality of the data source. Downloading and analyzing the data systems will be out of the scope of this review (i.e., conducting statistical analyses of the data systems to compute data quality measures are out of scope).

Similar to the data system lists we created for the Suicide Prevention report,32 we will provide a list of the highest quality data systems that are most usable for obesity and public health researchers.

Key Question 2

What methods have been used to link different population-based data sources?

For each U.S. data system identified in KQ1, we will assess whether and how the data systems have been linked together using information obtained from the article, and as described above, using publicly available data dictionaries from U.S. data sources. The coding schema (Appendix E) includes a series of coding items to identify how the identified studies linked data sources together. The main categories of 'data linkage' will include the following items (preliminary):

  • Data Linkage Denominator
    • Individual level: identifiable, anonymized
    • Aggregate level: clinical grouping, employer-based, other groupings
    • Ecological: geographical, temporal
  • Data Linkage Synchronization
    • Manual synchronization: ad-hoc, periodical, semi-automated
    • Real-time synchronization: one way, two ways (exchange)
  • Data Linkage Technical Method
    • Database level integration
    • Web service integration
    • Use of other Application Programing Interfaces (APIs)
  • Data Linkage Interoperability Specs
    • Common terminologies: International Classification of Diseases (ICD), Systematized Nomenclature of Medicine (SNOMED), Current Procedural Technology (CPT), RxNORM, National Drug Code (NDC) and others
    • Common exchange methods: Health Level Seven International (HL7), Consolidated Clinical Document Architecture (CCDA) and other methods
  • Data Linkage Purpose
    • Research
    • Clinical care
    • Public health
  • Other operational mandates/requirements

Key Question 3

What obesity measures, dietary and physical behaviors, and other outcomes have been assessed in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control?

We will abstract detailed information about what obesity measures, obesity-related behavioral outcomes, and other non-health related outcomes are reported in the papers. We will describe the data source (identified in KQ1) for each of the variables and instruments (e.g. pedometer, surveys) and the validation references. We will use a list of validated measures as illustrated in Appendix B.

To determine the categories of variables related to obesity and co-outcomes in each data system, we will take the following approach:

  1. Abstract a list of obesity measures, behavioral outcomes, and co-outcomes from each study.
    1. We will develop categories for outcomes using the NCCOR categories.
      1. Body weight
      2. BMI
      3. Individual dietary behavior
      4. Individual physical activity behavior
      5. Other outcomes:
        1. Food environment
        2. Physical activity environment
        3. Other: e.g. housing, economic
  2. For the obesity measures and behavioral outcomes, we will additionally abstract the measures or instruments used to assess the outcomes, whether assessed using self-report, objectively, or using clinical measures, and the data source(s) for each outcome variable (see KQ1).
    1. We will describe the reported units (e.g., change in caloric intake).
    2. We will describe the effect of the policy/program on each outcome as positive, neutral, or negative.
    3. When available, we will abstract the effect size and units, measure of variability, and 95% confidence interval (CI).
  3. For the co- outcomes (e.g. commuting behaviors, purchasing behaviors), we will qualitatively assess which ones have been reported and the effect on these outcomes as positive, neutral, or negative.

Key Question 4

Which experimental and non-experimental methods have been used in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

Key Question 5

What are the risks of bias in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

To address KQ4-5, we will abstract details to describe the study design and analytic approaches: both experimental and non-experimental. Table 3 shows the most commonly encountered types of non-experimental study designs and specific bias concerns. In addition to abstracting information about key design aspects, we will abstract details about sample selection, approaches to address missing data, examine interactions, and test differential effects in sub-groups.

The goal of this project is to describe existing research regarding policies, programs and built environment changes that may affect population-level rates of obesity. Ideally, there would be a large body of evidence from experimental studies. Well-executed experimental studies provide unbiased estimates of the effect of the intervention on the outcome because the intervention assignment is under the control of the investigator and randomly assigned to some study participants and withheld from others. This results in exposed and unexposed (or treatment and control) groups that are nearly identical on all factors except for the factor under study; because the groups being compared are only randomly different from one another, in expectation there is no bias due to confounding. Of course, experiments are also not without their complications, and analyses of experiments often have to deal with missing data, attrition, non-adherence with the assigned intervention group, or potential lack of representativeness relative to the target population of ultimate interest.

In addition, experimental studies are not always feasible or ethical. We expect to find many non-experimental studies (also referred to as "quasi-experimental," "natural experiments," or "observational"). In such cases, analytic methods can be used to mimic experimental study designs and provide valid estimates of the effect of the policy or program in question.34 Common analytic methods include instrumental variables, regression discontinuity, propensity score, and interrupted time series methods (Table 3). These methods can provide valid estimates of the effect of the policy or program under study, but each has shortcomings related to its departure from a randomized design and each relies on its own (untestable) assumptions.

To address KQ5, our primary tool for assessing risk of bias and threats to validity (especially selection, confounding, and information biases) will be the Effective Public Health Practice Project (EPHPP) tool for quality assessment tool for quantitative studies.35 This tool was developed for use in public health and has fair inter-rater agreement36 The tool will enable us to assess selection bias, including treatment selection bias or endogeneity, a term used by economists for treatment selection bias. This tool yields an overall classification of risk of bias, and has questions that address the following domains:

  1. Selection bias
  2. Study design
  3. Confounders
  4. Blinding
  5. Data collection methods
  6. Withdrawals and drop-outs

We will also abstract detailed information on confounders and types of adjustment. In addition to using the EPHPP tool to provide an overall risk of bias assessment, we will assess risk of bias concerns unique to each of the study designs and analytic methods employed.

Below we describe specific analytic methods and context for the non-experimental studies we anticipate identifying in the literature search:

  • Instrumental variable methods are sometimes referred to as "natural experiments" or "randomized encouragement designs". In cases where the intervention under study is not randomized, investigators attempt to identify an "instrument" that is related to receipt of the intervention and randomly distributed (or at least hypothetically randomly assigned), but is not related to the outcome.37 Instrumental variable analyses rely on finding a good instrument. Bias will be introduced if the instrument shares a common cause with the outcome, directly influences the outcome (not just through the treatment of interest), or the instrument is not randomly distributed. We will assess bias specific to instrumental variable methods.
  • Regression discontinuity methods take advantage of existing rules or cutoff points that determine receipt of the intervention of interest (e.g., individuals above some BMI threshold receive an intervention; those below that threshold do not). Persons just above or just below the cutoff are assumed to be very similar, so comparing these groups allows for a valid estimate of the effect of the intervention.38 Bias may be introduced if the cutoff for assigning the intervention is unclear or can be manipulated, and regression discontinuity methods also rely on functional form assumptions (relating the assignment variable to the outcome) that need to be assessed. We will assess bias specific to regression discontinuity methods.
  • Propensity score methods. As mentioned above, experimental designs are able to achieve balance between treatment and control groups on all covariates, both observed and unobserved. Propensity score methods attempt to achieve this covariate balance, at least among the observed covariates. Comparisons can then be made between those who are similar on important factors to isolate the effect of the exposure.39 Propensity score methods are only as strong as the data available: confounders that were unmeasured or not included in the development of propensity score may bias effect estimates. We will assess bias specific to propensity score methods.
  • Interrupted time-series methods aim to model changes in the outcome before and after the intervention occurs. The stronger interrupted time-series designs also incorporate a comparison group that did not experience the intervention of interest at any time point; this enables better modeling of trends over time in the absence of the intervention. Interrupted time-series methods make assumptions about how the outcome of interest would have changed over time in the absence of the intervention. For example, in a simple interrupted time-series the assumption is that the trend in outcome would have continued in the same way as before the intervention time point. In a comparative interrupted time-series model, the assumption is that the difference in trends between the exposed and comparison groups would have continued in the same way after the intervention time point. Another assumption is that there is no other change at the time of the intervention that may affect the outcome, as well as a reliance on the model forms modeling the outcome over time.40 We will assess bias specific to interrupted time series.
Table 3. Analytic methods that can be used in non-experimental studies, and specific bias concerns
Analytic Method Other names/ subtypes Brief Description Key assumptions Specific Bias Concerns
Instrumental variables Randomized encouragement design; Mendelian randomization, preference-based instruments Involves identifying an “instrument” that influences receipt of the program or policy of actual interest but does not directly influence the outcome. The instrument also needs to be (at least hypothetically) randomized.
  • Instrument is associated with receipt of the program or policy of interest (testable)
  • Exclusion restriction: no direct effect of the instrument on the outcome; Instrument associated with outcome only through exposure;
  • Instrument randomized (no common cause with outcome)
  • Violation of exclusion restriction
  • Non-randomization of instrument
Regression discontinuity None known Use a cut-off/ rule to assign intervention status; analysis compares those just above the cutoff to those just below the cutoff to estimate effect of the intervention
  • Smooth model underlying the outcome in the absence of the intervention
  • Intervention must have been assigned using the cut-off/rule
  • Unclear demarcation at cut-off
  • Manipulation of cut-off variable near the cut-off
  • Incorrect model specification above or below the cut-off
Comparison group designs/ Propensity score methods None known Model factors associated with exposure status to compare unexposed and exposed with similar values of covariates No unmeasured confounders
  • Lack of balance in covariates
  • Unmeasured confounders
Interrupted time series
  • Difference in differences
  • Comparative interrupted time series
  • Model change over time before and after a policy intervention
  • Design stronger if also includes data on an untreated comparison group
  • Unobserved differences between groups are fixed
  • No other “interruption” at the time of the policy change
  • Groups would have identical changes in trends in absence of intervention
  • Group composition changing over time
  • Comparison group not providing accurate estimate of what would have happened in absence of intervention (e.g., if trends over time differ in unobserved ways between comparison and intervention sites)
  • Change in measurement over time

Key Question 6

What methodological/analytic advances (e.g., data system features, approaches to linking data sources, or analytic methods) would help to strengthen efforts to estimate the effect of programs, policies or built environment changes on obesity prevention and control?

The goal is to identify research gaps in studies designed to evaluate obesity prevention and control. Research gaps are areas where more research is needed because of the importance of the issue and because no or few high-quality studies been conducted in this area. We plan to use the draft evidence report as a starting point for this key question, and will follow a process similar to that laid out by Saldanha et al., 2012.41

  1. We will abstract all the research gaps (by PICOT) highlighted in the evidence report and its included studies. Gaps are defined as deficiencies in the literature (research gaps), and pieces of information necessary for decision making that are unavailable (evidence gaps). Gaps will be abstracted by two independent reviewers during the data abstraction phase of the project. These gaps will be aligned with the KQs by PICOTS.
  2. The TEP and stakeholders will be provided a copy of the draft report for review.
  3. The TEP and stakeholders will be asked to review the research gaps identified during data abstraction.
  4. The TEP and stakeholders will be asked to discuss the gaps presented to them, and identify additional gaps if any are detected.
    1. Using an on-line tool, such as Qualtrics, the Technical Expert Panel (TEP) and internal advisors will be asked to provide comment on: 1) the gaps identified during data abstraction; 2) the benefits of addressing the gaps in future research; 3) the likelihood of being able to address the gaps; 4) additional gaps not identified by the reviewers.
    2. In a conference call, we will first describe the gaps from the review and feedback. We will then ask the TEP and stakeholders to give additional feedback on the identified gaps, as well as identify any additional gaps. Specifically, we will discuss the following questions:
      1. KQ1: What are the important gaps in existing population-based data sources used to estimate the effect of programs, policies, and built environment changes on obesity control? Did we miss any data sources?
      2. KQ2: What are the important gaps related to linking population-based data-sources?
      3. KQ3: What are the important gaps related to obesity and obesity-related behavioral outcomes in population-based data sources? What are the important gaps related to other outcomes in the above population-based data sources?
      4. KQ4: What are the important gaps related to the methods used to estimate the effect of programs, policies, and built environment changes on obesity prevention and control?
      5. KQ5: What are the important gaps related to risks of bias in population-level data sources used to estimate the effects of programs, policies, and built environment changes on obesity prevention and control?
      6. KQ6: Are there additional methodologic or analytic advances that are not addressed in the current literature base that could help strengthen the efforts to estimate the effects of programs, policies, and built environment changes on obesity prevention and control?
  5. During the conference call with our TEP and stakeholders, we will build consensus around the most important gaps to move the field forward.
  6. After comparing the important aspects identified by the TEP and stakeholders to the data we found, the team will summarize the gaps identified, and propose means to address these gaps in our report and final presentation.
Data Synthesis by Key Question

Key Question 1

What population-based data sources have been used in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

Key Question 2

What methods have been used to link different population-based data sources?

The main product from KQ1 will be a list and description of all data sources and data systems, and categories of variables they contain. We will describe the data sources by criteria for population-based data systems, whether the data source is in the NCCOR registry, and what country, state or community it is from. We will organize the list of data systems by their usefulness for research and policy stakeholders.

For KQ2 we will qualitatively describe the methods currently used to link these data systems (see KQ2 data abstraction).

Potential incompleteness of the data abstraction may introduce errors and consequently biases in these findings. For example, not all data systems provide enough details about their data specifications and thus may lead to missing information. This may result in incomplete coding for a number of data systems. Consequently, the data aggregation may not be accomplished across all coding attributes and all data systems. The data synthesis will probably involve various denominators of data systems thus limiting the generalizability of some of the findings.

Data will also be aggregated and summarized across various coding schema attributes. The summary report will include various data system attributes across the entire list of data systems. The report will be accompanied with suggestions on which data specifications (e.g., linkage methods) have been found to be effective and which ones have faced challenges.

Key Question 3

What obesity measures, dietary and physical behaviors, and other outcomes have been assessed in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control?

We will describe counts of the number of studies that report each outcome and categories of outcomes of interest. We will also describe the types of measures or instruments used to assess these outcomes, and whether the policies had a positive, negative or neutral effect on the outcomes, taking into consideration whether reported differences are statistically significant and/or clinically important. We will stratify these tables by outcomes, and by country.

Key Question 4

Which experimental and non-experimental methods have been used in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

Key Question 5

What are the risks of bias in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

We recognize that the ultimate goal of this key question is to identify methods that would provide lower bias and higher validity in using multiple data systems in evaluating obesity prevention and control policies and programs.

We will describe types of study designs and analytic methods being used, their frequency of use, and whether they are preferred for certain policies, policies or data sources. We will also describe the risk of bias assessments across all studies and by study design.

Key Question 6

What methodological/analytic advances (e.g., data system features, approaches to linking data sources, or analytic methods) would help to strengthen efforts to estimate the effect of programs, policies or built environment changes on obesity prevention and control?

The goal of this project is not to analyze or prioritize the gaps identified by the TEP. The goal is to identify potential gaps and to present them in an unbiased manner.

Grading the Strength of Evidence for Major Comparisons and Outcomes

We will not evaluate the strength of evidence for a particular comparison or outcome as we are not assessing the comparative effectiveness of interventions in this review.

Assessing Applicability

We will assess applicability of the evaluation approaches and methods to other settings, policies and populations. We will stratify our findings by country and region of the world, as well as by similar healthcare systems, and by governmental policies versus non-governmental programs.

V. References

  1. Ogden CL, Carroll MD, Kit BK, et al. Prevalence of childhood and adult obesity in the United States, 2011-2012. JAMA. 2014 Feb 26;311(8):806-14. doi: 10.1001/jama.2014.732. PMID: 24570244.
  2. WHO overweight and obesity fact sheet. World Health Organization; 2016. http://www.who.int/mediacentre/factsheets/fs311/en/. Accessed on April 8 2016.
  3. Ogden CL, Carroll MD, Kit BK, et al. Prevalence of obesity and trends in body mass index among US children and adolescents, 1999-2010. JAMA. 2012 Feb 1;307(5):483-90. doi: 10.1001/jama.2012.40. PMID: 22253364.
  4. Ogden CL, Carroll MD, Curtin LR, et al. Prevalence of high body mass index in US children and adolescents, 2007-2008. JAMA. 2010 Jan 20;303(3):242-9. doi: 10.1001/jama.2009.2012. PMID: 20071470.
  5. Gortmaker SL, Swinburn BA, Levy D, et al. Changing the future of obesity: science, policy, and action. Lancet. 2011 Aug 27;378(9793):838-47. doi: 10.1016/s0140-6736(11)60815-5. PMID: 21872752.
  6. Rossen LM, Schoendorf KC. Measuring health disparities: trends in racial-ethnic and socioeconomic disparities in obesity among 2- to 18-year old youth in the United States, 2001-2010. Ann Epidemiol. 2012 Oct;22(10):698-704. doi: 10.1016/j.annepidem.2012.07.005. PMID: 22884768.
  7. Executive summary: Guidelines (2013) for the management of overweight and obesity in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and the Obesity Society published by the Obesity Society and American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Based on a systematic review from the The Obesity Expert Panel, 2013. Obesity (Silver Spring). 2014 Jul;22 Suppl 2:S5-39. doi: 10.1002/oby.20821. PMID: 24961825.
  8. Sturm R, Cohen DA. Zoning for health? The year-old ban on new fast-food restaurants in South LA. Health Aff (Millwood). 2009 Nov-Dec;28(6):w1088-97. doi: 10.1377/hlthaff.28.6.w1088. PMID: 19808703.
  9. The CDC Guide to Strategies for Reducing the Consumption of Sugar-Sweetened Beverages Centers for Disease Control and Prevention. 2010. http://www.cdph.ca.gov/SiteCollectionDocuments/StratstoReduce_Sugar_Sweetened_Bevs.pdf.
  10. Shrestha N, Kukkonen-Harjula KT, Verbeek JH, et al. Workplace interventions for reducing sitting at work. Cochrane Database Syst Rev. 2016;3:CD010912. doi: 10.1002/14651858.CD010912.pub3. PMID: 26984326.
  11. Zhu X, Yu CY, Lee C, et al. A retrospective study on changes in residents' physical activities, social interactions, and neighborhood cohesion after moving to a walkable community. Prev Med. 2014 Dec;69 Suppl 1:S93-7. doi: 10.1016/j.ypmed.2014.08.013. PMID: 25158208.
  12. LJ C, R F, AY M, et al. Motivating systems-oriented research on environmental and policy changes for obesity prevention. Pediatric Obesity. 2016:In press.
  13. Wadden TA, Butryn ML, Hong PS, et al. Behavioral treatment of obesity in patients encountered in primary care settings: a systematic review. JAMA. 2014 Nov 5;312(17):1779-91. doi: 10.1001/jama.2014.14173. PMID: 25369490.
  14. S M, J D. Park Rx: Using Technology to connect us to Nature. Health IT Buzz; 2014. https://www.healthit.gov/buzz-blog/consumer/park-rx-technology-connect-nature/. Accessed on April 8 2016.
  15. Ludwig J, Sanbonmatsu L, Gennetian L, et al. Neighborhoods, obesity, and diabetes--a randomized social experiment. N Engl J Med. 2011 Oct 20;365(16):1509-19. doi: 10.1056/NEJMsa1103216. PMID: 22010917.
  16. Mayne SL, Auchincloss AH, Michael YL. Impact of policy and built environment changes on obesity-related outcomes: a systematic review of naturally occurring experiments. Obes Rev. 2015 May;16(5):362-75. doi: 10.1111/obr.12269. PMID: 25753170.
  17. Long MW, Tobias DK, Cradock AL, et al. Systematic review and meta-analysis of the impact of restaurant menu calorie labeling. Am J Public Health. 2015 May;105(5):e11-24. doi: 10.2105/ajph.2015.302570. PMID: 25790388.
  18. Helmerhorst HJ, Brage S, Warren J, et al. A systematic review of reliability and objective criterion-related validity of physical activity questionnaires. Int J Behav Nutr Phys Act. 2012 Aug 31;9:103. doi: 10.1186/1479-5868-9-103. PMID: 22938557.
  19. Measureas Registry Resources. National Collaboration on Childhood Obesity Research; 2014. http://nccor.org/nccor-tools/measures/other-resources. Accessed on September 1, 2016.
  20. England CY, Andrews RC, Jago R, et al. A systematic review of brief dietary questionnaires suitable for clinical use in the prevention and management of obesity, cardiovascular disease and type 2 diabetes. Eur J Clin Nutr. 2015 Sep;69(9):977-1003. doi: 10.1038/ejcn.2015.6. PMID: 25711954.
  21. Troiano RP, McClain JJ, Brychta RJ, et al. Evolution of accelerometer methods for physical activity research. Br J Sports Med. 2014 Jul;48(13):1019-23. doi: 10.1136/bjsports-2014-093546. PMID: 24782483.
  22. Cummins S, Flint E, Matthews SA. New neighborhood grocery store increased awareness of food access but did not alter dietary habits or obesity. Health Aff (Millwood). 2014 Feb;33(2):283-91. doi: 10.1377/hlthaff.2013.0512. PMID: 24493772.
  23. MacDonald JM, Stokes RJ, Cohen DA, et al. The effect of light rail transit on body mass index and physical activity. Am J Prev Med. 2010 Aug;39(2):105-12. doi: 10.1016/j.amepre.2010.03.016. PMID: 20621257.
  24. Kaushal N. Do food stamps cause obesity? Evidence from immigrant experience. J Health Econ. 2007 Sep 1;26(5):968-91. doi: 10.1016/j.jhealeco.2007.01.006. PMID: 17382418.
  25. Chen R, Smyser M, Chan N, et al. Changes in awareness and use of calorie information after mandatory menu labeling in restaurants in King County, Washington. Am J Public Health. 2015 Mar;105(3):546-53. doi: 10.2105/ajph.2014.302262. PMID: 25602868.
  26. Elbel B, Kersh R, Brescoll VL, et al. Calorie labeling and food choices: a first look at the effects on low-income people in New York City. Health Aff (Millwood). 2009 Nov-Dec;28(6):w1110-21. doi: 10.1377/hlthaff.28.6.w1110. PMID: 19808705.
  27. Cheskin LJ, Frutchey R, McDermott AY, et al. Motivating systems-oriented research on environmental and policy changes for obesity prevention. Pediatr Obes. 2016 Apr 6doi: 10.1111/ijpo.12132. PMID: 27060703.
  28. Sturm R, Hattori A. Diet and obesity in Los Angeles County 2007-2012: Is there a measurable effect of the 2008 "Fast-Food Ban"? Soc Sci Med. 2015 May;133:205-11. doi: 10.1016/j.socscimed.2015.03.004. PMID: 25779774.
  29. Sekhobo JP, Edmunds LS, Dalenius K, et al. Neighborhood disparities in prevalence of childhood obesity among low-income children before and after implementation of New York City child care regulations. Prev Chronic Dis. 2014;11:E181. doi: 10.5888/pcd11.140152. PMID: 25321632.
  30. Schwartz AE, Leardo M, Aneja S, et al. Effect of a School-Based Water Intervention on Child Body Mass Index and Obesity. JAMA Pediatr. 2016 Mar;170(3):220-6. doi: 10.1001/jamapediatrics.2015.3778. PMID: 26784336.
  31. DHHS. The Surgeon General's Call To Action To Prevent and Decrease Overweight and Obesity. Rockville MD; 2001.
  32. Wilcox HC, Kharrazi H, Wilson RF, et al. Data Linkage Strategies to Advance Youth Suicide Prevention: A Systematic Review for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med. 2016 Oct 4doi: 10.7326/m16-1281. PMID: 27699389.
  33. Kharrazi H, Weiner JP. IT-enabled Community Health Interventions: Challenges, Opportunities, and Future Directions. EGEMS (Wash DC). 2014;2(3):1117. doi: 10.13063/2327-9214.1117. PMID: 25848627.
  34. Rubin DB. The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials. Stat Med. 2007 Jan 15;26(1):20-36. doi: 10.1002/sim.2739. PMID: 17072897.
  35. Quality Assessment Tool for Quantitative Studies. Effective Public Health Practice Project; 2009. http://www.ephpp.ca/tools.html. Accessed on November 17, 2016.
  36. Armijo-Olivo S, Stiles CR, Hagen NA, et al. Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research. J Eval Clin Pract. 2012 Feb;18(1):12-8. doi: 10.1111/j.1365-2753.2010.01516.x. PMID: 20698919.
  37. Hernan MA, Robins JM. Instruments for causal inference: an epidemiologist's dream? Epidemiology. 2006 Jul;17(4):360-72. doi: 10.1097/01.ede.0000222409.00878.37. PMID: 16755261.
  38. Imbens G, Lemieux T. Regression discontinuity designs: A guide to practice. Journal of Econometrics. 2008;142(2):615-35.
  39. Rosenbaum P, Rubin D. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41-55.
  40. Biglan A, Ary D, Wagenaar AC. The value of interrupted time-series experiments for community intervention research. Prev Sci. 2000 Mar;1(1):31-49. PMID: 11507793.
  41. Saldanha IJ, Wilson LM, Bennett WL, et al. Development and pilot test of a process to identify research needs from a systematic review. J Clin Epidemiol. 2013 May;66(5):538-45. doi: 10.1016/j.jclinepi.2012.07.009. PMID: 22995855.

VI. Definition of Terms

All relevant terms are defined in the above text and detailed in Appendix A.

VII. Summary of Protocol Amendments

If we need to amend this protocol, we will give the date of each amendment, describe the change and give the rationale in this section. Changes will not be incorporated into the protocol.

VIII. Review of Key Questions

There was no public review of the Key Questions.

IX. Key Informants

There were no Key Informants for this Project. The NIH Working Group for this Pathways to Prevention Project acted in the capacity of Key Informant and provided input to the EPC investigative team during the topic refinement/development portion of the project.

X. Technical Experts

Technical Experts constitute a multi-disciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, or outcomes and identify particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicting opinions are common and perceived as health scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design, and methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind nor do they contribute to the writing of the report. They have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.

Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

XI. Peer Reviewers

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. The EPC considers all peer review comments on the draft report in preparation of the final report. Peer reviewers do not participate in writing or editing of the final report or other products. The final report does not necessarily represent the views of individual reviewers. The EPC will complete a disposition of all peer review comments. The disposition of comments for systematic reviews and technical briefs will be published three months after the publication of the evidence report.

Potential Peer Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

XII. EPC Team Disclosures

EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Related financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify EPC core team investigators.

XIII. Role of the Funder

This project was funded under Contract No. HHSA 290-2012-00007 I from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

Appendixes

Appendix A. Definitions of Terms

Changes in the Built Environment: Built environments are the totality of places built or designed by humans, including buildings, grounds around buildings, layout of communities, transportation infrastructure, and parks and trails.1
Note: Examples include supermarkets, farmers markets, as well as infrastructure.

Data system: A data system involves the systematic collection of data, such as in a database, as well as the information technology infrastructure to maintain and operate the system.2

Natural experiment: Natural experiment refers to ways of evaluating policy, programmatic and environmental interventions using unplanned variation in exposure to assess the impact on health outcomes. The key features of these definitions are that: (1) the intervention (policy, program, environment change) is not undertaken for the purposes of research; and (2) the variation in exposure and outcomes is analyzed using methods that attempt to make causal inferences. Outside of a randomized controlled trial it is rare for variation in exposure to an intervention to be random, so special care is needed in the design, reporting and interpretation of evidence from natural experimental. Definition adapted from Craig, 2012.3

Policy: is broadly defined to include both formal public policies at local, state and federal levels of government, and organizational level policies, such as those implemented by large organizations, worksites or school districts. Examples include, but are not limited to, the development of supermarkets in underserved areas, calorie labeling requirements, taxes on foods and/or beverages, after-school and summer programs, modification of the built (or human-made) environments to encourage walking or cycling for transportation or leisure.4

Program: is defined as a set of activities initiated by governmental or other organizational bodies to enhance obesity prevention and control. Examples might include programs implemented worksites, healthcare organizations, after-school or summer programs, or communities that can be expected to improve obesity related behaviors such as energy intake and activity level.4

Appendix A References

  1. Transportation Research Board and Institute of Medicine. Does the Built Environment Influence Physical Activity? Examining the evidence. Special Report 282. National Academies Press. Washington, DC: 2005. http://onlinepubs.trb.org/onlinepubs/sr/sr282.pdf
  2. Wilcox H, Wissow L, Kharrazi H, et al. Data Linkage Strategies To Advance Youth Suicide Prevention. Evidence Report/Technology Assessment No. 222. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. 290-2012-00007-I.) AHRQ Publication No. 16-E001-EF. Agency for Healthcare Research and Quality. Rockville, MD: 2016. www.effectivehealthcare.ahrq.gov/reports/final.cfm.
  3. Craig P, Cooper C, Gunnell D, et al. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Community Health. 2012 Dec;66(12):1182-6. doi: 10.1136/jech-2011-200375. PMID: 22577181.
  4. Department of Health and Human Services, Part 1. Overview Information. Department of Health and Human Services. http://grants.nih.gov/grants/guide/pa-files/PA-16-165.html. Accessed on October 24, 2016.

Appendix B. List of Commonly Used Validated Dietary and Physical Activity Measures

Dietary Measures

Common short dietary assessment instruments (dietary screeners):

  • Fruit & Vegetable Intake Screeners in the Eating at America's Table Study (EATS)
  • Five-Factor Screener in the 2005 National Health Interview Survey (NHIS) Cancer Control Supplement (CCS)
  • Dietary Screener in the 2005 California Health Interview Survey (CHIS)
  • Dietary Screener in National Health and Nutrition Examination Survey (NHANES) 2009-2010

24-hour dietary recall

  • Canadian Healthy Eating Index
  • Diet Quality Index

24-hour dietary recall using common Food frequency questionnaires:

  • Behavioral Risk Factor Surveillance System (BRFSS)
  • Block
  • Healthy Eating Vital Signs
  • Greene GW
  • Resincow K
  • NHANES

Food Checklists, e.g.

  • Dietary Approaches to Stop Hypertension (DASH) online questionnaire
  • Murphy S Food Behavior checklist
  • Technology-enabled (e.g., taking a picture with your cell phone)
Physical Activity Measures
  • Self-report questionnaire (http://circ.ahajournals.org/content/128/20/2259#T4)
  • Global physical activity questionnaire
    • The European Investigation into Cancer and Nutrition study (EPIC) Physical Activity Questionnaire (PAQ): (EPAC)
    • Godin Leisure Time Exercise
    • Global Physical Activity Questionnaire (GPAQ)
    • Minnesota Heart Health
    • Stanford Usual PAQ
  • Short recall of physical activity questionnaire
    • BRFSS, 2001
    • Atherosclerosis Risk in Communities (ARIC)/Baecke
    • Coronary Artery Risk Development in Young Adults Study (CARDIA)
    • Childhood and Adolescent Migraine Prevention Study (CHAMPS)
  • Quantitative history of physical activity questionnaire
    • Modifiable Activity Questionnaire
    • Minnesota Leisure-time Physical Activity (LTPA)
  • Self-report activity diary/logs
    • Bouchard Physical Activity Record
  • Direct observation
  • Measures of energy expenditure (indirect calorimetry, doubly labeled water method)
  • Physiologic measures (heart rate monitors)
  • Motion sensors (accelerometers, pedometers)

Appendix C. Preliminary Search Strategies

Table C1. PubMed Search Strategy
# Term/string
1 obesity[mh]
2 obese[tiab]
3 obesity[tiab]
4 overweight[tiab]
5 “over weight”[tiab]
6 BMI[tiab]
7 “body mass index”[tiab]
8 “Body mass index”[mh]
9 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8
10 "Policy"[Mesh]
11 policy[tiab]
12 policies[tiab]
13 law[tiab]
14 Purchasing[tiab]
15 Purchases[tiab]
16 “food labeling”[mh]
17 “calorie information”[tiab]
18 “calorie labeling”[tiab]
19 Environment[tiab]
20 10 OR 11 OR 12 OR 13 OR 14 OR 15 OR 16 OR 17 OR 18 OR 19
21 9 AND 20
Table C2. CINAHL, PsycINFO, and EconLit Search Strategy
Search ID# Search Terms
S48 S33 AND S46
S47 S33 AND S46
S46 S34 OR S35 OR S36 OR S37 OR S38 OR S39 OR S40 OR S41 OR S42 OR S43 OR S44 OR S45
S45 TI environment OR AB environment
S44 TI "calorie labeling" OR AB "calorie labeling"
S43 TI "caloric information" OR AB "caloric information"
S42 TI "calorie information" OR AB "calorie information"
S41 TI "food labeling" OR AB "food labeling"
S40 TI purchases OR AB purchases
S39 TI purchasing OR AB purchasing
S38 TI law OR AB law
S37 TI policies OR AB policies
S36 TI policy OR AB policy
S35 (MH "Food Labeling+")
S34 (MH "Public Policy+")
S33 S25 OR S26 OR S27 OR S28 OR S29 OR S30 OR S31 OR S32
S32 TI BMI OR AB BMI
S31 TI "body mass index" OR AB "body mass index"
S30 TI "over weight" OR AB "over weight"
S29 TI overweight OR AB overweight
S28 TI obese OR AB obese
S27 TI obesity OR AB obesity
S26 (MH "body mass index+")
S25 (MH "Obesity+")
S24 S9 AND S22
S23 S9 AND S22
S22 S10 OR S11 OR S12 OR S13 OR S14 OR S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21
S21 TI environment OR AB environment
S20 TI "calorie labeling" OR AB "calorie labeling"
S19 TI "caloric information" OR AB "caloric information"
S18 TI "calorie information" OR AB "calorie information"
S17 TI "food labeling" OR AB "food labeling"
S16 TI purchases OR AB purchases
S15 TI purchasing OR AB purchasing
S14 TI law OR AB law
S13 TI policies OR AB policies
S12 TI policy OR AB policy
S11 (MH "Food Labeling+")
S10 (MH "Public Policy+")
S9 S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8
S8 TI BMI OR AB BMI
S7 TI "body mass index" OR AB "body mass index"
S6 TI "over weight" OR AB "over weight"
S5 TI overweight OR AB overweight
S4 TI obese OR AB obese
S3 TI obesity OR AB obesity
S2 (MH "body mass index+")
S1 (MH "Obesity+")

Appendix D. Organizations and Agencies for Handsearching as part of Grey Literature Search

Appendix D. Organizations and Agencies for Handsearching as part of Grey Literature Search
Organization Website
Robert Wood Johnson Foundation http://www.rwjf.org/
WHO http://www.who.int/en/
Rudd Center http://www.uconnruddcenter.org/
Bloomberg Foundation https://www.bloomberg.org/
National Institute for Child Health Quality http://www.nichq.org/
National Policy and Legal Analysis Network to Prevent Childhood Obesity http://www.changelabsolutions.org/childhood-obesity
ChangeLab Solutions http://www.changelabsolutions.org/
Informing Healthy Public Policy – Medical Research Council https://www.sphsu.mrc.ac.uk/programmes/policy.html
Center on Social Dynamics and Policy https://www.brookings.edu/center/center-on-social-dynamics-and-policy/
Active Living Research www.activelivingresearch.org

Appendix E: Preliminary Draft of Data System Classification/Coding Schema (KQ 1)

Data Relatedness / Availability
  • Directly available
    • Dependent variables
      • Required outcomes (e.g., weight-related outcomes: Weight and BMI; obesity-related dietary or physical activity behavioral outcomes). See Appendix B for list of outcomes
      • Other outcomes (e.g., food environment, physical activity environment, purchasing behavior, urban renewal)
    • Independent variables
      • Genomic data
      • Demographic data
      • Social data
      • Environmental/Geographic data
      • Health/Medical/Clinical data
  • Indirectly available
    • Dependent variable
      • Primary
      • Secondary
    • Independent variable
    • Current form of the variable
    • Method to make it available
      • Simple statistics/arithmetic conversions (e.g., metric to imperial)
      • Inference models (e.g., logical rules)
      • Imputation method (e.g., regression methods)
      • Spatial-triangulation (e.g., time-trend analysis)
      • Geo-triangulation
      • Other methods?
  • Data definitions
    • Has clear definitions for data fields (e.g., data dictionary) – if yes, include them
    • Follows a standard definition for these data fields – if yes, mention which ones
Data Granularity
  • Patient-level
    • Cross-sectional
    • Repeated
      • Retrospective/Historical
      • Time-series/Ongoing
      • Aggregate on certain dimension
    • Demographics (e.g., age, gender, socioeconomic status)
    • Geographical (e.g., zip code, census block)
    • Other patient specifications (e.g., diagnosis, treatment, and other attributes)
    • Entity (e.g., payer, provider, center)
  • Total sums / aggregates with no levels
Data Denominator Coverage
  • Geographic coverage
    • National
    • Regional (one or more)
    • State (one or more)
    • Locality (e.g., one or more)
    • Specific geographical boundary smaller than State
      • County
      • Zip code
      • Census Block
      • Tribal
  • Demographic coverage
  • Data Source / Entity coverage
    • Payer
    • Provider
    • Department of Health