The capacity to measure primary care spending is essential to federal, state, and health systems efforts to promote the ability of primary care to improve healthcare delivery and outcomes. Thought leaders and researchers in this field have started to identify the range of primary care and primary care spending conceptualizations,^1-3 and begun the work of documenting pros and cons of different approaches,^4,5 but these efforts have not been comprehensive. Although most approaches to assessing primary care spending start by stating a definition, this has led to divergence as different conceptual definitions translate into use of different claims, diagnosis codes, and accounting terms. In 2017, a convening to develop a framework for measuring primary care spending was held that brought together health economists, health services researchers, and policymakers from the United States and abroad. A consensus on definitions or methodology was not reached, but a framework for understanding definitions⁶ and measurement was developed that can be a guide when evaluating approaches as it aims to increase definitional reliability using available data.

Designing, implementing, and monitoring systemic healthcare payment reform requires a standard way of measuring and monitoring resources devoted to primary care. The purpose of this Technical Brief is to organize the data and experience available from published peer-reviewed articles, gray literature, and experts on approaches used to estimate primary care spending in the United States. The Technical Brief will synthesize and present information about estimation methods including: 1) who has used different methods; 2) how estimates have been used; 3) details on underlying definitions of primary care and primary care spending; 4) data sources employed; and 5) expert consensus, which we will use to inform our search strategy and guide us in assessing whether a standard measure or best estimate of spending can be identified. The Brief will also add assessment to this information by including: 1) the advantages and disadvantages of the different approaches to estimating primary care spending, including how primary healthcare spending, which is the focus of international estimates, differs from primary care spending, which is the focus in the United States and therefore this Technical Brief; 2) an overview (map) of the evidence linking different primary care spending estimates to outcomes; 3) gaps that future research needs to address; and 4) key considerations for developing primary care spending estimates that are valid and may be standardized.

The five questions below guide our work in synthesizing a description of research, ongoing efforts, and directions in measuring primary care spending.

1. What are the definitions, data sources, and methodologies used to estimate primary care spending in published reports?
   1. How do these various primary care spending estimation methods vary by:
      1. Relative pros and cons of each estimation method
      2. Administrative burden
      3. Range of spending estimates
      4. Sensitivity analyses
   2. What is the evidence of the relationship between different primary care spending estimation methods and the absolute and relative levels of primary care spending and health outcomes including morbidity, mortality, quality of life, and health equity?
2. What are the research gaps in understanding primary care spending estimation methods based on the findings of the evidence map?
3. What are considerations for developing valid and standardized estimation of primary care spending?
4. What are approaches that health economists, health services researchers, payers, health systems, and policymakers can employ to develop and implement a standardized measure of primary care spending and to assess spending over time, across payers/populations, and across states?
5. Contextual Questions:
   1. Is there any emerging consensus among experts in the field toward a standard or preferred method for assessment of primary care spending?
   2. How have policymakers and other decision makers used primary care spending measures?

As there is no consensus on definitions or methodology for measuring primary care spending and published literature of the definitions, approaches to estimation, and usage of different estimates is expected to be limited this Technical Brief will integrate information from discussions with Key Informants and findings from the gray literature, with published literature. Key Informant expertise on the topic will be used to inform our search strategy, supplement what we find in the published literature and gray literature, and provide suggestions on how to organize our findings to answer the Guiding Questions.

1. Data Collection

A. Discussions with Key Informants

In order to ensure we include information on current and promising definitions and measures of primary care spending we will convene a group of Key Informants to provide broad and balanced perspectives relevant to this topic, including persons with experience and expertise in developing frameworks for measuring primary care spending; developing guidelines or policies for measuring primary care spending; health economics; and health spending expertise as well as people who use the estimates. The Key Informants will include researchers, clinicians, policymakers, representatives of professional societies and organizations, and state and federal agency representatives. We will select Key Informants to ensure appropriate multidisciplinary representation, including persons with expertise in adult and child/adolescent primary care spending, and aim to include Key Informants with expertise in equity and social determinants of health.

Key Informants will have two main sets of tasks. The first will focus on helping us refine our search strategy, particularly by helping identify sources of gray literature. The second will involve advising us on how we organize and present what we find.

As the first task has to be completed quickly, we will give Key Informants targeted questions and request comments on our proposed search strategy and suggestions for data sources. We will have opportunities to tap into Key Informants’ expertise during several ‘open office hours’ with the project team. If Key Informants cannot attend these ‘open office hours’ we will email them to gain access to their advice and insights. We will discuss potential revisions to Guiding Questions with Key Informants. We will ask that they recommend sources covering the scope of the Technical Brief including: known states or organizations with approaches to measuring primary care spending; databases (public and proprietary) used to estimate primary care spending, and the attributes of those databases that might influence estimates; identification of core clinicians, healthcare professionals, and services that should be included in the definition of primary care; the pros and cons of different definitions; recommendations for moving toward consensus on or harmonization of definitions of primary care and primary care spending, as well as other related issues that may arise during the discovery process.

We will post input as it is received in a secure, accessible space (e.g., OneDrive) so Key Informants can build on and react to colleagues’ comments.

For the second task, we will convene small groups, supplemented with individual discussions with Key Informants, as needed. We will ask if they are aware of any sources we have not included and ask for their input on how we propose to organize our findings. The feedback from the Key Informants will inform the approach our project team uses to organize the different definitions of primary care, the various ways of measuring primary care spending, the methodologies used, and the key characteristics of those methods (e.g., purpose and audience, definition of primary care, administrative burden, etc.). Our team will weigh the expert input from Key Informants, who may have different perspectives, in order to identify and organize – in a balanced way – the research and discussions on this important topic.

B. Published Literature search.

An experienced research librarian will create search strategies of search terms and medical subject headings (MeSH) for definitions and methods of estimating primary care spending, and perform searches on the following databases: Ovid MEDLINE and the Cochrane Library. The MEDLINE search strategy is shown in Table 1. We propose to focus our searches on the last 10 years because any spending method prior to this are unlikely to be relevant today. However, we will ask Key Informants for their input on this, and look back in the literature further than 10 years if the experts recommend doing so. We will supplement the searches with a review of reference lists of identified publications for additional relevant studies.

We will update the searches while the report is undergoing peer and public review to capture any recently added publications. If any new eligible studies are identified from the update searches or are identified based on peer or public review comments, they will be added to the Brief prior to finalization.

Table 1. MEDLINE Search Strategy

C. Gray Literature search.

In order to ensure we capture relevant information on definitions and methodologies for measuring primary care spending, we will conduct gray literature searches with the assistance of a reference librarian. These searchers will focus on identifying reports generated by U.S. States and Federal Agencies, as well as health systems, and topic focused foundations. For example, Oregon Health Authority publishes an annual primary care spending report. We can search each state to see if they have a comparable report. We will also search any identified clearinghouse or databases with relevant gray literature such as the National Library of Medicine digital collection, Health Policy and Services Research. We will be limited to the U.S. experience and estimates of spending for primary care will be the focus. While we will conduct a rigorous search, we acknowledge that it may not be exhaustive and will ask Key Informants, content experts, and other individuals and organizations to suggest additional key sources.

Supplementary Evidence and Data: AHRQ will publish an announcement in the Federal Register to notify stakeholders about the opportunity to submit information via the SEADS portal on the Effective Health Care Website.

2. Data Organization and Presentation

We will include systematic reviews, other types of reviews, and primary studies of primary care spending estimates in the United States. All citations identified through searches will undergo dual independent review for eligibility based on whether they address the Guiding Questions. For Guiding Question 1b, we will review proposed inclusion/exclusion criteria in consultation with Key Informants, the Task Order Officer at AHRQ, and the partner. Included studies will at a minimum be applicable to U.S. populations, report on models or methods that have been used to generate estimates not just hypothetical or proposed models, and provide information from published or publicly available sources that include sufficient detail to ascertain key characteristics of the method used. Each citation will undergo dual review by trained members of the research team. Studies marked for possible inclusion by either team member will undergo full-text review. Each full-text article will be independently reviewed by two trained members of the research team for inclusion or exclusion on the basis of the eligibility criteria. If the reviewers disagree, conflicts will be resolved by discussion and consensus or by consulting another member of the review team. Reasons for exclusion at the full text stage will be recorded. We will use DistillerSR+ to assist in managing the study selection process and EndNote reference management software.

Throughout the search process, we will identify and retain the subgroup of studies that look at primary care spending in the non-U.S.-based (international community). While these articles will not be analyzed in the same manner as publications that meet the inclusion criteria, specific international studies may be provided for purposes of contrast or discussion.

A. Information Management

For studies meeting inclusion criteria, data will be entered into abstraction forms summarizing pertinent information from each study, such as characteristics of study populations, details on methodologies used for estimating primary care spending, including the database(s) used, definitions of primary care and primary care spending employed, as well as aggregation and analysis methods, including sensitivity, if reported, that are part of the approach, and the spending estimate as a number or range, depending on how it is reported. All data abstractions will be reviewed for completeness and accuracy by another member of the team.

B. Data Presentation

We will present our findings in the Technical Brief as responses to the Guiding Questions. We will categorize and summarize findings from the published literature, gray literature, and the Key Informant discussions qualitatively in the text of the Brief.

We will develop an evidence map describing the current definitions, data sources, and methodologies used to estimate primary care spending and where the current gaps in evidence are. Our approach to organizing and presenting the evidence map results will be informed by prior evidence mapping efforts and examples of recently produced evidence maps,^7-14 as well as guidance from Key Informants.

Using the evidence map, we will layout considerations for how to develop estimations of primary care spending that can be standardized or harmonized, if the findings allow, with input from Key Informants and informed by our literature searches.

After developing our synthesis plan, we will schedule one or two larger meetings with our Key Informants to review our synthesis and presentation prior to completing the draft, which will benefit from the interaction of the Key Informants at this critical stage.

1. Michael H. Bailit MWF, Margaret L. Houy. Standardizing the Measurement of Commercial Health Plan Primary Care Spending. Milbank Memorial Fund. 2017.
2. Primary Care Collaborative. State Primary Care Investment Initiatives. 2022.
3. Taylor E, ,Bailit, M., Kanneganti., D. Measuring Non-Claims-Based Primary Care Spending. Milbank Memorial Fund. 2021.
4. Basu S, Berkowitz SA, Phillips RL, et al. Association of Primary Care Physician Supply With Population Mortality in the United States, 2005-2015. JAMA Intern Med. 2019;179(4):506-14. doi: 10.1001/jamainternmed.2018.7624. PMID: 30776056.
5. Friedberg MW, Hussey PS, Schneider EC. Primary care: a critical review of the evidence on quality and costs of health care. Health Aff (Millwood). 2010;29(5):766-72. doi: 10.1377/hlthaff.2010.0025. PMID: 20439859.
6. Jabbarpour Y, Petterson S, Jetty A, et al. The Health of US Primary Care: A Baseline Scorecard Tracking Support for High-Quality Primary Care, The Milbank Memorial Fund and The Physicians Foundation. 2023.
7. Allmark P, Baxter S, Goyder E, et al. Assessing the health benefits of advice services: using research evidence and logic model methods to explore complex pathways. Health Soc Care Community. 2013;21(1):59-68. doi: 10.1111/j.1365-2524.2012.01087.x. PMID: 23039788.
8. Bragge P, Clavisi O, Turner T, et al. The Global Evidence Mapping Initiative: scoping research in broad topic areas. BMC Med Res Methodol. 2011;11:92. doi: 10.1186/1471-2288-11-92. PMID: 21682870.
9. Callahan P, Liu P, Purcell R, et al. Evidence map of prevention and treatment interventions for depression in young people. Depress Res Treat. 2012;2012:820735. doi: 10.1155/2012/820735. PMID: 22496974.
10. Chou R, Dana T, Buckley DI, et al. Update Alert 11: Epidemiology of and Risk Factors for Coronavirus Infection in Health Care Workers. Ann Intern Med. 2022;175(8):W83-w4. doi: 10.7326/l22-0235. PMID: 35816711.
11. Hempel S, Taylor SL, Marshall NJ, et al. VA Evidence-based Synthesis Program Reports. Evidence Map of Mindfulness. Washington (DC): Department of Veterans Affairs (US); 2014.
12. Hempel S, Taylor SL, Solloway MR, et al. VA Evidence-based Synthesis Program Reports. Evidence Map of Acupuncture. Washington (DC): Department of Veterans Affairs (US); 2014.
13. Hetrick SE, Parker AG, Callahan P, et al. Evidence mapping: illustrating an emerging methodology to improve evidence-based practice in youth mental health. J Eval Clin Pract. 2010;16(6):1025-30. doi: 10.1111/j.1365-2753.2008.01112.x. PMID: 20337833.
14. Totten A, Womack DM, McDonagh MS, et al. Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication. Comparative Effectiveness Review No. 254. (Prepared by the Pacific Northwest Evidence-based Practice Center under Contract No. 75Q80120D00006.) AHRQ Publication No. 22(23)-EHC023. Rockville, MD: Agency for Healthcare Research and Quality; December 2022. doi: https://doi.org/10.23970/AHRQEPCCER254. Posted final reports are located on the Effective Health Care Program search page.

In the event of protocol amendments, the date of each amendment will be accompanied by a description of the change and the rationale.

Within the Technical Brief process, Key Informants serve as a resource to offer insight into the clinical context of the technology/intervention, how it works, how it is currently used or might be used, and which features may be important from a policy standpoint. They may include clinical experts, researchers, payers, or other perspectives. Differing viewpoints are expected, and all statements are crosschecked against available literature and statements from other Key Informants. Information gained from Key Informant interviews is identified as such in the report. Key Informants do not do analysis of any kind nor contribute to the writing of the report and will not review the report, except as given the opportunity to do so through the public review mechanism.

Key Informants must disclose financial conflicts of interest greater than $5,000 and other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the draft report are considered by the EPC in preparation of the final report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and may be published three months after the publication of the Evidence report.

Potential Reviewers must disclose financial conflicts of interest greater than $5,000 and other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $5,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

Most EPC core team members have no financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. However, one of our Co-Investigators, Robert Phillips, has disclosed financial conflicts of interest greater than $1,000. Although financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify EPC core team investigators, the lead Investigator and EPC leadership for this Technical Brief have reviewed his disclosures and have concluded they are manageable and not unexpected due to the limited expertise in the field of study. Dr. Totten will oversee any conflicts that arise and manage and reassign work as necessary to avoid any bias. It should also be noted that the Task Order Officer has approved Dr. Phillips after disclosures.

This project was funded under Contract No. 75Q80120D00006 from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The AHRQ Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.