Background and Objectives for the Technical Brief

Consumers who utilize healthcare facilities can benefit from access to comparative information on costs and quality (Sinaiko and Rosenthal, 2011; Berger et al, 2013a). The Patient Protection and Affordable Care Act (2010) promoted price and quality transparency by strengthening the Department of Health and Human Facilities (HHS) commitment to the public reporting of performance measures on cost and quality. The Agency for Healthcare Research and Quality (AHRQ) supports these activities through the development of measures, as well as research on public reporting (James, 2012). Public reporting describes any formal or informal effort to provide consumers with access to information that allows the assessment and/or comparison of providers or facilities. Traditionally, indicators related to the quality of healthcare are measured through various indicators of processes and/or outcomes. To promote efficiency and affordability, recent efforts have aimed to incorporate measures of costs into existing and emerging public reporting efforts (Sofaer, 2011). By promoting price transparency, such activities promote competition among providers, and encourage consumers to account for costs in their healthcare decisions (NCQA, 2013).

Public reporting has been defined as data about a healthcare structure, process, or outcome at any provider level that is publicly available or available to a broad audience free of charge or at a nominal cost (AHRQ, 2012). For the purposes of the Technical Brief, we will focus on public reporting practices that present cost data that compare providers or organizations and which are directed to consumers.

Despite the number of public reports that exist today (Christian et al., 2010), there are very little data on consumer experiences with these reports. A majority of consumers who access public reporting websites are looking for information about provider quality, and some are using quality data to make hospital selections (Bardach et al, 2011). This said, several limitations about public reports from the consumer’s perspective have been noted and these span report content, design, and accessibility (Sinaiko et al 2012). Some experimental data are available to demonstrate the potential for public reporting of cost and quality data to guide consumers towards high-valued healthcare (Hibbard et al, 2012). However, the impact of reporting costs on healthcare decisions is not known, and there is some possibility of unintended consequences on the demand and supply side (Berger et al, 2013b). Consequently, strategies to improve results from public reporting have emerged (Mehrota et al, 2012; Luft, 2012).

This Technical Brief will provide a snapshot of current practices of the public reporting of cost measures in healthcare. Using a scan of public reporting practices and a targeted literature review, we will both catalog the approaches to public reporting of costs and address which are most likely to be valuable to consumers. The scope of the brief is limited to costs associated with healthcare providers and facilities (including clinics, hospitals, skilled nursing facilities, nursing homes) in the US, and thus will exclude public reporting on products (e.g. pharmaceuticals and medical devices/aids), healthcare plans and foreign practices. Emphasis will be placed on financial measures of costs (including data presented numerically, graphically and pictorially), but may extend to include resource utilization measures when appropriate. Finally, the focus of our search strategies will be on actual practices for the public reporting of costs that compare healthcare providers or facilities across defined geographic areas and that are freely available to the public. We acknowledge that individual providers/facilities may report their own costs against some benchmark (e.g. national average) and other public reporting sites may be exclusively accessible via membership or after the payment of a fee. We will include such practices when they are detailed in the literature or are indirectly identified, but they will not be searched for de novo.

Guiding Questions

1. What measures of costs about healthcare providers and facilities have been publicly reported?

1. Who produces these reports and where are they available?
2. For what facilities are costs reported?
3. At what level are these data aggregated (e.g. provider, facility, etc.)?
4. How are the cost data reported (e.g., dollar amounts, symbols, graphs etc.)?
5. How are the costs of providers/facilities compared (e.g., how many facilities, regional verses national comparisons etc.)?

2. Are the measures of costs that are being reported consumer centered?

1. How are consumers instructed to use the data?
2. What techniques are used to guide consumers to interpret the data appropriately?
3. Is there evidence that the data is used by consumers?
4. Is the data relevant to consumers making healthcare decisions?
5. Is the data easily accessible and presented in a consumer friendly way?

3. What are the intended and unintended consequences of consumers’ use of public-reported cost data?

1. Do consumers find the public reporting of cost measures relevant and are consumers satisfied with the experience?
2. Does the public reporting of cost measures impact (or have the potential to impact) consumers’ decisions or behaviors?
3. What are the potential unintended consequences of public reporting of cost measures?
4. Are there key research gaps and needs for future research?

Methods

Guided by key informant discussions, we will incorporate data from the content of public-reporting websites and a targeted literature review into an integrated review. Data from the review of actual public reporting sites will be used to answer guiding questions 1 and 2. The data from the targeted literature review will be used to supplement our response to guiding questions 1 and 2, and will be the primary source of data for answering guiding question 3.

1. Data collection

A. Discussion with Key Informants

We will discuss with key informants how we plan to identify and synthesize the data from our review of actual public reporting practices and targeted literature review (Appendix 1). Key informants will include experts knowledgeable about current and emerging practices for the public reporting of healthcare costs. Engagement of key informants will help ensure that our review of actual public reporting practices is complete by identifying emerging or alternative practices for the public reporting of costs.

Consistent with Office of Management and Budget (OMB) guidelines, we will include no more than nine non-Federal employees in the discussion. We envision conducting two group discussions with our key informants. The first set of discussions will focus on identifying public reporting practices (key Web sites) and relevant publications (Guiding Question 1). The second set of discussions, which will be held after examination of initial sampling of websites, will focus on methods for reporting the data gleaned from public websites and methods for assessing the consumer-centeredness of sites (Guiding Question 2). Both calls will also inform our approach for the targeted literature review, which is primarily targeted at assessing the intended and unintended consequences of the public reporting of cost measures (Guiding Question 3). Discussions via teleconference are likely to last 45-60 minutes. Field notes will be taken during the key informants’ discussion calls and representatives from AHRQ will be invited to participate on the calls.

B. Analysis of actual public reporting practices

We will focus our review on public websites reporting comparative data. A preliminary list of candidate public reporting websites has been developed, based on several recently completed environmental scans aimed at cataloging public reporting websites that may be relevant to consumers (Appendix 2). This list was derived from multiple sources. First, a recent review by the Robert Wood Johnson Foundation (RWJF), completed in October 2013, updated an existing directory of web sites that are aimed at helping consumers find reliable information on healthcare providers in defined geographic areas (RWJF, 2013). For our review, we will access the sites included in the RWJF study, and also the sites that the RWJF investigators chose to exclude. Second, other public reporting sites were identified from several other key publications (Kullgren et al, 2013; Yegian et al, 2013; O’Neil et al, 2010). In addition to this preliminary list, additional public reporting sites, particularly those that may present novel and consumer-centric approaches to the public reporting of cost data, will be identified through our discussions with the key informants and the targeted literature review.

During the initial review, EPC investigators and staff will determine if the public sites contain any cost measures that may be relevant to consumers. Once we have identified the candidate sites, we will do an in-depth analysis of those sites including cost measures. Field notes will document the approach and time taken to use each website. For those sites containing any cost data or supporting material such as instructions for use, electronic copies will be recorded to document the approach to public reporting of cost measures. In a second stage, these copies will be qualitatively assessed for content and for their relevance to consumers. Consistent with the guiding questions, the measures, data sources and other reporting approaches will be catalogued. To answer Guiding Question 2, we will use information learned from our targeted literature review and discussion with key informants to determine methods for assessing the consumer centeredness of the websites. This will be quantitative if we identify appropriate tools (such as a scale), and descriptive if not.

C. Targeted literature review

Another important source of data will come from a targeted literature review. These data will identify additional public websites that report cost data (Guiding Question 1), will identify definitions and criteria that will be used to assess the consumer centeredness of websites (Guiding Question 2), and will address Guiding Question 3 (i.e. identifying the intended and unintended consequences of consumers’ use of public-reported cost data on healthcare providers and facilities).

This review will be aided by our experience with two previous reviews on the topic of public reporting (Berger et al 2013a, b). Specifically, we will conduct a review of the MEDLINE®, Scopus, and EconLit databases. If insufficient content is found in these literatures, we will expand our search to include EMBASE® and Social Science Citation Index (Web of Science).

Table 1 outlines the inclusion criteria. All types of literature that reference public reports of cost data in healthcare will be included, including peer-reviewed research articles, editorials, reviews, letters, opinion pieces, professional guidelines, white papers, and reports. We will include literature that reflects the real-world use of cost data. We will only include literature written in English.

Our preliminary search strategy is detailed in Appendix 3. For this review, additional citations may be identified through the key informant discussions, through the scan of actual public reports and through a hand search of references in the included literature.

We will review all the titles and abstracts identified through our searches against our inclusion/exclusion criteria. Each abstract will be reviewed by at least two members of the investigative team. When differences between the reviewers arise, we will err on the side of inclusion. For studies without adequate information to make the determination, we will retrieve the full-text articles and review them against the inclusion/exclusion criteria. Each article will be summarized by one member of the investigative team into a few main points, which will be used when we synthesize the findings.

As the review will include all types of admissible evidence, formal data synthesis methods will not be used. Rather, the synthesis will highlight the common themes that were identified in each included article. The synthesis will take the form of a narrative review that will describe the current status of the published literature regarding public reports of healthcare cost data.

2. Data Organization and Presentation

A. Information Management

The primary data source for this project is the actual public reporting of cost data – the vast majority of which is published on public websites. As it is likely that such websites will be updated frequently, we will limit this review to content that is publically available in November and December of 2013. To ensure that an accurate record is kept of this data, we will save PDFs of all relevant web pages containing public reporting. If public reporting of costs is repetitive on a given site, we will aim to capture a sample of the various techniques, but not copies of all such materials. We will also maintain field notes detailing our review of each public reporting website that will detail how we used the site and how long it took to find information.

Data from the targeted literature review will be managed consistent with the practices of our Evidence-based Practice Center. Given that this Technical Brief is somewhat qualitative, data abstraction from the articles will be largely descriptive (as with a scoping review) and will also focus on the identification of any seminal articles beyond the window of review. Qualitative content analysis and meta-synthesis methods will be used to describe the content of the literature. We will maintain records of our analysis and synthesis.

B. Data Presentation

Our findings will be presented in the order of the guiding questions. We will catalog the information available from websites identified through previous reviews, key informants and targeted literature search (Guiding Question 1). We will provide a qualitative assessment of the consumer centeredness of the information provided in websites (Guiding Question 2). This assessment will be informed by findings from the literature searches and discussion with key informants. We will provide a qualitative assessment of the targeted literature search to answer Guiding Question 3.

References

1. AHRQ (2012) Public Reporting as a Quality Improvement Strategy. Closing the Quality Gap: Revisiting The State of the Science. Evidence Report No 208. (Prepared by the Oregon Evidence-Based Practice Center under Contract No. 290-2007-10057.July 2012
2. Berger Z, Hutfless S, Joy S, Bridges J. (2013) Can public reporting impact patient outcomes and disparities? A systematic review. Patient Education and Counseling, doi: 10.1016/j.pec.2013.03.003. [Epub ahead of print].
3. Berger Z, Karmarkar T, Boss E, Hutfless S, Bridges J. (2013) Does public reporting have an impact on healthcare disparities? A systematic review of claims and evidence gaps, Draft.
4. Christianson JB, Volmar KM, Alexander J, Scanlon DP. (2010) A report card on provider report cards: Current status of the healthcare transparency movement. J Gen Intern Med. 25(11):1235-41.
5. Cook WK. (2008) Integrating research and action: a systematic review of community-based participatory research to address health disparities in environmental and occupational health in the USA. Journal of Epidemiology & Community Health;62:668-676.
6. Hibbard J, Greene J, Sofaer S, Firminger K, Hirsh J (2012) An Experiment Shows That a Well-Designed Report on Costs and Quality Can Help Consumers Choose High-Value Healthcare, Health Affairs, 31: 560–8.
7. Kullgren JT, Duey KA, Werner RM. (2013) A census of state healthcare price transparency websites. JAMA. 309:2437-2438
8. Israel BA, Eng E, Schulz AJ, Parker EA, Becker AB. (1998) Review of Community-Based Research: Assessing Partnership Approaches to Improve Public. Annual Reviews 19:173-202.
9. James, J (2012) “Health Policy Brief: Public Reporting on Quality and Costs,” Health Affairs, http://healthaffairs.org/healthpolicybriefs/brief_pdfs/healthpolicybrief_65.pdf Joy M, Little E, Maruthur N, Purnell T, Bridges J (2013) Patient preferences for the treatment of type 2 diabetes: A scoping review, Pharmacoeconomics, Under review.
10. Luft H (2012) “Advancing Public Reporting through a New ‘Aggregator’ to Standardize Data Collection on Providers’ Cost and Quality,” Health Aff (Millwood). 31: 619–26.
11. Mehrotra A, Hussey PS, Milstein A, Hibbard JH. (2012) Consumers' and providers' responses to public cost reports, and how to raise the likelihood of achieving desired results. Health Affairs. 31(4):843-51.
12. NCQA (2013) Helping Healthcare Consumers Use Quality and Cost Information. http://www.ncqa.org/portals/0/Public%20Policy/CHCF%20ValueJudgmentQualityCostInformation.pdf
13. NCSL (2013) National conference of State Legislatures. Available at: http://www.ncsl.org/
14. O’Neil S, Schurrer J, Simon S (2010) Environmental Scan of Public Reporting Programs and Analysis, Final report, Sept, 30. Mathematica Policy Research Inc. #06738.500. Cambridge, MA.
15. Sinaiko AD, Rosenthal MB. (2011) Increased price transparency in healthcare—challenges and potential effects. N Engl J Med; 364(10):891–4.
16. Sinaiko AD, Eastman D, Rosenthal M. (2012) How report cards on physicians, physician groups, and hospitals can have greater impact on consumer choices. Health Affairs. 31(3):602-611.
17. Smith JA (2008) Qualitative Psychology: A Practical Guide to Research Methods (2nd Ed). Sage: London.
18. Smith JA, Flowers P, Larkin M (2009) Interpretative Phenomenological Analysis: Theory, Methods and Research. Sage: London.
19. Sofaer S. (2011) Engaging consumers with a high value healthcare system: public reporting of cost and resource use. Presentation at: Aligning Forces for Quality National Meeting; 2011 May; Denver, CO. http://forces4quality.org/sites/default/files/Set3b_DataDisplay_Sofaer.pdf
20. RWJF (2013) Comparing Healthcare Quality: A National Directory, Version 2. The Robert Wood Johnson Foundation. Available from: http://www.rwjf.org/en/research-publications/find-rwjf-research/2013/09/national-directory.html?cid=XEM_A7678
21. Viswanathan M, Ammerman A, Eng E, Gartlehner G, Lohr K, Griffith D, et al. Community-Based Participatory Research. (2004). RTI International-University of North Carolina Agency for Healthcare Research and Quality. Retrieved from: http://www.ahrq.gov/clinic/epcsums/cbprsum.htm.
22. Yegian JM, Dardess P, Shannon M, Carman KL. (2013) Engaged patients will need comparative physician-level quality data and information about their out-of-pocket costs. Health affairs. 32:328-337

Definition of Terms

Public reporting of cost data: Data on healthcare costs of providers or facilities that are publicly available to a broad audience of consumers (either free of charge, at a nominal cost, or granted based on group affiliation) that allow for comparisons within a defined geographic area.

Consumer: Any actual or potential recipient of healthcare services and their families or advocates who act on their behalf.

Cost measure: A financial measure of cost, charge, reimbursement, payment, or out-of-pocket expenses associated with a visit to a healthcare provider or facility.

Summary of Protocol Amendments

In the event of protocol amendments, the date of each amendment will be accompanied by a description of the change and the rationale.

Key Informants

Within the Technical Brief process, key informants serve as a resource to offer insight into the clinical context of the technology/intervention, how it works, how it is currently used or might be used, and which features may be important from a patient or policy standpoint. They may include clinical experts, patients, manufacturers, researchers, payers, or other perspectives, depending on the technology/intervention in question. Differing viewpoints are expected, and all statements are crosschecked against available literature and statements from other key informants. Information gained from key informant discussions is identified as such in the report. Key informants do not do analyses nor contribute to the writing of the report and will not review the report, except as given the opportunity to do so through the public review mechanism.

Key informants must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals invited to serve as key informants who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer Reviewers

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and will be published three months after the publication of the Evidence Report.

Potential Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

Appendixes

Appendix 1: Key informants discussion guide

The Evidence-based Practice Center at the Johns Hopkins University is currently developing a Technical Brief for the Agency for Healthcare Research and Quality (AHRQ) entitled “Public Reporting of Cost Measures in Healthcare”. There are three primary guiding questions the Technical Brief aims to address:

1. What measures of costs about healthcare providers and facilities have been publicly reported?
2. Are the measures of costs that are being reported consumer-centered?
3. What are the intended and unintended consequences emerging about consumers’ use of publicly reported cost data?

Our engagement with key informants will guide how we identify and synthesize the data from a review of actual public reporting practices and targeted literature review. This first call is targeted at the first of these questions: How do we identify current practices for the public reporting of cost measures? The second call is targeted at the second question.

To aid discussion, three definitions have been developed:

Public reporting of cost data: Data on healthcare costs of providers or facilities that are publicly available to a broad audience of consumers (either free of charge, at a nominal cost, or granted based on group affiliation) that allow for comparisons within a defined geographic area.

Consumer: Any actual or potential recipient of healthcare services and their families or advocates who act on their behalf.

Cost measure: A financial measure of cost, charge, reimbursement, payment, or out-of-pocket expenses associated with a visit to a healthcare provider or facility.

Agenda for discussion:

1. Introductions and overview of the project
2. Review of the definitions
3. Identifying actual practices for the public reporting of cost data
4. Identifying relevant literature
5. Methods for assessing consumer-centeredness
6. Identifying intended and unintended consequences
7. Methods for data abstraction and synthesis
8. Possible structure of the report