Powered by the Evidence-based Practice Centers
Evidence Reports All of EHC
Evidence Reports All of EHC



A Framework for Assessing Quality Indicators for Cancer Care at the End-of-Life

Research Report May 13, 2010
Download PDF files for this report here.

Page Contents

Author Affiliations

Hsien Seow, Ph.D.a
Claire F. Snyder, Ph.D., M.H.S. a
Richard A. Mularski, M.D., M.S.H.S.b
Lisa R. Shugarman, Ph.D.c
Jean S. Kutner, M.D., M.S.P.H.d
Karl A. Lorenz, M.D., M.S.H.S.e
Albert W. Wu, M.D., M.P.H.a
Sydney M. Dy, M.D., M.Sc. a

a Johns Hopkins Bloomberg School of Public Health, Baltimore, MD.
bThe Center for Health Research, Kaiser Permanente Northwest, Portland, OR.
cRAND Corporation, Santa Monica, CA.
dUniversity of Colorado School of Medicine, Denver, CO.
eVA Greater Los Angeles Healthcare System, Los Angeles, CA.



Patients with advanced cancer often do not receive high-quality pain and symptom management or support with coordination of care, communication, and decision-making. Implementing quality indicators which are reflective of the scope of care, feasible to implement, and supported by evidence might help to identify areas and settings most in need of improvement. However, recent reviews and policy initiatives identified only a few indicators that met these criteria. In order to help advance quality indicator development and implementation in this area, we developed a conceptual framework based on previous related initiatives, updated reviews of end-of-life cancer quality indicators and relevant data sources, and expert input. The framework describes five steps for developing and assessing a quality indicator for end-of-life care, defining the (1) population of focus; (2) broad quality domains; (3) specific target areas; (4) steps of the care process; and (5) evaluation criteria for quality indicators. The defined population includes seriously or terminally ill cancer patients, who are unlikely to recover or stabilize, and their families. Domains include the structure and processes of care; physical, psychiatric, psychosocial, spiritual, and cultural aspects of care; as well as the care of the imminently dying, ethical legal issues, and the delivery of care. Evaluation criteria include importance; scientific acceptability, including validity, evidence to improve outcomes, reliability, responsiveness, and variability; usability; and feasibility, including ready data sources. By using this conceptual framework, indicator developers, researchers, and policymakers can refine and implement indicator sets to effectively evaluate and improve care at the end of life.



The lack of readily available data on quality of care for patients with advanced cancer has been a major barrier to improving palliative and end-of-life care, which is care focusing on quality of life, communication, and decision-making for patients with advanced disease.1,2 Such data requires common usage of quality indicators or quality measures. A quality indicator or quality measure is an agreed-upon process or outcome measure that is used to assess quality of care,3 specified with a numerator and denominator to indicate the intended population, recommended care, and exclusions. Quality indicators can be classified as pertaining to structure (the environment in which health care is provided), process (the method by which health care is provided), and outcome (the consequences of health care).4 Measuring and improving the quality of cancer end-of-life care requires quality indicators that are reflective of the domains of quality cancer care, feasible to implement, and supported by experts and research evidence. Indicators should also be responsive to outcomes and link to valued health outcomes for patients and families.

Due to the growing need to address deficits in palliative and end-of-life care for cancer patients, the National Cancer Institute and other Federal agencies initiated the Cancer Care Quality Measurement Project (CANQUAL). CANQUAL reviewed existing quality standards for cancer symptom management and advance care planning through a National Quality Forum (NQF) call for measures and an accompanying systematic review.5,6 This process, however, identified only a handful of indicators with sufficient scientific evidence or field testing in cancer populations to meet NQF criteria. The NQF process highlighted challenges in developing and implementing indicators specific to end-of-life care.3 Although recent initiatives such as the National Consensus Project (NCP)7 and NQF Palliative Care Framework8 have defined clinical domains for palliative care, these domains must be translated into measurable aspects of quality in end-of-life care and the dying experience. Unlike in some clinical areas (e.g. cardiovascular care), there are few randomized trials to support quality indicators for end-of-life care, and quality measurement often depends on care that may be poorly documented (e.g. the hospice medical record). Indicators in end-of-life care may thus need to rely more on expert opinion and may need more evaluation for reliability in measurement and validity for quality improvement compared to quality indicators in other clinical areas. Defining the population or denominator to which indicators apply may also be problematic in end-of-life care. All the indicators approved by the NQF defined the denominator retrospectively from the time of death, making it difficult to translate into information that can prospectively guide quality improvement. Independent initiatives to develop quality indicators relevant to particular care settings (e.g. outpatient oncology, hospice) have not been well-coordinated, making combining or comparing results across settings difficult. Given these challenges, a framework conceptualizing quality end-of-life cancer care and informing the development and evaluation of quality indicators could advance the field of cancer quality measurement and improvement.

The National Cancer Institute and the Agency for Healthcare Research and Quality requested a process to develop such a framework. To develop the framework, we built on the previous initiatives described above, updated reviews of existing indicators6,9 and data sources,1,10 and obtained input from experts through a national symposium. This paper presents a broad framework for assessing quality indicators for end-of-life care in cancer patients. The framework is intended for use by indicator developers, researchers, policymakers, and quality measurement and improvement programs. It endeavors to facilitate the development, assessment, and implementation of quality indicators. Using this framework, we can more consistently and effectively measure and improve the quality of cancer end-of-life care.


Framework Overview

Figure 1 displays the proposed framework for assessing cancer quality indicators for end-of-life care. The framework focuses on the "end-of-life" stage in the National Cancer Institute’s cancer continuum of care (i.e. prevention, screening, diagnosis, treatment, survivorship, and end-of-life).11 Developing a framework focusing specifically on end-of-life care is useful because of its place within this continuum and the pronounced and distinct physical and psychosocial burdens faced by patients and their families during this period of illness. Indicators are also needed for quality issues unique to the end of life, such as overuse of chemotherapy or the specifics of receiving care in hospice.

The use of the term end-of-life is intended to emphasize key issues in this time period and emphasize symptom management and quality of life, but does not exclude the relevance of these domains to other parts of cancer care. Identifying quality indicators specific to cancer patients is important because some issues, such as the benefits and burdens of providing chemotherapy, radiotherapy, and cancer surgery at the end of life are best addressed in a cancer-specific framework. However, since end-of-life care is often provided across diseases and many patients have multiple comorbidities at the end-of-life, isolating cancer issues is difficult and often impractical. Therefore the framework is purposefully broad so as to include the range of issues cancer patients might face at the end of life, while not neglecting patients with comorbidities or facing these issues at other points in the cancer continuum. The framework should also be applicable, with modifications, to other serious and terminal diseases or more general palliative care populations.

The framework addresses quality indicators, which are population-level measures expressed using rates, as opposed to measurement tools such as a visual analog scale for pain, which collect data at the individual patient level. For example, while pain on the individual patient level could be assessed with a visual analog scale, the corresponding quality indicator would assess the rate of pain screening in the specified population. Indicators also specify data elements and instructions for data collection, timing, and descriptions of data analysis and reporting. They are intended to be used routinely, can provide feedback to guide short-term quality improvement or longer-term restructuring of care, and generally draw on administrative, chart, or electronic medical record sources, or data collected directly from patients and families (e.g. via survey). Quality indicators should be based on evidence or expert opinion, supported by constituent groups, reported publicly, and amenable to feedback from health care systems and professionals. Ideally, a set of quality indicators should cover all eligible patients and apply to multiple levels of the health care system.

The framework presented here identifies five steps for developing and assessing a quality indicator for end-of-life care, defining the (1) population of focus; (2) broad quality domains; (3) specific target areas; (4) steps of the care process; and (5) evaluation criteria of quality indicators. That is, the first step identifies who to measure, steps 2-4 identify what to measure, and step 5 assesses how well the indicator works. The framework addresses a broad range of quality indicators, system-level issues (general structure and processes of care) as well as person-level concerns (e.g., patient and family quality of life). The framework can be applied to a specific quality indicator or more broadly to an entire target area of quality indicators, such as quality indicators about pain management. In areas where there has been more active indicator development, the framework can help to identify barriers that might prevent widespread adoption or implementation of the indicators; for areas without well-developed quality indicators, the framework can help guide indicator development. Ultimately, the framework can help guide indicator developers and researchers in understanding where quality indicators are needed and how their quality indicator fits within the broader definition of quality end-of-life cancer care. Below, we discuss each of the steps in the framework.


Population of Focus

The framework’s first step defines the relevant population and denominator for end-of-life quality indicators. The expert consensus was to adapt the NCP palliative care population definition7 to specifically focus on cancer patients at the end of life and to expand the definition to include the family. The revised population definition includes "seriously or terminally ill cancer patients, who are unlikely to recover or stabilize, and their families." We recognize the challenges in using any of these terms, particularly "terminally ill", because of the lack of specificity and agreement on definitions as well as clinicians’ reluctance or inability to define patients as at the end of life. For certain cancer populations, "seriously ill" or other terms such as "living with advanced cancer" or "at significant risk of dying" might be more appropriate.

Defining the "end-of-life" population poses inherent and practical challenges, including underdocumentation and the uncertainty of prognosis. Quality issues relevant at the end of life often also occur throughout the spectrum of cancer care, and for some domains, the population may be better defined by needs than by prognosis. Moreover, the most appropriate denominator for an indicator may vary by setting, the domain of interest, or cancer type. Quality issues focus on individual settings or across multiple settings where end-of-life cancer care occurs, including primary care, community oncology practices, hospitals, and hospice, making the population of focus very complex to define explicitly. Therefore, the framework purposely uses a broad end-of-life definition so as to encompass the large range of quality issues, rather than to define the population with adequate specificity for data collection. As such, a critical first step in using this framework for a particular domain or project would be to further specify how this broad consensus definition can be applied to the setting and purposes of interest and an appropriate and definable population for measurement. For example, the population of focus might include all patients from the time of cancer diagnosis when measuring quality of communication, but only include patients at high risk of dying (such as those with widely metastatic disease and poor function or no longer eligible for chemotherapy) when measuring receipt of hospice care.


Broad Quality Domains

Quality end-of-life care has many dimensions; categorizing quality into broad domains can help focus measurement efforts. All domains generally relate to health-related quality of life, communication, and decision-making, as described in recent reviews of the qualitative and quantitative literature relevant to end-of-life care.12,13 The framework builds on the broad domains of quality palliative care endorsed by the NCP and NQF, which have been widely recognized and circulated.7,8 However, the domains from these two seminal documents are not cancer-specific and were written for the purposes of defining clinical guidelines (NCP) and describing preferred practices (NQF) for hospice and palliative care programs. The proposed framework therefore adapts these more general domains and applies them to quality indicators for end-of-life care focusing on but not exclusive to cancer. Specifically, we reorganized two of the domains into Psychiatric and Psychosocial to fit best with the cancer context based on a related Institute of Medicine (IOM) report.14 We also added a domain, Delivery of Care, to capture issues relevant to measuring quality at a population or healthcare system level, such as utilization of technology.

Existing quality indicator sets include indicators that fit within the framework’s domains, depending on the purpose, setting, and approach for the set. Some indicator sets are more comprehensive or apply to multidisciplinary care settings and include most of the domains; others are more oriented towards physicians or areas where there is an evidence base. For example, domains where extensive indicator development has occurred, such as the physical symptoms domain, are often included in most indicator sets; others, such as spirituality, are more challenging to support with evidence or expert opinion. Some quality indicators may also fit into or combine aspects of care from more than one domain.


Specific Target Areas

Within each broad domain of quality end-of-life cancer care, the next column in Figure 1 lists specific target areas for indicators of quality end-of-life care, though a variety of other potential target areas could be included within domains. While various quality indicator sets define these target areas and classify them within domains differently,15 we defined the areas listed in the framework to fit best with cancer-specific indicator sets. Examples of quality indicators within each target area are listed in Table 1. Given the earlier discussion on defining the population for the framework, clearly many of the symptom indicators are more important in patients with advanced cancer or near the end of life given increased symptom prevalence and severity, but not necessarily specific to this population. Similarly, many of the indicators related to communication and prognosis could apply throughout the cancer continuum, but have particular importance when the risk of dying is high. Indicators about aggressiveness of care at the end of life (such as chemotherapy in the last two weeks) are specific to this population, although they relate to broader quality issues about the appropriate use of chemotherapy.


Steps of the Care Process and Outcomes

Quality indicators may also target a specific step in the care process or a patient outcome. We have adapted the steps developed by the QA (Quality Assessment) Tools project and used in the Cancer-Quality-ASSIST project.15 These process steps include screening, assessment/diagnosis (usually conceptualized as a more detailed evaluation of those with a positive screen), treatment/implementation (implementation for target areas, such as communication, where the concept of treatment does not apply), and follow-up (where follow-up applies to whether the provider checks on the results of the intervention with the patient and/or family). In addition to these steps, the expert input suggested care coordination should be added due to the multidisciplinary and multi-site nature of cancer care. We also combined patient outcomes in the follow-up category.

While some quality indicator sets explicitly categorize indicators by these steps, some also use a comprehensive approach, defining the series of structural issues, processes, and outcomes that need to occur for quality care.4 Adjusting the structure of care can also be conceptualized as a step in the care process; setting a staffing policy or system for routine pain screening, for example, provides the basis for a pain screening quality indicator. However, the structure of care should not be overemphasized, as it leads to improved outcomes only indirectly through accompanying improved processes in care. Moreover, quality indicators may focus on the transitions between steps in the care process, such as the time between diagnosis and treatment, and the transfer of information and care responsibilities between different providers and care settings. Considering the steps of the care process can help contextualize quality measurement in the delivery of end-of-life care, helping identify when the documentation will occur and by whom, which is crucial for widespread implementation of quality indicators.


Evaluation Criteria of Quality Indicators

The framework’s final step evaluates the quality indicators’ appropriateness for use for quality improvement and accountability. The framework adapted and expanded upon the current NQF criteria used to evaluate quality indicators: importance, scientific acceptability, usability, and feasibility.16 All four criteria (Box 1) are applicable to and should be considered by indicator developers, researchers, quality improvement programs, and policymakers. The first criterion, importance, denotes whether an indicator addresses a domain, target area, and step in care that is important. That is, does expert consensus and/or evidence support that the indicator addresses a critical component in care, affects outcomes, and has room for improvement. Ideally, evidence should exist that performance can practically be improved; this may be supported by studies showing that some institutions perform better on the indicator. The concept of importance informs inclusion of indicators for quality improvement or reporting programs and provides evidence to support prioritization of further research.


Box 1.

Evaluation criteria (adapted from NQF)16

Importance: The extent to which the indicators capture key aspects of care that require improvement.
Scientific Acceptability The degree to which the indicators produces consistent and credible results when implemented. This includes aspects such as:

  • Validity: The degree to which the indicators accurately reflect the quality of care.
  • Evidence to improve outcomes: Evidence that changes as measured by the indicators lead to changes in outcomes.
  • Reliability: The extent to which the indicators can measure the construct consistently.
  • Responsiveness: The indicators' ability to detect change from interventions.
  • Variability: The extent to which the indicators' scores demonstrate sufficient variability between populations for useful comparisons.

Usability: The extent to which intended audiences (e.g. consumers, clinicians) can understand the results of the indicators and are likely to find them useful for decision-making.
Feasibility: The way in which data can be obtained within the normal flow of clinical care and the extent to which the indicators can be implemented.

  • Ready data sources: The extent to which the data required can be assessed or determined from available data sources or measurement tools.

Scientific acceptability includes the concepts of validity and reliability. Most indicators have some evidence of face validity based on expert opinion, and some indicators have gone through formal expert face validity rating processes. To date, none of the candidate indicators has been evaluated for validity in accurately reflecting what occurred in the patient encounter, or for evidence to improve outcomes. Indicator sets sometimes provide evidence from the medical literature, such as studies demonstrating that there are improvable relationships between a process and an outcome, or that a treatment is evidence-based. However, more work in this area is needed, particularly relevant to cancer end-of-life care. Reliability depends on various factors, including objectively defined concepts, precise specifications, and guidance for record abstraction; several of the indicator sets have been or are undergoing reliability testing through medical record audits.

The concept of feasibility depends on the availability of quality data that can be obtained or extracted without undue burden, and usability applies to how the results of the indicator can be applied. For example, an indicator addressing chemotherapy use at the end of life may be challenging to measure in settings or databases where deaths are not tracked, as is frequently the case; and applying the results of this outcome indicator to improving quality may be challenging, as there may be disagreement about whether it represents poor quality of care.


Applying the Framework

As an example, we applied the framework to reviewing the state-of-the-science in cancer quality indicator development for the psychosocial domain, including psychological and social services and interventions as described in an Institute of Medicine report.14 Specific target areas within this domain included support with emotional coping, practical and educational needs, and life, work, and financial burdens. Steps of the care process included identifying patients with psychosocial needs, linking them to appropriate services or providing support, following up on care delivery, and coordinating psychosocial with biomedical care.

Review of available indicators found that most were not well-defined and met few of the evaluation criteria. Screening and Assessment/Diagnosis indicators addressed routine psychosocial assessment, whether patients were assessed for emotional and financial stress, and whether caregiving burden and assistance needs had been assessed for patients who died. Some evidence suggests that addressing caregiver burden may have positive effects.17 Multicenter evaluation of a nonspecific indicator for the University Health Consortium on whether a formal psychological assessment by a psychologist, social worker, psychiatrist, or other expert was completed within 1 year prior or 4 days after hospital admission found that only 26.2% of eligible cases (range 0-95% by hospital) met the criteria.18

Treatment indicators included the presence of a psychosocial intervention and whether social resources were organized for patients without family or friends. In a multicenter internal pilot evaluation of a general intensive care indicator of whether psychosocial support (social work) was offered within 3 days of admission, only 61% of cases met the indicator.19

Patient outcomes/Follow-up indicators included whether patients and/or families were satisfied with psychosocial care. In the Family Evaluation of Hospice Care (FEHC) from the NHPCO (National Hospice and Palliative Care Organization), completed by families after hospice patients’ deaths (in a general hospice population including patients with cancer and other illnesses), 18.2% of family members noted problems with attending to family needs for support (variation among hospices, 25th and 75th percentiles, 12.6% and 21.4%), and 9.8% of family members reported unmet need for emotional support (25th and 75th percentiles among hospices, 5.4% and 13.3%).20 These indicators are based on a psychometrically evaluated tool, the After-Death Bereaved Family Interview, which has shown variability in outcomes by setting, although it has not been evaluated for responsiveness to quality improvement.



This article presents a framework for developing and evaluating indicators of quality of cancer care at end of life. Indicator developers can use the framework to select, categorize, and assess their indicators. The framework may also inform quality programs seeking to choose indicators and policymakers searching for priority areas where further evidence is needed.

Important practical barriers remain to implementing end-of-life cancer care quality indicators. Identifying the relevant population is challenging, in part due to the difficulties in predicting the end of life period. Although the framework is defined as end-of-life, end-of-life issues may be relevant from the time of diagnosis, as in the NQF and NCP frameworks. The framework is purposefully broad so as to include quality domains particularly relevant at the end of life, such as patient-centered care or communication, but can also apply throughout the spectrum of cancer care. The framework aims to facilitate the development and widespread use of quality indicators for end-of-life cancer care; when end-of-life quality indicators become routinely reported, we hope the attention focuses on areas for improvement and addressing patient and family unmet needs instead of the definitions of palliative versus end-of-life care.

Other barriers include limited data sources, especially where indicators depend on precise documentation of issues such as communication, patient-reported outcomes, or preferences. Many indicators depend on burdensome medical record abstraction. Some indicator sets are retrospective from the time of death, which limits their use prospectively or in settings where this information may not be clearly available. Initiatives such as electronic health records can be implemented and organized to facilitate the needed documentation and collate information on quality indicators. For example, electronic health records can expand billing coding to reflect quality issues, incorporate patient-reported information directly into the care process or health record, and include more relevant information in registries may all help to facilitate the needed data collection.

The proposed framework has a number of limitations. Quality measurement is an ever-changing area, with rapid advances in the science and an evolving and fluctuating policy context. This framework will need to be updated periodically, just like practice guidelines and indicator sets, to reflect shifts in the policy environment and advances in indicator development, the science supporting them, and the evidence base for cancer end-of-life care. The domains, target areas, and steps in care are not of equal importance and may not equally apply to all situations. Future research will need to prioritize and further develop the most important quality indicators by setting, population, or clinical circumstance. In addition, there is demand for valid, aggregate indicators of the quality of end-of-life care, although this is beyond the scope of the current project. Development of these measures would extend beyond individual indicators of specific aspects in the process of care to meaningful aggregated assessments that are related to important outcomes. This will require determining appropriate weights for individual components, as well as considering potential interactions among them.

The framework focuses on processes of care, but indicators that are oriented toward "never events," or unacceptable care that should never occur under any circumstances, are also relevant in some cases. The framework also does not address confounding factors that may affect the results of quality measurement, such as differences in the population or quality in the other settings where the patient previously received care. The evaluation criteria adapted from NQF do not specifically address generalizability. This is a critical concept in cancer end-of-life care, since these patients are treated in many different settings. Due to efforts to keep the framework straightforward and relatively simple, it does not explicitly address the wide variation in populations and settings, which can strongly impact many aspects of indicator development and use. For example, some indicators or measurement initiatives may work best when applied to all cancer patients in a setting; in other cases, screens for advanced disease or risk factors may be helpful to identify eligible patients. Moreover, indicators that work well in one setting may not in others due to differences in populations or systems, and supporting evidence may not translate across settings. When an indicator set has been developed for a specific setting or population, evaluation may be needed to determine whether the indicator can be applied to other populations or settings. An important future direction will be to evaluate which indicators are most important in specific settings, cancer populations, and specific clinical circumstances across the continuum of disease.



Patients diagnosed with advanced cancer sometimes do not receive adequate information, coordination of care, or control of pain and other symptoms. Many indicators are currently available, but much more coordination, rigorous evaluation, and further development and supporting evidence are needed to advance the field and make end-of-life indicators an accepted part of measuring the quality of cancer care. By using this conceptual framework, indicator developers can continue to work toward building indicators that are scientifically acceptable and reflect the scope of cancer end-of-life care, expanding the evidence to support their use, and refining effective and efficient ways to use indicators to evaluate and improve care at the end of life.



The authors wish to acknowledge Christine Weston, PhD, MSEd, Eric B. Bass, MD, MPH, Craig C. Earle, MD, Stephen Connor, PhD, R. Sean Morrison, MD, Thomas J. Smith, MD, and Joan Teno, MD, MS for their input into the development process and framework.


1. Institute of Medicine. Improving palliative care for cancer. Foley KM, Gelband H, eds. 2001.

2. Institute of Medicine. Enhancing data systems to improve the quality of cancer care. Hewitt M, Simone JV, eds. 2000.

3. National Quality Forum. Quality of cancer care measures’ project symptom management/end-of-life cancer technical panel. 2006.

4. Donabedian A. The quality of care. How can it be assessed? JAMA 1988;260(12):1743-1748.

5. Lorenz K, Lynn J, Dy SM, et al. Cancer care quality measures: symptoms and end-of-life care. Evid Rep Technol Assess (Full Rep) 2006;137:1-77.

6. Lorenz KA, Lynn J, Dy SM, et al. Quality measures for symptoms and advance care planning in cancer: a systematic review. J Clin Oncol 2006;24(30):4933-4938.

7. National Consensus Project for Quality Palliative Care. Clinical practice guidelines for quality palliative care. 2004: Pittsburgh, PA.

8. National Quality Forum. A national framework and preferred practices for palliative and hospice care quality. 2006: Washington, DC.

9. Carolinas Center for Medical Excellence. Palliative care quality measurement project. Medicare quality improvement community (MedQIC - Hospice). Available at: http://www.medqic.org. 2008.

10. Gotay C, Lipscomb J. Data for cancer outcomes research: identifying and strengthening the empirical base. In: Lipscomb J, Gotay CC, Snyder C, eds. Outcomes assessment in cancer: Measures, methods, and applications. Cambridge University Press: Cambridge, United Kingdom. 2005.

11. Epstein RM, Street Jr. RL. Chapter 4: Key Communication Tasks and Outcomes: The Cancer Care Continuum. In: Epstein RM, Street Jr. RL, eds. Patient-Centered Communication in Cancer Care: Promoting Healing and Reducing Suffering. National Cancer Institute, NIH Publication No. 07-6225: Bethesda, MD. 2007: 68-88.

12. Dy SM, Shugarman LR, Lorenz KA, Mularski RA, Lynn J. A systematic review of satisfaction with care at the end of life. J Am Geriatr Soc 2008;56(1):124-129.

13. Lorenz KA, Lynn J, Dy SM, et al. Evidence for improving palliative care at the end of life: a systematic review. Ann Intern Med 2008;148(2):147-159.

14. Institute of Medicine. Cancer care for the whole patient: Meeting psychosocial health needs. 2008. Available at http://www.iom.edu.

15. Lorenz K, Dy SM, Naeim A, et al. Quality measures for supportive cancer care: The cancer quality-ASSIST (Assessing Symptoms Side Effects and Indicators of Supportive Treatment) project. In press, J Pain Symptom Manage.

16. National Quality Forum. Measure Evaluation Criteria. 2008: Washington, DC.

17. Lorenz KA, Rosenfeld K, Wenger N. Quality indicators for palliative and end-of-life care in vulnerable elders. J Am Geriatr Soc, 2007. 55 Suppl 2: p. S318-26.

18. Twaddle ML, Maxwell TL, Cassel JB, et al. Palliative care benchmarks from academic medical centers. J Palliat Med 2007;10(1):86-98.

19. Nelson JE, Mulkerin CM, Adams LL, et al. Improving comfort and communication in the ICU: a practical new tool for palliative care performance measurement and feedback. Qual Saf Health Care 2006;15(4):264-271.

20. Connor SR, Teno J, Spence C, et al. Family evaluation of hospice care: results from voluntary submission of data via website. J Pain Symptom Manage 2005;30(1):9-17.

21. Earle CC, Neville BA, Landrum MB, et al. Evaluating claims-based indicators of the intensity of end-of-life cancer care. Int J Qual Health Care 2005;17(6):505-509.

22. Jacobson JO, Neuss MN, McNiff KK, et al. Improvement in oncology practice performance through voluntary participation in the Quality Oncology Practice Initiative. J Clin Oncol 2008;26(11):1893-1898.

23. McNiff KK [personal communication], Summary of QOPI measures. Spring 2008.


Table and Figure


Table 1. Example end-of-life quality indicators in relation to the proposed framework steps

Domain Target area Indicator Step of care Evaluation criteria applied and results Indicator set, reference
Multiple Policy/procedure specifying frequency with which pain and dyspnea should be assessed Screening None PEACE9
Multiple Percent of patients admitted to hospice or palliative care who have a screening for symptoms during the admission visit Screening None PEACE9
Physical Dyspnea Percent of hospitalized patients with use of an objective scale for documentation of severity of dyspnea Assessment Feasibility/usability: 38% of cancer patients met criteria, multiple hospitals UHC18
Psychiatric Depression IF depression is diagnosed in a cancer patient, THEN a treatment plan for depres-sion should be documented BECAUSE there are a variety of effective approaches to treating depression associated with cancer. Treatment Validity (face): expert panel review
Reliability, feasibility:
Cancer Quality-ASSIST15
Psychosocial Patient and family support Percentage of family members reporting unmet need for emotional support Outcome Feasibility/usability: NHPCO20
Spiritual, religious, and existential Spiritual Percentage of patients with documentation that spiritual support was offered to the patient/family within the first 72 hrs of admission to the intensive care unit Implementation Feasibility/usability: 38% of patients met criteria, multiple intensive care units VHA19
Cultural Cultural Provision of interpreter or translators for non-English-speaking or deaf patients (%) Implementation None PEACE9
Care of the imminently dying Prognosis Percentage of patients with chemotherapy in the last two weeks of life Outcome Feasibility/usability: 14% of patients met criteria, wide variation among multiple oncology practices
Reliability: results pending (Quality Oncology Practice Initiative, QOPI)
Dana Farber21, QOPI22
Ethical and legal Advance care planning Percentage of patients with advance directive status documented within1 day of intensive care admission Implementation Feasibility/usability: 72% of patients met criteria, multiple intensive care units VHA19
Delivery of care Communication / decision-making, chemotherapy Chemotherapy intent (palliative vs. curative) discussion with patient documented (%) Implementation Feasibility/usability: ongoing QOPI 200823


Figure 1. Framework for developing and assessing quality indicators for cancer care at the end-of-life

Figure 1 displays the proposed framework for assessing cancer quality indicators for end-of-life care. The following steps are shown: • Population of focus:(See reference 7 in reference list.) Seriously or terminally ill cancer patients who are unlikely to recover or stabilize, and their families. • Broad quality domains(See reference 7 and 8 in reference list.) and the specific target areas associated with them: Structure—Policies for care Process—Care delivery processes Physical—Pain, dyspnea, nausea/vomiting Psychiatric—Depression, anxiety, delirium Psychosocial—Patient and family support Spiritual, religious, and existential—Spiritual care, closure Cultural—Cultural sensitivity Care of imminently dying—Prognosis discussion and documentation Ethical and legal—Advance care planning Delivery of care—Communication and decisionmaking, continuity and coordination, utilization • Steps of the care process: Screening, assessment/diagnosis, treatment/implementation, patient outcomes/follow-up • Evaluation criteria of the quality indicators:(See reference 16 in reference list.) Importance, scientific acceptability (validity, evidence to improve outcomes, reliability, responsiveness, variability), usability, feasibility (ready data sources)

Journal Publications

Sharma RK, Dy SM. Documentation of information and care planning for patients with advanced cancer: associations with patient characteristics and utilization of hospital care. Am J Hosp Palliat Care 2011 Mar 30. [Epub ahead of print]

Project Timeline

Developing and Testing a Framework for Cancer Quality Measures for End-of-Life Care

Apr 14, 2010
Topic Initiated
May 13, 2010
Research Report
Page last reviewed January 2019
Page originally created November 2017

Internet Citation: Research Report: A Framework for Assessing Quality Indicators for Cancer Care at the End-of-Life. Content last reviewed January 2019. Effective Health Care Program, Agency for Healthcare Research and Quality, Rockville, MD.

Select to copy citation