Powered by the Evidence-based Practice Centers
Evidence Reports All of EHC
Evidence Reports All of EHC



Cancer Quality-ASSIST Supportive Oncology Quality Indicator Set: Feasibility, Reliability, and Validity Testing

Research Report ARCHIVED May 19, 2010
Download PDF files for this report here.

Page Contents

Note: This report is greater than 5 years old. Findings may be used for research purposes but should not be considered current.

Author Affiliations

Sydney M. Dy, M.D., M.Sc.a
Karl A. Lorenz, M.D., M.H.S.b
Sean M. O’Neillc
Steven M. Asch, M.D., M.P.Hb
Anne M. Walling, M.D.d
Diana Tisnado, Ph.D.d
Anna Liza Antonioe
Jennifer L. Malin, M.D., Ph.D.b

aJohns Hopkins Bloomberg School of Public Health, Baltimore, MD.
bVA Greater Los Angeles Healthcare System, RAND Health, David Geffen School of Medicine at UCLA, Los Angeles, CA.
cRAND Pardee Graduate School, Santa Monica, CA.
dDivision of General Internal Medicine and Health Services Research, UCLA, Los Angeles, CA.
eUCLA School of Public Health, Los Angeles, CA.



Purpose: Although measuring the quality of symptom management and end-of-life care could help provide a basis for improving supportive care for advanced cancer, few quality indicators in this area have been rigorously developed or evaluated.

Methods: We conducted a pilot evaluation of a comprehensive set of 92 supportive oncology quality indicators, Cancer Quality-ASSIST, including outpatient and hospital indicators for symptoms commonly related to cancer and its treatment and information and care planning. We operationalized the indicators and developed an electronic abstraction tool and extensive guidelines and training materials. Quality assurance nurses abstracted the medical record for 356 advanced cancer patients in two settings: a Veterans Administration hospital and an academic hospital and cancer center. We evaluated the indicators’ feasibility, interrater reliability, and validity.

Results: We successfully evaluated 78 indicators across the domains; results were similar in the two settings. We could not feasibly evaluate 3 indicators because of low prevalence; 22 indicators had significant interrater reliability issues, 9 had significant validity issues, and 3 had both reliability and validity issues, leaving a set of 41 indicators most promising for further testing and use in this population, with an overall kappa score of 0.85 for specified care.

Conclusion: Of 92 Cancer Quality-ASSIST quality indicators for symptoms, treatment toxicity, and information and care planning, 41 were sufficiently feasible, reliable and valid to be used for patients with advanced cancer in these settings. This set of indicators shows promise for describing key supportive care processes in advanced cancer.



Nearly half of cancer expenditures occur during the final year of life, which is often marked by severe patient and family suffering.1 Recently developed quality indicators approved by national organizations and frequently used in research focus on variation in2 and overutilization of3 costly, high-intensity care that may reflect poor quality. In a systematic literature review, we found that few tools to measure underutilization of evidence-based supportive cancer care have been available, rigorously evaluated, or widely used.4 To accurately assess quality of care and provide tools for targeting and evaluating quality improvement, quality indicators are needed with good reliability and validity, variation among settings, and sensitivity to change.5,6

The Cancer Quality-ASSIST (Assessing Symptoms Side Effects and Indicators of Supportive Treatment) Project developed evidence-based quality indicators (QIs) to evaluate supportive cancer care from medical records. ASSIST addresses pain, dyspnea, depression, nausea and vomiting, diarrhea, fatigue, and information and care planning, including symptoms related to cancer, common complications, and treatment-related toxicities. ASSIST QIs were based on systematic literature reviews and developed using the RAND/UCLA (University of California, Los Angeles) modified Delphi process, with multidisciplinary input and representation from oncology, geriatrics, and internal medicine professional societies.7-11

The objective of this study was to characterize the feasibility, reliability and validity of the ASSIST quality indicators for advanced cancer care in two health care settings with different patient populations and data systems, the Veterans Administration Greater Los Angeles Health Care System and the Johns Hopkins Hospital and Sidney Kimmel Comprehensive Cancer Center. The goal was to determine a common robust quality indicator set for future comparative studies and quality monitoring efforts.



Study Sites

We report here on two separate but coordinated pilot efforts that tested the ASSIST indicators in diverse settings and patient populations. A pilot evaluation at the Johns Hopkins Hospital (JHH) and Sidney Kimmel Comprehensive Cancer Center (SKCCC) focused on end-of life care and addressed the ASSIST indicators relevant to symptoms, communication, and care planning in patients with advanced cancer. A concurrent pilot at the Veterans Affairs Greater Los Angeles Health Care System (VAGLAHS) was broader in focus, addressing all the ASSIST supportive oncology domains (including toxicity of chemotherapy and radiation therapy) in patients with advanced disease in an integrated delivery system. Each site’s IRB approved the respective study protocol.

The JHH is a large academic medical center; patients undergoing chemotherapy and radiation therapy receive integrated care at the SKCCC. VAGLAHS is an integrated inpatient and ambulatory medical center and regional referral center for veterans with cancer requiring specialized services (e.g. radiation therapy). At the time of the study, the JHH Department of Medicine had a palliative care program, the SKCCC had a cancer pain program, and VAGLAHS had a palliative care program. JHH/SKCCC had a combination of several paper and electronic health records, and VAGLAHS used the VA integrated electronic health record. JHH/SKCCC also had supplemental data sources, including a cancer registry, a cancer center information system, and hospital utilization data.

Study Sample

Study inclusion criteria differed somewhat between the sites, based on local populations and study goals. At VAGLAHS, we used the cancer registry to identify patients diagnosed in 2006 with one of the following tumor types and stage IV disease: breast, colorectal, esophagus/stomach, genitourinary, head and neck, liver/biliary, lung, pancreas, and prostate. Additional inclusion criteria included being alive 30 days post diagnosis and having at least one outpatient visit at VAGLAHS following diagnosis.

At JHH/SKCCC, the cancer registry identified patients with stage IV lung, pancreatic, breast, and colorectal cancer (and Stage IIIB for lung cancer) disease in 2003-2005 who died 2-15 months after diagnosis. Additionally, patients needed at least three outpatient visits at Johns Hopkins or at least two outpatient visits and a hospitalization of at 3-30 days in length after the diagnosis. Most patients had at least three visits and a hospitalization, and only 20 had only outpatient care.

Quality Indicators

The ASSIST indicators address supportive care, symptom management, and information and care planning needs of cancer patients, and were developed following a systematic literature synthesis.7-10 A panel of multidisciplinary international leaders from oncology, palliative care, geriatrics, primary care, nursing, and social work evaluated the initial indicator set for validity and feasibility using the RAND appropriateness method, resulting in a revised set of 92 indicators. Indicators use an "If-then" statement, which is represented as a ratio with "if" as the denominator and "then" as the numerator:

Quality Indicator = # patients who received the specified intervention
# patients for whom the intervention was indicated

The numerator describes the care that should be provided. The denominator identifies the patient population to whom the care should be provided. For an individual patient, the indicator could have a value of "pass" or "not pass"; for the population, the range of values is 0-1. For example, the ASSIST quality indicator "IF a cancer patient is receiving a long-acting opioid for cancer pain, THEN he or she should also be prescribed a short-acting opioid for breakthrough pain," is expressed as

The number of patients receiving a short and long-acting opioid
The number of cancer patients receiving a long-acting opioid

Data Sources

We developed a detailed medical record abstraction instrument to collect the data elements for scoring each indicator and detailed abstraction guidelines, which were customized to the abstraction process used at each site, including a Microsoft Access-based abstraction tool. To balance the need to abstract detailed information and the time and cost of abstraction, each site specified abstraction windows for the different indicator domains. At JHH/SKCCC, nurses abstracted data from up to 3 cancer-related outpatient visits, including the first 2 visits and last visit. At VAGLAHS, nurses abstracted data from up to 3 months of visits (mean 10, SD 7) for symptom management and up to 12 months for information and care planning. For the patients’ last cancer-related hospitalization, both sites abstracted the first 3 days of the hospitalization for symptom management, and the entire hospitalization for information and care planning. At JHH/SKCCC, after abstracting 148 medical records, we concluded that we had an adequate sample to evaluate the reliability and validity for inpatient symptoms since these indicators were relevant to all patients, and we stopped abstracting this inpatient symptom data only for the remainder of the patients.

At VAGLAHS, following a week-long training and one-week pilot, three experienced nurse abstractors abstracted the electronic medical record (EMR) from the VA Computerized Patient Record System (CPRS), from June – September 2008. At JHH/SKCCC, following similar training, a nurse project manager and three professional nurse abstractors from Delmarva, a Quality Improvement Organization (QIO), abstracted records from July – September 2008. We held regular meetings with abstractors at each site and weekly conference calls with investigators from both sites to ensure shared understanding of the guidelines, and developed a shared abstraction FAQ (Frequently Asked Questions).

Assessment of Feasibility, Reliability and Validity

We assessed feasibility in three ways: during tool development, qualitatively during abstraction, and, during analysis, by evaluating how many patients were eligible for indicators. While developing the tool and guidelines, we excluded indicators at each site irrelevant to the setting or population or infeasible to operationalize or abstract. We modified several indicators at one or both sites to improve feasibility of abstraction and/or analysis. Abstractors took notes on feasibility, including abstraction time and difficulties in finding information or reliably determining a data element, and impressions of abstraction. Finally, we excluded indicators for low prevalence if too few patients were eligible by the denominator n (<3% at VAGLAHS and n=5 at the JHH/SKCCC).

To assess inter-rater reliability (IRR), all nurses at each site abstracted a sample of records (5%, n=8 at VAGLAHS and 4%, n=9 at JHH/SKCCC). At VAGLAHS, we assessed validity by physician implicit review (JLM or KAL) of records of cases that failed quality indicators. At JHH/SKCCC, we assessed validity by comparing nurse abstraction to a physician investigator (SMD) gold standard abstraction for the 9 record IRR sample, and assessed quality control by comparing nurses’ assessments and nurse-physician agreement.


We calculated observed agreement and the kappa statistic for indicator eligibility (denominator) and, conditional on eligibility, the care specified by the QI to evaluate IRR. Although no standard for minimum reliability of a quality indicator exists, high reliability is desirable if indicators are intended for accountability or quality improvement.12 We considered IRR to be inadequate when the kappa statistic was < 0.8 for eligibility (denominator) or < 0.6 for care specified by the QI (numerator).13 When possible, we modified initial indicator specifications to determine if reliability could be improved.



Patient characteristics are shown in Table 1; age and ethnicity were relatively similar across the 2 sites, but VAGLAHS patients were almost all male, included more types of cancer, and only half were decedents. Of the total set of 92 Cancer Quality-ASSIST indicators, we excluded three because they were duplicative of other indicators, and 9 because either the data elements were not routinely in the medical record or we concluded that they would require excessive abstraction time. We also excluded 3 indicators not applicable to the sample or setting (e.g. care during stem cell transplant). One included indicator (for pain screening) was split into 2 indicators (inpatient and outpatient). We therefore attempted to abstract 78 and 47 of the 92 quality indicators in the VAGLAHS and JHH/SKCCC study samples, respectively (the JHH/SKCCC study did not include chemotherapy or radiation therapy indicators). The mean abstraction time at JHH/SKCCC was approximately 2.5 hours for patients with a hospitalization and at VAGLAHS was 2.3 hours. Of the successfully evaluated indicators, 22 had inadequate reliability (kappa <0.8 for eligibility or <0.6 for adherence), 9 inadequate validity, and 3 both inadequate reliability and validity in these settings (examples in Table 2). Finally, we excluded 3 indicators that did not meet prevalence criteria in at least one site (8 indicators at JHH/SKCCC did not meet prevalence criteria).

The 41 indicators that met our criteria for feasibility, reliability and validity include 10 of the 15 indicators for pain, 2/5 for depression and psychosocial distress, 7/15 for nausea and vomiting, 2/5 for anorexia and weight loss, 5/8 for fatigue and anemia, 2/3 for diarrhea, 6/8 for dyspnea, 1/5 for delirium, 1 for rash, and 5 for information and care planning (Table 3). Overall kappa was 0.87 for eligibility and 0.86 for specified care; we were also able to calculate individual kappa scores for 9 of the indicators, which ranged from 0.73 – 1.0.



In this pilot study of the 92 medical-record based Cancer Quality-ASSIST supportive oncology quality indicators, following development and implementation of an abstraction tool, use of trained abstractors, and analysis of quantitative and qualitative abstraction results, including interrater reliability, 41 met strict criteria for feasibility, reliability and validity for advanced cancer across two clinical settings. These indicators represent all domains of the original ASSIST set except mucositis, insomnia, and fever/neutropenia.

Most quality indicators are adopted and published without rigorous development or evaluation despite evidence that these methods improve validity and risks and misallocation of resources from inappropriate implementation.1 Although several related medical record-based indicator sets addressing different domains or settings have been evaluated,14 including the PEACE Palliative Care Quality Measurement project for hospice and palliative care,15 ASCO (American Society for Clinical Oncology) QOPI (Quality Oncology Practice Initiative) for oncology outpatient care,16 and the VHA set for critical care,17 none has undergone this rigorous development and evaluation process. This study demonstrates the importance of evaluation, since only approximately half the ASSIST indicators, already rigorously developed through a systematic review and expert panel process, met criteria in these pilot tests.

Thirty-seven indicators did not meet our criteria for reliability (n=22), validity (n=9), both reliability and validity (n=3) or prevalence (n=3) in these settings. Most indicators did not meet criteria due to reliability issues. The subjective nature of symptoms and inconsistent documentation augment the challenge of obtaining clinical data manually from records. QIs assessing symptoms and communication are very different from those assessing discrete, concrete events like fecal occult blood testing followup by colonoscopy. For example, absence of symptom(s) was often documented with non-specific language, such as "Appears comfortable," limiting both reliability and validity. However, the indicators might perform differently in other settings. All of these criteria may be affected by documentation practices, practice patterns, site of care (e.g. inpatient vs. outpatient), as well as disease and patient characteristics (which may affect the prevalence of a condition). For example, obtaining records from prior sites of care such as nursing homes in order to assess whether an advance directive was appropriately communicated was not feasible with our settings or resources. However, a similar quality indicator was successfully implemented in a study of quality of care for vulnerable community-dwelling older patients, which obtained medical records from primary care and specialist providers, acute care hospitals, skilled-nursing facilities, and home health agencies.18

Different documentation practices can affect performance for indicators dependent on documentation. However, the Cancer-ASSIST indicator set includes only indicators where experts agreed that poor documentation by itself constituted poor quality of care, because lack of documentation could cause gaps in care processes or communication. For example, lack of documentation about care planning conversations may affect other providers’ ability to effectively address urgent end-of-life decisions. Nevertheless, we found that variation in documentation practices sometimes limited indicators’ reliability. For example, we could not reliably abstract newly-identified moderate to severe pain in ambulatory care in our settings. However, a recent study in patients dying in the hospital found that pain score documentation in inpatient standardized nursing notes was reliable for pain indicators.19 Electronic medical records (EMRs) hold promise for improving the feasibility of data collection for quality assessment, although much of the detailed clinical information in EMRs remains in free text fields which still require abstraction. Potential solutions include natural language processing, structured templates for documenting these issues, and obtaining symptom information directly from patients, potentially with automatic capture in a related database.

An ideal supportive care indicator system should include multiple approaches to quality assessment. Routine symptom screening and recording could improve capacity to evaluate outcomes. Some types of indicators difficult to operationalize in this study, such as symptom followup, may require additional collection of specific outcome data, similar to 48-hour pain followup in hospice.20 Carefully-evaluated available claims-based indicators can be helpful for regional quality evaluation or in databases (e.g., managed care) that cross clinical settings, although their focus is on aggressiveness of care.3 Issues such as communication and spiritual care are challenging to standardize in documentation, and indicators from the patient and family perspective may be needed for these domains and to evaluate the patient’s experience.21,22 Registry-based measures, such as in the CMS (Center for Medicare Services) PQRI (Physician Quality Reporting Initiative)23 or cancer registries, can be another method of measuring quality. These approaches vary in feasibility, cost of obtaining data, validity, usefulness at different points in the clinical trajectory, and how well they reflect current evidence. Advancing clinical evidence, changes in routine clinical data capture, and evaluation of efficient strategies that combine process-outcomes and different report perspectives (e.g, patient and family) will eventually define integrated quality assessment approaches.

This study has several limitations. Although the overall reliability of this indicator set is high, not all indicators were triggered frequently enough for full individual reliability or validity assessment. Further evaluation of these indicators requires better methods for identifying eligible patients (e.g., spinal cord compression), or these rarer quality issues may be better addressed using other methods, such as event reporting. The use of trained nurses experienced in abstraction may have increased reliability beyond what might be expected in community settings. Further refinement of the indicators, abstraction tool, and training guide, and focusing on the subset of indicators with acceptable reliability and validity, could potentially improve some indicators’ performance.

Documentation and the use of EMR use may also differ in other settings, which could either improve or limit these indicators’ performance depending upon the situation. These two urban academic centers shared many features, including relatively coordinated cancer care, a teaching environment and strong support services. However, there were also important differences, including their organizational structures (private vs. VA hospital), the patient populations they serve, and the presence of a strong hospital-wide palliative care program only at the VA at the time of the study, which may have affected the level of documentation. Despite these differences, reliability and validity results were similar across almost all indicators. Not all common malignancies were well-represented in this study (e.g., breast cancer), and further evaluation of these indicators in different cancer types is needed. Although the review times for this initial evaluation were lengthy, their purpose was to determine which indicators had the best potential for reliability and validity; abstraction with a smaller number of indicators would be less time-consuming.

In summary, this report presents the first evaluation of a comprehensive supportive oncology quality indicator set, developed for face validity with clinician experts through a rigorous process of systematic review and expert panel consensus, and evaluated for feasibility, reliability, and validity in two sites. This robust set of quality indicators can be used to evaluate the quality of supportive and end-of-life care for patients with advanced cancer and to identify areas for quality improvement efforts. Future research should evaluate which indicators are critical to efficient and effective quality measurement in these domains and have the greatest correlation with desired patient outcomes.



1. Lorenz KA. Promoting efficiency and quality in the Hummer health care system. J Clin Oncol 2007;26:5664-5665.

2. Wennberg JE, Fisher ES, Stukel TA, et al. Use of hospitals, physician visits, and hospice care during last six months of life among cohorts loyal to highly respected hospitals in the United States. BMJ 2004;328:607.

3. Earle CC, Landrum MB, Souza JM, et al. Aggressiveness of cancer care near the end of life: is it a quality-of-care issue? J Clin Oncol 2008;26:3860-3866.

4. Lorenz KA, Lynn J, Dy S, et al. Quality measures for symptoms and advance care planning in cancer: a systematic review. J Clin Oncol 2006;24:4933-4938.

5. McGlynn EA, Asch SM. Developing a clinical performance measure. Am J Prev Med 1988;4:14-21.

6. National Quality Forum. Measure Evaluation Criteria. Available at: http://www.qualityforum.org/docs/measure_evaluation_criteria.aspx.

7. Lorenz KA, Dy SM, Naiem A, et al. Quality Measures for Supportive Cancer Care: The Cancer Quality-ASSIST Project. J Pain Symptom Manage 2009;37(6):943-964.

8. Walling A, Lorenz KA, Dy SM, et al. Evidence-based recommendations for information and care planning in cancer care. J Clin Oncol 2008;26:3896-3902.

9. Dy SM, Asch SM, Naeim A, et al. Evidence-based recommendations for cancer pain management. J Clin Oncol 2008;26:3879-3885.

10. Dy SM, Asch SM, Naeim A, et al. Evidence-based recommendations for cancer fatigue, anorexia, depression, and dyspnea. J Clin Oncol 2008;26:3886-3895.

11. Naeim A, Dy SM, Lorenz KA, et al. Evidence-based recommendations for cancer nausea and vomiting. J Clin Oncol 2008;26:3903-3910.

12. Quality Measurement and Health Assessment Group (QMHAG), Center for Medicare and Medicaid Services. Measure evaluation criteria and subcriteria descriptions. Adopted from NQF Final Evaluation Criteria. Available at: http://www.hsag.com/mms.

13. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1997;33:159-174.

14. Pasman HR, Brant HE, Deliens L, et al. Quality indicators for palliative care: A systematic review. J Pain Symptom Manage 2008;38(1):145-156.

15. The Carolinas Center for Medical Excellence. Palliative Care Quality Measures Project (PEACE). Available at: http://www.qualitynet.org/dcs/ContentServer?c=MQParents&pagename=Medqic%2FContent%2FParentShellTemplate&cid=1228695715754&parentName=Topic.

16. Jacobson JO, Neuss MN, McNiff KK, et al. Improvement in oncology practice performance through voluntary participation in the Quality Oncology Practice Initiative. J Clin Oncol 2008;26:1893-1898.

17. Nelson JE, Mulkerin CM, Adams LL, et al. Improving comfort and communication in the ICU: a practical new tool for palliative care performance measurement and feedback. Qual Saf Health Care 2006;15:264-271.

18. Wenger NS, Solomon DH, Roth CP, et al. The Quality of Medical Care Provided to Vulnerable Community-Dwelling Older Patients. Ann Intern Med 2003;139:740-747.

19. Walling AM, Asch SM, Lorenz K, et al. Quality of end of life care in the hospital. J Gen Intern Med 2009;23:153-154.

20. Connor SR, Tecca M, Lund Person J, et al. Measuring hospice care: The National Hospice and Palliative Care Organization National Hospice Data set. J Pain Symptom Manage 2004;28:316-328.

21. Teno JM, Clarridge B, Casey V, et al. Validation of Toolkit After-Death Bereaved Family Member Interview. J Pain Symptom Manage 2001;22:752-758.

22. Finlay E, Shreve S, Casarett D. Nationwide veterans affairs quality measure for cancer: the family assessment of treatment at end of life (FATE). J Clin Oncol 2008;26:3838-3844.

23. American Medical Association Physician Consortium for Performance Improvement. Consortium Measures. Available at: http://www.cms.hhs.gov/pqri.



Table 1. Patient characteristics
  Johns Hopkins Veterans Affairs Greater Los Angeles
N N=238 N=118
Age, mean (SD) 60.5 (12.0) 65.9 (9.9)
Female 125 (52.5%) 2 (2%)
Race/Ethnicity Caucasian 155 (65%) 73 (62%)
African American 76 (32%) 36 (30%)
Other 7 (2%) 9 (8%)
Cancer Type Breast 6 (3%) 1 (1%)
Colorectal 28 (12%) 15 (13%)
Esophagus/stomach   10 (8%)
Genitourinary   8 (7%)
Head and neck   27 (23%)
Liver/biliary   4 (3%)
Lung 132 (55%) 27 (23%)
Pancreas 72 (30%) 6 (5%)
Prostate   20 (17%)
Died during study period 100% 55 (49%)


Table 2. Examples of quality indicators with significant issues with feasibility, reliability or validity in these settings

  Indicator Issue
Measure not feasible IF a patient with head and neck cancer is undergoing radiation treatments THEN midline radiation blocks and three-dimensional radiation treatments should be used Use of blocks not documented in radiation oncology notes
IF a patient with advanced cancer has an advance directive/DNR and the patient receives care in a second venue, THEN the advance directive/DNR should be present in the medical record at the second venue or documentation should acknowledge its existence, its contents, and the reason that it is not in the medical record Challenging to obtain records from different settings
Measure not reliable IF a patient is diagnosed with cancer, THEN s/he should be screened for depression within 1 month following the diagnosis Inadequate reliability for specified care (numerator, kappa=-0.29); variability in documentation for mood assessment
IF a patient with cancer has new fatigue, THEN there should be an assessment within 1 month of the initial documentation of fatigue for either insomnia or depression No patients eligible with original specification. Alternate specification dropping "new": inadequate reliability for indicator eligibility (denominator, kappa=0.3); variability in documentation indicating absence of fatigue
IF a patient is newly known to have advanced cancer after a surgery, diagnostic test, or physical exam, THEN a discussion including prognosis and advance care planning should be documented within 1 month or a reason why such a discussion did not occur Inadequate reliability for specified care (numerator, kappa=0.56); challenging to identify content of discussions from documentation
Measure not valid IF a hospitalized patient has a change in his/her pain regimen to treat severe, sustained cancer pain THEN there should be an assessment of whether or not the change in treatment reduced the pain within 4 hours Challenging to identify severe sustained pain, time medication administered and time of pain assessment. Time of documentation may not reflect the time of assessment
IF a patient with advanced cancer has documentation of dyspnea despite treatment with non-opioid medications or underlying causes, THEN they should be offered opioids within one month or there should be documentation of contraindications to opioid therapy Requires experienced clinician to distinguish dyspnea refractory to treatment with non-opioid medications; elements necessary to identify eligible patients often not well documented
IF a cancer patient has a positive screening for pain THEN the provider should assess the likely etiology/ies of the pain Etiology of pain often indirectly addressed rather than directly linked to pain evaluation, especially when related to cancer


Table 3. Quality indicators that met criteria for feasibility, reliability, and validity in these settings
Indicator Criteria
Only one indicator met criteria at JHH/SKCCC but not at VAGLAHS: IF a patient with cancer over the age of 65 or with advanced disease is admitted to the hospital THEN cognitive status should be evaluated within 48 hours of admission (100% agreement).
Pain If a cancer patient has a cancer-related outpatient visit then there should be screening for the presence or absence and intensity of pain using a numeric pain score
IF a cancer patient is admitted to a hospital then there should be screening for the presence or absence of pain
If a patient with cancer pain is started on a long-acting opioid formulation, then a short-acting opioid formulation for breakthrough pain should also be provided
If a patient with cancer pain is started on chronic opioid treatment then he/she should be offered either a prescription or nonprescription bowel regimen within 24 hours or there should be documented contraindication to a bowel regimen
If a patient's outpatient cancer pain regimen is changed, then there should be an assessment of the effectiveness of treatment at or before the next outpatient visit with that provider or at another cancer-related outpatient visit
If a patient has advanced cancer and receives radiation treatment for painful bone metastases then he/she should be offered single-fraction radiation OR there should be documentation of a contraindication to single-fraction treatment
If a cancer patient has new neurologic symptoms or findings on physical examination consistent with spinal cord compression then he/she should be treated with steroids as soon as possible, but within 24 hours or a contraindication to steroids should be documented
If a cancer patient has new neurologic symptoms or findings on physical examination consistent with spinal cord compression then a whole-spine magnetic resonance imaging (MRI) scan or myelography should be performed as soon as possible, but within 24 hours OR there should be documentation of why an MRI scan was not appropriate
If a cancer patient has confirmation of spinal cord compression on radiologic examination, then radiotherapy or surgical decompression should be initiated within 24 hours or a contraindication for such therapy should be documented
If a cancer patient is treated for spinal cord compression then there should be follow-up of neurologic symptoms and signs within 1 week after treatment is completed
Depression and Psychosocial Distress If depression is diagnosed in a cancer patient, then a treatment plan for depression should be documented
If a patient with cancer is treated for depression, then response to therapy should be documented within 6 weeks
Nausea and Vomiting If a patient with cancer undergoing moderately or highly emetic chemotherapy or with advanced cancer affecting the gastrointestinal tract or abdomen is seen for a visit in a cancer-related outpatient setting, then the presence or absence of nausea or vomiting should be assessed at every visit
If a patient with advanced cancer affecting the gastrointestinal tract or abdomen is admitted to a hospital, then the presence or absence of nausea or vomiting should be assessed within 24 hours
If a patient with cancer is undergoing chemotherapy treatment with a high acute emetic risk, then a 3-drug regimen including single doses of a 5-HT3 receptor antagonist, dexamethasone, and selective neurokinin-1 receptor blocker should be given immediately prior to chemotherapy
If a patient with cancer is undergoing chemotherapy treatment with a moderate acute emetic risk, then a 2-drug regimen including a 5-HT3 receptor antagonist and dexamethasone should be given immediately prior to chemotherapy
If a patient with cancer reports nausea or vomiting on admission to the hospital, then within 24 hours potential underlying causes should be assessed
If an inpatient with cancer has nausea or vomiting, then within 24 hours of the initial report of nausea and vomiting, the patient should be offered a change in therapy
If an outpatient with cancer not receiving chemotherapy or radiation is treated for nausea or vomiting with an antiemetic medication, then the effectiveness of treatment should be evaluated before or on the next visit to the same outpatient site
Fatigue/Anemia If a cancer patient is seen for an initial visit or any visit while undergoing chemotherapy at a cancer-related outpatient site, then there should be an assessment of the presence or absence of fatigue
If a known cancer patient is newly diagnosed with advanced cancer, then there should be an assessment of the presence or absence of fatigue
If a patient with cancer is found to have anemia with a hemoglobin <10 g/dl, then the presence and severity of anemia-related symptoms (e.g. fatigue, dyspnea, and lightheadedness) should be evaluated
If a patient with cancer is found to have severe, symptomatic anemia (hemoglobin <8 g/dL),then transfusion with packed red cells should be offered to the patient within 24 hours
Anorexia/Weight loss If a patient presents for an initial visit for cancer affecting the oropharynx or gastrointestinal tract or advanced cancer at a cancer-related outpatient site, then there should be an assessment for the presence or absence of anorexia or dysphagia
If a cancer patient is treated with an appetite stimulant for anorexia, then there should be an assessment before or on the next visit to the same outpatient site of whether or not there was an improvement in anorexia
If a cancer patient is treated with enteral or parenteral nutrition, then there should be an assessment prior to starting nutrition that there was difficulty maintaining nutrition due to significant gastrointestinal issues and that expected life expectancy was at least one month
Dyspnea If a patient with cancer reports new or worsening dyspnea, then there should be documentation of cause or of investigation of at least one of the following: hypoxia, anemia, bronchospasm or chronic obstructive pulmonary disease, pleural effusion, tumor obstruction of bronchi or the trachea, pneumonia, or pulmonary embolism
If an outpatient with primary lung cancer or advanced cancer reports new or worsening dyspnea, then they should be offered symptomatic management or treatment directed at an underlying cause within one month
If an inpatient with primary lung cancer or advanced cancer has dyspnea on admission, then they should be offered symptomatic management or treatment directed at an underlying cause within 24 hours
If a patient with cancer in the hospital is treated for dyspnea, then there should be an assessment within 24 hours that the treatment was effective in relieving dyspnea OR that a change in treatment for dyspnea was made
If a cancer patient has dyspnea and a malignant pleural effusion, then they should be offered thoracentesis within 1 month of the initial diagnosis of the effusion, or other treatment (e.g., diuresis) should result in a reduction in the effusion or symptomatic dyspnea
If a cancer patient with a malignant pleural effusion undergoes thoracentesis, then there should be a repeat assessment of dyspnea within one week
Treatment-Associated Toxicities
Diarrhea If a patient with cancer is undergoing chemotherapy and has diarrhea then in order to classify the diarrhea as complicated or uncomplicated all of the following should be assessed: · history of onset and duration,· number of stools and stool composition, and· at least one of the associated symptoms[fever, dizziness, abdominal pain/cramping, nausea/vomiting, decreased performance status, sepsis, fever, bleeding, or dehydration]
If a patient with cancer is undergoing chemotherapy with a high risk (>10%) of chemotherapy-induced diarrhea then an antidiarrheal agent should be prescribed on or before treatment is initiated
Delirium IF a hospitalized patient with cancer over the age of 65 or with advanced cancer has delirium THEN there should be an assessment for the presence or absence of at least one of the following potential causes and their association with delirium: medication effects, central nervous system disease, infection, or metabolic processes
Skin Rash If a patient with cancer who is being treated with agents that block epidermal growth factor receptors (EGFR), then the presence and severity of skin rash should be evaluated within 1 month after starting the treatments and at each visit
Information and Care Planning If a patient with advanced cancer dies an expected death, then there should be documentation of an advance directive or a surrogate decision maker in the medical record
If a patient with advanced cancer dies an expected death, then s/he should have been referred for palliative care prior to death (hospital-based or community hospice) or there should be documentation why there was no referral
If a patient with advanced cancer is admitted to the ICU and survives 48 hours, then within 48 hours of ICU admission, the medical record should document the patient's preferences for care or attempt to identify them
If a patient with advanced cancer is mechanically ventilated in the ICU, then within 48 hours of admission to the ICU, the medical record should document the patient's preference for mechanical ventilation or why this information is unavailable
If a patient with cancer undergoes chemotherapy, then prior to chemotherapy, s/he should be informed about the risks and benefits of treatment, including likely symptoms and side effects, and whether the treatment intent is curative or palliative

Journal Publications

Dy SM, Lorenz KA, O'Neill SM, et al. Cancer Quality-ASSIST supportive oncology quality indicator set: feasibility, reliability, and validity testing. Cancer 2010;116:3267-75.

Project Timeline

Cancer Quality-ASSIST Supportive Oncology Quality Indicator Set: Feasibility, Reliability, and Validity Testing

May 18, 2010
Topic Initiated
May 19, 2010
Research Report Archived
Page last reviewed January 2020
Page originally created November 2017

Internet Citation: Research Report: Cancer Quality-ASSIST Supportive Oncology Quality Indicator Set: Feasibility, Reliability, and Validity Testing. Content last reviewed January 2020. Effective Health Care Program, Agency for Healthcare Research and Quality, Rockville, MD.

Select to copy citation