Abstract
Background and objectives: Prognostic information is rarely conveyed by nephrologists because of clinical uncertainty about accuracy. The objective of this study was to develop an integrated prognostic model of 6-mo survival for patients who receive hemodialysis (HD).
Design, setting, participants, & measurements: A short-term prognostic model was developed using prospective data from a derivation cohort of 512 patients who were receiving HD at five dialysis clinics. Patient charts were reviewed for actuarial predictors (e.g., Charlson Comorbidity), and nephrologists answered the “surprise” question (SQ), “Would I be surprised if this patient died within the next 6 mo?” Survival was monitored for up to 24 mo. The prognostic model was tested with a validation cohort of 514 patients from eight clinics.
Results: In a Cox multivariate analysis of the derivation cohort, five variables were independently associated with early mortality: Older age (hazard ratio [HR] for a 10-yr increase 1.36; 95% confidence interval [CI] 1.17 to 1.57), dementia (HR 2.24; 95% CI 1.11 to 4.48), peripheral vascular disease (HR 1.88; 95% CI 1.24 to 2.84), decreased albumin (HR for a 1-U increase 0.27; 95% CI 0.15 to 0.50), and SQ (HR 2.71; 95% CI 1.76 to 4.17). Area under the curve for the resulting prognostic model predictions of 6-mo mortality were 0.87 (95% CI 0.82 to 0.92) in the derivation cohort and 0.80 (95% CI 0.73 to 0.88) in the validation cohort.
Conclusions: An integrated 6-mo prognostic tool was developed and validated for the HD population. The instrument may be of value for researchers and clinicians to improve end-of-life care by providing more accurate prognostic information.
Despite an explicit recommendation by the Renal Physicians Association and American Society of Nephrology, nephrologists rarely have discussions with patients about prognosis and end-of-life care (1–3). Dialysis patients want to discuss these matters (4,5), and the Robert Wood Johnson Foundation's Promoting Excellence in End-of-Life Care ESRD Workgroup has recommended that the National Institute of Diabetes and Digestive and Kidney Diseases encourage more research on the prognosis of renal patients (6).
Although nephrologists have both a logical and a moral duty to disclose prognosis (7), they are apprehensive about extinguishing patient hope and the lack of sufficiently accurate tools for prediction (8,9). The first concern is not supported by research, which has instead found that honest, ongoing communication of prognosis reinforces both trust and hope (10), enhances a mutually respectful doctor–patient relationship (11–13), and facilitates treatment decisions that are consistent with underlying values (14,15). The second concern is more valid, although two promising methods for formulating prognosis have been applied to ESRD: (1) An actuarial estimation of survival and (2) the clinician's prediction of survival (16,17). Actuarial factors include age, albumin, activities of daily living, hemoglobin variability, phosphorus, and parathyroid hormone (18,19). Investigators have attempted to improve the ability to discriminate mortality outcomes by taking account of patient comorbidities (e.g., through the use of a modified Charlson Comorbidity Index [CCI]) (20,21). The CCI examines both age and the presence of comorbid disorders and allows for identification of a subpopulation of sicker dialysis patients with approximately a 50% 1-yr mortality rate (22,23). One promising means for eliciting the clinician prediction of survival is the “surprise” question (SQ): “Would I be surprised if this patient died in the next year?” (24–26). In one study, the unadjusted odds of dying within 1 yr for the patients with ESRD in the “No, I would not be surprised” group was 3.5 times higher than for patients in the “Yes” group (27).
It has been suggested that combining actuarial estimations with staff predictions into an integrated prognostic model may represent a gold standard for prognostic determinations (28,29). The purpose of this research was to conduct a longitudinal prospective cohort study and develop an integrated prognostic model of 6-mo survival for patients who receive hemodialysis (HD).
Materials and Methods
Derivation Cohort
Between July 2006 and September 2007, all of the adult HD patients (n = 512) at five Fresenius Medical Care North American dialysis clinics in Western Massachusetts directed by the nephrologists of the Western New England Renal Transplant Associates were screened. A chart review and brief contact with the primary nephrologists provided information about actuarial predictors (e.g., age, comorbid disorders) and a modified SQ response. The latter involved asking each nephrologist whether he or she would be surprised if his or her patient died within the next 6 mo. This duration was chosen because it is the time requirement for hospice eligibility. Patient mortality data were collected through June 30, 2008.
Validation Cohort
Between September and October 2008, the same procedures were followed for a validation cohort of all adult HD patients (n = 514) at eight New England HD clinics (the five original clinics plus three additional). Once again, a chart review and brief contact with the primary nephrologists were conducted to collect the necessary data. Six-month and 1-yr patient mortality was collected for analysis. The research was funded by an R21 grant from the National Institutes of Health, and the protocol was approved by the Baystate Medical Center institutional review board.
Statistical Analyses
The characteristics of the derivation and validation cohorts were summarized by computing means and SD for continuous variables (e.g., age, albumin) and frequency distributions for categorical variables. Distributions of continuous variables were also plotted and summarized as medians and interquartile ranges when highly skewed (e.g., months of HD).
As the first step in building a predictive model, patient characteristics were individually evaluated as markers of survival using univariate Cox proportional hazards (PH) regression models. Univariate, or unadjusted, Cox PH survival models were used to examine gender, age, race, ethnicity, insurance status, duration of dialysis, the CCI score and its individual components, albumin level, and the SQ as potentially important markers of survival in the derivation cohort. Martingale residuals were used to evaluate the functional form of continuous variables to determine whether the linear form was justified. On the basis of this evaluation, albumin was truncated at 3.0 and 4.5 to improve the model fit. Albumin and CCI both were analyzed as continuous and dichotomized forms. Individual components of the CCI were also evaluated.
A multivariable Cox PH model was built by initially using a stepwise selection process to choose among all variables found to have P ≤ 0.15 from univariate relations. Best subsets models were also reviewed to determine whether any other covariates may have been missed by the stepwise selection process. The investigators then reviewed the model and covariates to confirm clinical relevance. Deviance and score residuals were plotted to look for poorly predicted subjects or subjects that may have had large influences on model parameters. Shoenfeld residuals plotted with smoothed curves and P values that were associated with correlations of these residuals with ranked time were checked to assess the assumption of PH. There was no evidence of outliers, nonlinear relationships, or any terms in the model that violated the assumption of PH (30).
The performance of the integrated prognostic model was assessed by computing a non–time-specific area under the curve (AUC) using the measure of concordance from Somer's Dxy statistic; AUC at specified time intervals (6, 12, and 18 mo) were also computed. For assessment of model calibration, patients were risk-stratified on the basis of their predictive index, which is the linear combination of the product of the multivariate model β coefficients and their individual covariate values. Kaplan-Meier survival curves were then plotted for patients who were stratified into quintiles of their predictive indices. The validation cohort's AUC was determined, and predicted and observed mortalities were compared. Predicted mortalities were calculated using the parameter estimates from the Cox model in conjunction with each patient's individual covariate values and the underlying baseline survival (survival function for a hypothetical patient with all covariates having a value of 0) to compute a predicted survival at any time by using the formula predicted survival at time t = [So(t)]exp(xβ) where So(t) is baseline survival estimate from the derivation data at time t from the Cox multivariable model and xβ is the linear combination of parameter estimates multiplied by their covariate values or predicted index of risk. Predicted and observed mortalities were compared. Analyses were done using two statistical packages: SAS 9.1.3 (31) and R 2.5.0 (32) in a Microsoft Windows XP environment.
Results
Figure 1 shows the identification of the patients for both the derivation and validation cohorts. Patient characteristics, summarized in Table 1, were comparable to regional and national demographics for the dialysis patient population. A larger percentage of patients who received dialysis for <3 mo was evident in the derivation cohort, because it had a longer enrollment period. The SQ was answered for 88% of the derivation sample (n = 450), and nephrologists classified 71 (15.8%) of the 450 patients into the “No” group. At the completion of the study, 54.9% of the “No” group patients were dead compared with 17% of the “Yes” group patients, with an unadjusted hazard ratio (HR) of 4.88 (95% confidence interval [CI] 3.27 to 7.29). Other factors that were found to have unadjusted independent relationships with poorer survival at the P ≤ 0.05 level included older age; lower albumin; higher values of the CCI; and history of congestive heart disease, chronic obstructive pulmonary disease, cerebrovascular disease, dementia, myocardial infarction, peripheral vascular disease (PVD), diabetes, and cancer (data not shown).
Patient flow diagram for derivation and validation cohorts.
Patient characteristics of screened cohort (n = 512 screened patients), overall and stratified by last known survival status
Table 2 shows characteristics of patients from the derivation cohort stratified by the SQ response. Patients in the “No” group of the SQ response were more likely to be female, be older, be of white race, have received dialysis for a shorter time, have lower albumin, and have more comorbidities, as compared with patients in the “Yes” group.
Patient characteristics of derivation cohort, stratified by response to the SQ
A multivariate model was constructed from univariate factors that were found to have unadjusted relationships with the outcome of P < 0.15, and the results are presented in Table 3. The five-term multivariable Cox PH model resulting from the variable selection process improved on the predictive accuracy of the SQ. A “No” response to the SQ continued to be a statistically significant marker for worse survival, although, as might be expected, the magnitude of the adjusted HR of 2.71 (95% CI 1.76 to 4.17) was less than when not adjusted for any other covariates. Lower albumin, older age, presence of dementia, and presence of PVD were also found to be markers of poorer survival. When albumin was modeled as a continuous variable, it was a stronger predictor than when dichotomized. Although age, PVD, and dementia all were components of the CCI, the full CCI was not statistically significant after controlling for these other terms, and it was therefore not included in this final model. The addition of study site as a stratification term had little impact on the HR or associated P values and was also not included in the final multivariable model.
Multivariable model of survival in the derivation cohort (n = 449 patients with complete data, 103 died)
In the derivation cohort, the baseline survival estimate at 6 mo was 0.58. The probability of being alive at 6 mo could be estimated using this value together with the parameter estimates presented in Table 3 and with each patient's individual covariates.
The AUC for 6-mo survival in the derivation cohort using this integrated prognostic model was 0.87 (95% CI 0.82 to 0.92). The AUCs for 12- and 18-mo survival were 0.82 (95% CI 0.76 to 0.88) and 0.79 (95% CI 0.73 to 0.86), respectively, and the overall AUC area for this model across all time, based on Somers Dxy, was 0.77 (95% CI 0.72 to 0.82).
To evaluate further how well the model discriminated patients with higher and lower likelihoods of survival, we ranked patients in the derivation cohort by their predictive index of risk and calculated Kaplan-Meier plots of survival for quintiles of risk. As seen in Figure 2, the model successfully predicted which patients had worse and better survival over time with patients in quintile 5—the highest risk quintile—having had the poorest survival and patients in quintile 1—the lowest risk quintile—having had the best survival.
Survival across quartiles of predicted risk. The model successfully predicted which patients had worse and better survival over time with patients in quintile 5 (q5)—the highest risk quartile—having the poorest survival and patients in q1—the highest quartile—having the best survival. The survival in q5 was significantly worse than all other quintiles pooled together (log-rank test P < 0.0001).
The multivariable prognostic model that we created using the derivation cohort was subsequently tested in the validation cohort. As evident in Table 4, patients in the validation cohort were similar to those of the derivation cohort but were a little older and had been on HD longer, and the group had a lower proportion of nonwhite patients. The mean predicted 6-mo mortality for the 427 patients in the validation cohort with available data (as defined in the Figure 1) was 5.6% compared with the observed mortality of 8.2%, and the model had an AUC of 0.80 (95% CI 0.73 to 0.88).
Comparison of derivation and validation cohorts
Discussion
On account of a justifiable lack of confidence in the accuracy of existing predictive measures, nephrologists are hardly alone among physicians in hesitating to respond to patient and family wishes for prognostic information (33). Simple and more accurate instruments for prognostication are needed to enable staff to identify dialysis patients who have a poor prognosis and could benefit from palliative care, including advance care planning; pain and symptom management; and psychosocial, spiritual, and bereavement support (11).
In this study, the model-building process began with the selection of literature-based actuarial factors and clinician predictions, and mortality analyses then demonstrated the significance of each of these components, including the innovative tool, the SQ. The resultant five-term multivariable Cox PH model has a robust (0.87) AUC for 6-mo survival in the derivation cohort and a strong (0.80) AUC in the validation cohort (34). It was not surprising that the AUC areas declined at later time points, because a patient's condition can change over time; as a result, the further predictions that are based on the initial status of the patient are less accurate. The Kaplan-Meier plots of survival calculated for quintiles of risk suggest that this model is sufficiently accurate for both research and clinical applicability.
Although the CCI has hitherto received much of the attention in ESRD prognostic research, in this study, it overlapped with the SQ; however, although the SQ is similar to the CCI, they are not equivalent. Neither instrument is sufficiently sensitive and specific alone to permit clinical use for individual patients with the necessary degree of accuracy. The multivariable analysis led the CCI to be dropped from the final mathematical model, although three of its elements (age, dementia, and PVD) proved to be significant components. Age has repeatedly been recognized to be an independent, powerful predictor of death for patients with ESRD (1). Whereas geriatric patients with illnesses of dementia are widely known to have high mortality rates (35), the association of PVD with increased mortality in patients with ESRD is less widely appreciated but has been previously noted (36–39).
The derivation and validation cohorts were drawn from one geographic region, New England, and included mainly prevalent and a small number of incident patients. Although the demographics of the patient population closely approximate the national ESRD population, it remains to be seen whether the resultant model may be as accurate in other regions or in clinics with differing populations. Although there was some overlap, the validation cohort data were collected at least a full year after the derivation data collection was completed. The derivation cohort's enrollment period was longer than that of the validation cohort, and this resulted in a proportionately larger number of incident patients and those who received dialysis for <3 mo. Also, as is evident in Table 1, nearly every variable had some missing data; however, the percentage of cases with missing data were very low. Despite all of these considerations, the integrated prognostic model satisfactorily predicted mortality in the validation cohort.
The SQ is a new instrument, and it has been previously used only in a single ESRD study that relied on a 1-yr rather than a 6-mo period for prediction (27). In this research, SQ estimates from nephrologists were not obtained for 13% of the total patient samples from the derivation and validation cohorts, because the data collectors were unable to contact the physicians in a timely manner. The subgroup with missing data did not reflect uncertainty on the part of the clinicians; although these patients had lower albumin and worse survival, they were similar with respect to all other parameters, including age, gender, race, duration of dialysis, and CCI (data available upon request). There were an insufficient number of physicians in the study to allow for meaningful analysis of potential associations between nephrologist training, experience, and accuracy of SQ predictions.
Large ESRD databases have been retrospectively queried for predictors of mortality (40). Many factors, such as hemoglobin, calcium, parathyroid hormone, Kt/V, phosphate, C-reactive protein, troponin, IL-6, body mass index, high pulse pressure, use of venous catheter for access, Karnofsky index, vitamin D levels, hypotension, advanced heart failure, and dialysis “vintage,” have shown varying degrees of prognostic value (41–44). Most of the research has had the goal of identifying correctable factors and evaluating long-term mortality, whereas this study's goal differs in the creation of a short-term, integrated mortality prediction model that is based on uncorrectable factors. The integrated prognostic model may eventually be strengthened by inclusion of some these variables, as well as those that are newly associated with cardiovascular mortality (45).
Conclusions
The purpose of this study was to develop and validate a novel prognostic model with sufficient accuracy to identify patients who have an increased risk for short-term mortality. Clinically, this is necessary to allow patients and families to be informed of prognosis and provided with the opportunity to engage in meaningful advance care planning discussions (46). Depending on patient symptoms and preferences, the identification of a poor prognosis can lead to a palliative care consultation and hospice referral. The integrated prognostic model lends itself to risk stratification of patients, it is more specific and sensitive than any of its components (e.g., the SQ), and it seems to be a considerable improvement over other existing instruments at predicting survival in the dialysis population. Further research is needed to confirm that use of this prognostic model provides adequate predictive accuracy in other dialysis clinic settings and to explore additional instruments that combine actuarial and clinician estimations of survival.
Disclosures
None.
Acknowledgments
This project was supported by grant 5 R21 DK076563-02, “A Hospice Intervention for Older Adults with ESRD: Sharing the Caring” from the National Institutes of Health. The funding source had no role in the design, analysis, or reporting of the study or in the decision to submit the article for publication.
Our thanks to Vincent Ferratusco for help with the graphics and Margaret Butorac and Janet Hakanson-Stacy for assistance with data collection.
Footnotes
Published online ahead of print. Publication date available at www.cjasn.org.
The views in this article are those of the authors and do not necessarily represent the official views of the National Institutes of Health.
- Received June 11, 2009.
- Accepted September 27, 2009.
- Copyright © 2010 by the American Society of Nephrology
References
In this issue
Jump to section
More in this TOC Section
Original Articles
Dialysis
Cited By...
- Multivariable prognostic model for dialysis patients with end stage renal disease: An observational cohort study of Pakistan by external validation
- Age, Gender and Diabetes as Risk Factors for Early Mortality in Dialysis Patients: A Systematic Review
- Quality of life in advanced renal disease managed either by haemodialysis or conservative care in older patients
- Lower Extremity Amputation and Health Care Utilization in the Last Year of Life among Medicare Beneficiaries with ESRD
- Unique palliative care needs of patients with advanced chronic kidney disease - the scope of the problem and several solutions
- Dementia, Alzheimers Disease, and Mortality after Hemodialysis Initiation
- Nephrology Provider Prognostic Perceptions and Care Delivered to Older Adults with Advanced Kidney Disease
- Frailty Screening Tools for Elderly Patients Incident to Dialysis
- Time to Improve Informed Consent for Dialysis: An International Perspective
- Serious Illness Conversations in ESRD
- Dementia and Alzheimer's Disease among Older Kidney Transplant Recipients
- The "surprise question" for predicting death in seriously ill patients: a systematic review and meta-analysis
- How long do patients with chronic disease expect to live? A systematic review of the literature
- Conservative Management and End-of-Life Care in an Australian Cohort with ESRD
- Supportive Care: Time to Change Our Prognostic Tools and Their Use in CKD
- Functional and Cognitive Impairment, Frailty, and Adverse Health Outcomes in Older Patients Reaching ESRD--A Systematic Review
- How I treat acute myeloid leukemia presenting with preexisting comorbidities
- Association of Performance-Based and Self-Reported Function-Based Definitions of Frailty with Mortality among Patients Receiving Hemodialysis
- The Ethics of Chronic Dialysis for the Older Patient: Time to Reevaluate the Norms
- Caring for Older Patients on Peritoneal Dialysis at End of Life
- Predicting palliative care needs and mortality in end stage renal disease: use of an at-risk register
- Four Plus Forty-Four: Hours to Modify, Theirs to Enjoy
- A Palliative Approach to Dialysis Care: A Patient-Centered Transition to the End of Life
- A Communication Framework for Dialysis Decision-Making for Frail Elderly Patients
- The Ethics of Offering Dialysis for AKI to the Older Patient: Time to Re-Evaluate?
- A Patient-Centered Vision of Care for ESRD: Dialysis as a Bridging Treatment or as a Final Destination?
- Prognosis and management of chronic kidney disease (CKD) at the end of life
- Dying on Dialysis: The Case for a Dignified Withdrawal
- Five Policies to Promote Palliative Care for Patients with ESRD
- What Determines Whether a Patient Initiates Chronic Renal Replacement Therapy?
- Conservative care for end-stage kidney disease: joint medical conference with the Renal Association, British Geriatrics Society and Association for Palliative Medicine
- Can the impact of an acute hospital end-of-life care tool on care and symptom burden be measured contemporaneously?
- Communication Skills Training for Dialysis Decision-Making and End-of-Life Care in Nephrology
- Predicting 12-Month Mortality for Peritoneal Dialysis Patients Using the "Surprise" Question
- The Ethics of End-of-Life Care for Patients with ESRD
- Advance Care Planning in CKD/ESRD: An Evolving Process
- Renal Replacement Therapy in the Elderly Population
- Informing Our Elders About Dialysis: Is an Age-Attuned Approach Warranted?
- Ethical Principles and Processes Guiding Dialysis Decision-Making
- End-of-Life Care Preferences and Needs: Perceptions of Patients with Chronic Kidney Disease