Visual Abstract
Abstract
Background and objectives The kidney failure risk equation is a clinical tool commonly used for prediction of progression from CKD to kidney failure. The kidney failure risk equation’s accuracy in advanced CKD and whether this varies by CKD etiology remains unknown. This study examined the kidney failure risk equation’s discrimination and calibration at 2 and 5 years among a large tertiary care population with advanced CKD from heterogeneous etiologies.
Design, setting, participants, & measurements This retrospective cohort study included 1293 patients with advanced CKD (median eGFR 15 ml/min per 1.73 m2) referred to the Ottawa Hospital Multi-Care Kidney Clinic between 2010 and 2016, with follow-up clinical data available through 2018. Four-variable kidney failure risk equation scores for 2- and 5-year risks of progression to kidney failure (defined as dialysis or kidney transplantation) were calculated upon initial referral and correlated with the subsequent observed kidney failure incidence within these time frames. Receiver operating characteristic curves and calibration plots were used to measure the discrimination and calibration of the kidney failure risk equation both in the overall advanced CKD population and by CKD etiology: diabetic kidney disease, hypertensive nephrosclerosis, GN, polycystic kidney disease, and other. Pairwise comparisons of the receiver operating characteristic curves by CKD etiology were performed to compare kidney failure risk equation discrimination.
Results The kidney failure risk equation provided adequate to excellent discrimination in identifying patients with CKD likely to progress to kidney failure at the 2- and 5-year time points both overall (2-year area under the curve, 0.83; 95% confidence interval, 0.81 to 0.85; 5-year area under the curve, 0.81; 95% confidence interval, 0.77 to 0.84) and across CKD etiologies. The kidney failure risk equation displayed adequate calibration at the 2- and 5-year time points both overall and across CKD etiologies (Hosmer–Lemeshow P≥0.05); however, the predicted risks of kidney failure were higher than the observed risks across CKD etiologies with the exception of polycystic kidney disease.
Conclusions The kidney failure risk equation provides adequate discrimination and calibration in advanced CKD and across CKD etiologies.
- chronic kidney disease
- chronic kidney failure
- kidney failure
- progression of chronic renal failure
- kidney failure risk equation
- KFRE
- end-stage kidney disease
Introduction
CKD is a major global health issue associated with higher morbidity, mortality, and health care costs (1–⇓3). Patients with advanced CKD require a multifaceted health care approach including targeted strategies to slow disease progression; education about KRT modalities, including transplantation; and access creation planning should dialysis be needed. Most patients with CKD do not progress to kidney failure requiring KRT; however, the incidence of kidney failure is increasing at a disproportionately faster rate than the incidence of CKD (4–⇓6). Thus, assessment of kidney failure risk among the CKD population is critical for patient counseling and advanced care planning.
A number of risk prediction tools are available to estimate the risk of progression from CKD to kidney failure (7). The most widely utilized and easily applicable risk prediction tool with comprehensive validation is the kidney failure risk equation (KFRE) (8,9). This equation incorporates age, sex, eGFR calculated by the Chronic Kidney Disease Epidemiology Collaboration formula (10), and urine albumin-creatinine ratio (UACR) to predict the need for KRT (dialysis or kidney transplantation) within 2- and 5-year time frames. The KFRE has been adopted as a population-level tool to allocate funding for specialized programs that provide multidisciplinary care to patients with CKD at high risk for progression to kidney failure (11,12).
Although the KFRE has been extensively validated, data in regard to model performance in advanced CKD (Kidney Disease Improving Global Outcomes [KDIGO] stages 4 and 5) (13) are limited. Moreover, the rate of CKD progression is significantly influenced by kidney disease etiology (14–⇓16); yet, to our knowledge, disease-specific validation of the KFRE has not been conducted. The study objectives were to assess the validity of the KFRE in regard to discrimination and calibration among patients with advanced CKD and to determine whether these performance characteristics varied by kidney disease etiology.
Materials and Methods
Study Design
We performed a retrospective cohort study of adults (≥18 years) with advanced CKD referred to the Ottawa Hospital Multi-Care Kidney Clinic between January 1, 2010 and December 31, 2016. The reporting of this study follows guidelines for prediction model validation (Supplemental Table 1) (17).
Data Source and Study Cohort
This study was conducted at the Ottawa Hospital Multi-Care Kidney Clinic (Ottawa, ON, Canada). The Ottawa Hospital is a 1150-bed academic tertiary care center with a catchment area of approximately 1.3 million. The Multi-Care Kidney Clinic is a specialty nephrology clinic designed to provide comprehensive, multidisciplinary care for patients with advanced CKD. The Ottawa Hospital Multi-Care Kidney Clinic is the sole such program in the catchment area. Timing of referral is at the discretion of the primary nephrologist, although referrals are suggested when the eGFR is <25 ml/min per 1.73 m2 or the 2-year KFRE is >20%. Patients are seen in the clinic regularly, from as often as every 2 weeks to a minimum of semiannually, although the exact interval is at the discretion of the nephrologist. At each visit, patients are seen by a nurse, a dietician, and a nephrologist, with pharmacist and social work support available as needed. Patients are educated about kidney failure treatment options, including hemodialysis, peritoneal dialysis, kidney transplantation, and conservative management. Patients who select conservative management continue to be followed in the clinic alongside a palliative care physician who is integrated into the clinic structure.
The study cohort was derived from a database of all patients referred to the Multi-Care Kidney Clinic since January 1, 2010. We included patients initially seen in the clinic between the years 2010 and 2016 (n=1369). Follow-up data were available through December 31, 2018. We excluded patients who (1) had their nephrology care transferred to a provider outside of the Multi-Care Kidney Clinic, (2) moved away from the Ottawa area, or (3) left the clinic for another reason (Figure 1). These patients (n=76 of 1369 [6%]) were excluded as we did not have access to outcome data in regard to kidney failure or death. To assess the 5-year KFRE, we included only patients initially seen in the clinic between the years 2010 and 2013 to ensure at least 5 years of follow-up. We identified 1293 patients for inclusion to assess the 2-year KFRE and 637 patients for inclusion to assess the 5-year KFRE. Patients were subclassified by kidney disease etiology as determined by their referring nephrologist: diabetic kidney disease, hypertensive nephrosclerosis, GN, autosomal dominant polycystic kidney disease (ADPKD), or other. All protocols were approved by the Ottawa Health Science Network Research Ethics Board (protocol identification no. 20190174–01H). Informed consent requirements were waived due to the deidentified nature of the data. This study adhered to the Declaration of Helsinki.
Study flow diagram.
Outcome
The outcome of interest was the observed incidence of kidney failure (dialysis or kidney transplantation) at 2 and 5 years from initial clinic referral. These time points were selected as the KFRE was designed and is used in clinical practice to predict the risk for progression to kidney failure at 2 and 5 years.
Predictors
The predictors used in this study were the 2- and 5-year four-variable KFRE scores at the time of initial clinic referral. KFRE performance was assessed for the total population and across kidney disease etiologies.
Statistical Analyses
For baseline data, continuous variables are expressed as medians (25th to 75th percentile interquartile ranges [IQRs]), whereas categorical variables are expressed as numbers and percentages, unless otherwise specified. The variables necessary to calculate the KFRE were collected at the initial visit as per clinic protocol; therefore, these data were available for the entire cohort with no missing data. UACR was obtained from a random spot sample. The KFRE-predicted risk for kidney failure was then correlated with the observed incidence of kidney failure at the 2- and 5-year time points. Complete outcome data were available for the cohort in regard to kidney failure and death prior to kidney failure, with no missing data. There was no loss to follow-up within the study cohort. If a patient died prior to kidney failure, the patient was included in the analysis as not having developed kidney failure within that time frame. Death was not treated as a competing risk in our primary analyses so as to align with the design of the original derivation of the KFRE equation (9); however, a sensitivity analysis accounting for competing risk was included.
Discrimination.
Discrimination was assessed via receiver operating characteristic (ROC) curves, where the true positive rate (sensitivity) was plotted against the false positive rate (1– specificity), and the area under the curve (AUC) was measured. ROC curves were generated using logistic regression models comparing the predicted versus observed risk for kidney failure at 2 and 5 years. ROC curves were created for both the overall population and by kidney disease etiology. We defined the quality of discrimination on the basis of the AUC as follows: ≥0.90= outstanding, 0.80–0.89= excellent, 0.70–0.79= acceptable, 0.51–0.69= poor, and 0.50= no discrimination (18). Pairwise comparisons of the AUCs by kidney disease etiology were performed using the equation: chi-square =(AUC1–AUC2)2/(s12+s22), where s represents SEM (19). Brier scores measured the accuracy of KFRE prediction. Brier scores range from zero (most accurate) to one (least accurate).
Calibration.
Calibration plots compared the predicted and observed risks for kidney failure at both the 2- and 5-year time points. Within these plots, the study population was divided into deciles on the basis of the predicted probability of kidney failure. Within each decile, the mean predicted probability and the proportion of the observed outcome of kidney failure (along with 95% confidence intervals [95% CIs]) were plotted. The Hosmer–Lemeshow test assessed goodness of fit.
Sensitivity Analyses.
To evaluate the effect of the competing risk of death prior to kidney failure, we performed survival analyses comparing the Kaplan–Meier curve (where death prior to kidney failure was censored) with the cumulative incidence function (where death prior to kidney failure was treated as a competing risk).
All statistical analyses were performed using SAS v9.4. All P values are two sided, with values of 0.05 considered significant.
Results
Patient Characteristics
In total, 1293 patients were included in the 2-year KFRE analysis (Table 1), while 637 patients were included in the 5-year KFRE analysis (Supplemental Table 2). Among the 2-year cohort, the most common kidney disease etiology was diabetic kidney disease (637 of 1293 [49%]), followed by hypertensive nephrosclerosis (213 of 1293 [16%]), GN (196 of 1293 [15%]), ADPKD (74 of 1293 [6%]), and other (173 of 1293 [13%]). The median age was 68 years (IQR, 58–78 years). The majority of the population was men (792 of 1293 [61%]) and of White race (964 of 1293 [75%]). The median eGFR was 15 ml/min per 1.73 m2 (IQR, 12–19 ml/min per 1.73 m2) and similar across kidney disease etiologies. The median UACR was 1277 mg/g (IQR, 322–2903 mg/g), with higher levels of albuminuria in diabetic kidney disease and GN and lower levels in hypertensive nephrosclerosis and ADPKD. Approximately half (633 of 1293 [49%]) of patients were prescribed an angiotensin-converting enzyme inhibitor or angiotensin II receptor blocker, with higher usage in diabetic kidney disease, GN, and ADPKD and lower usage rates in hypertensive nephrosclerosis. Given the design of the Multi-Care Kidney Clinic, these baseline data were collected for all study participants with no missing data. Supplemental Table 3 compares the original KFRE development cohort (9) with this validation cohort.
Baseline characteristics of 1293 patients with advanced chronic kidney disease referred to the Ottawa Hospital Multi-Care Kidney Clinic from 2010 to 2016
Baseline Kidney Failure Risk Equation Scores
For the 2-year KFRE cohort, the median 2-year KFRE score was 47% (IQR, 23%–71%) (Table 1). The highest 2-year KFRE scores were seen with GN (66%; IQR, 44%–83%) and diabetic kidney disease (54%; IQR, 30%–75%). The lowest KFRE scores were seen with hypertensive nephrosclerosis (22%; IQR, 12%–47%) and ADPKD (34%; IQR, 20%–50%). For the 5-year KFRE cohort, the median baseline 5-year KFRE score was 87% (IQR, 53%–98%) (Supplemental Table 2). Similar to the 2-year KFRE cohort, the highest 5-year KFRE scores were seen with GN (97%; IQR, 87%–100%) and diabetic kidney disease (91%; IQR, 63%–99%), whereas the lowest KFRE scores were seen with hypertensive nephrosclerosis (61%; IQR, 33%–84%) and ADPKD (75%; IQR, 51%–90%).
Kidney Failure and Mortality Events
Among the 1293 patients included in the 2-year KFRE analysis, there were 541 kidney failure events and 144 deaths prior to kidney failure. Among the 637 patients included in the 5-year KFRE analysis, there were 406 kidney failure events and 111 deaths prior to kidney failure. Supplemental Table 4 shows the breakdown of kidney failure and death prior to kidney failure by kidney disease etiology.
Discrimination of the Kidney Failure Risk Equation
Figures 2 and 3 display the ROC curves along with AUCs and Brier scores for the 2- and 5-year KFRE analyses, respectively. For the total population, the KFRE shows excellent discrimination at both the 2-year (AUC, 0.83; 95% CI, 0.81 to 0.85) and 5-year (AUC, 0.81; 95% CI, 0.77 to 0.84) time points. The 2-year KFRE showed excellent discrimination across all etiologies: diabetic kidney disease (AUC, 0.81; 95% CI, 0.78 to 0.84), hypertensive nephrosclerosis (AUC, 0.81; 95% CI, 0.74 to 0.87), GN (AUC, 0.83; 95% CI, 0.76 to 0.89), ADPKD (AUC, 0.85; 95% CI, 0.76 to 0.94), and other (AUC, 0.86; 95% CI, 0.80 to 0.92). The 5-year KFRE showed excellent discrimination for diabetic kidney disease (AUC, 0.81; 95% CI, 0.75 to 0.86), GN (AUC, 0.83; 95% CI, 0.73 to 0.92), and other (AUC, 0.82; 95% CI, 0.72 to 0.92), while showing acceptable discrimination for hypertensive nephrosclerosis (AUC, 0.77; 95% CI, 0.68 to 0.86) and ADPKD (AUC, 0.76; 95% CI, 0.55 to 0.96). Pairwise comparison of the ROC curves for both the 2- and 5-year KFRE by disease etiology showed no statistically significant differences (all P≥0.05) (Table 2). Brier scores assessing KFRE prediction accuracy were all <0.20 for the total population and across kidney disease etiologies in both the 2- and 5-year analyses.
Receiver operating characteristic (ROC) curves for the 2-year kidney failure risk equation (KFRE) demonstrating excellent discrimination in the total population and by kidney disease etiology. (A) ROC curves; (B) area under the curve (AUC, 95% confidence interval) and Brier scores. ROC curves were generated using logistic regression models comparing the predicted versus observed risk for kidney failure. ADPKD, autosomal dominant polycystic kidney disease.
ROC curves for the 5-year KFRE demonstrating adequate to excellent discrimination in the total population and by kidney disease etiology. (A) ROC curves; (B) AUC (95% confidence interval) and Brier scores. ROC curves were generated using logistic regression models comparing the predicted versus observed risk for kidney failure.
Pairwise comparison of receiver operating characteristic curves for 2- and 5-year kidney failure risk equations by kidney disease etiology
Calibration of the Kidney Failure Risk Equation
Figure 4 displays the calibration plots comparing the predicted versus observed risk of kidney failure in the 2- and 5-year analyses. There was adequate calibration for the total population using both the 2-year (Hosmer–Lemeshow P=0.36) and 5-year KFRE (Hosmer–Lemeshow P=0.31). Supplemental Tables 5 and 6 show the calibration data across kidney disease etiologies for the 2- and 5-year KFRE, respectively. There was adequate calibration across all kidney disease etiologies (all Hosmer–Lemeshow P≥0.05). However, given the smaller sample size in certain kidney disease etiologies, especially in the 5-year KFRE cohort, we may have had limited power to detect differences in model fit. Notably, we did find that the predicted risk of kidney failure was higher than the observed risk across all kidney disease etiologies with the exception of ADPKD, where the observed risk was higher.
Adequate calibration of the 2- and 5-year KFRE. Calibration plots are displayed by deciles on the basis of the predicted probability of kidney failure at 2 years (A) and 5 years (B). Error bars represent the 95% confidence intervals for the proportion of the observed binary response of kidney failure.
Sensitivity Analyses
Supplemental Figures 1 and 2 display the results of sensitivity analyses comparing the probability of kidney failure within our 2- and 5-year cohorts, respectively, using the Kaplan–Meier estimate (death prior to kidney failure treated as a censoring event) versus the cumulative incidence function (death prior to kidney failure treated as a competing risk). These results demonstrated a mild overestimation of the cumulative incidence of kidney failure when not accounting for the competing risk of death (as the KFRE was designed) (9).
Discussion
In this validation study, we found that the KFRE adequately predicted CKD progression to kidney failure in a large cohort of patients with advanced CKD. The KFRE provided adequate to excellent discrimination in identifying patients with CKD likely to progress to kidney failure at the 2- and 5-year time points both overall (2-year AUC, 0.83; 95% CI, 0.81 to 0.85; 5-year AUC, 0.81; 95% CI, 0.77 to 0.84) and across etiologies. The KFRE displayed adequate calibration at the 2- and 5-year time points both overall and across etiologies (Hosmer–Lemeshow P≥0.05).
Our study expands upon prior studies that have demonstrated the clinical utility of the KFRE tool in predicting CKD progression to kidney failure in numerous locations around the world (8,9,20–⇓22). However, data in regard to KFRE accuracy by kidney disease etiology, by which rate of CKD progression is known to vary widely (14–⇓16), have been lacking. Our study found no clear difference in KFRE performance across disease etiologies. In addition, data on performance of KFRE in patients with KDIGO stages 4 and 5 CKD are limited (21). The median eGFR in our study was 15 ml/min per 1.73 m2, and 75% of patients had an eGFR<20 ml/min per 1.73 m2. Our results extend previous observations and confirm that even in advanced CKD, the KFRE remains a valuable tool to predict progression to kidney failure.
Despite the adequate discrimination and calibration seen in this validation study, we found that the 2- and 5-year predicted risks of kidney failure were higher than the observed risk for the overall population and across all disease etiologies with the exception of ADPKD. One possible explanation for this may be the effect of death as a competing risk. The KFRE was designed without incorporating death as a competing risk (9). In a sensitivity analysis, we show that this may lead to a mild overestimation of the cumulative incidence of kidney failure. This observation corresponds with the findings of a recent large cohort study by Ravani et al. (23), where the authors demonstrate that not accounting for death as a competing event in risk prediction models may lead to clinically meaningful overestimates in risk (23), although we found the differences in risk in this study to be modest and cannot confirm their effects on clinical management. In regard to why ADPKD may be the exception with higher observed than predicted risk, this may relate to its unique pathophysiology where disease progression associates more strongly with cyst growth and total kidney volume rather than the variables incorporated into the KFRE (24).
Our study has several strengths. First, it included a large sample size, along with a high event rate, from a heterogeneous population in regard to disease etiologies. Second, given the clinic protocol for laboratory data collected at the initial referral visit, we were able to calculate the KFRE for all patients seen in our clinic throughout the study period. Third, as the Ottawa Hospital Multi-Care Kidney Clinic is the only such clinic within the catchment area, the vast majority of patients continued to be followed in the clinic (>90%) until the end of the study period, kidney failure, or death.
We acknowledge several limitations. First, comparison of KFRE predictive performance by disease etiology was subject to potential misclassification. Kidney disease etiology was defined on the basis of the assessment by the patient’s primary nephrologist prior to referral. Disease etiology was proven by biopsy in a minority of patients, which could lead to misclassification and thereby limit comparisons of KFRE performance across etiologies. However, it is not commonplace to biopsy most patients with CKD, particularly those with diabetic kidney disease, hypertensive nephrosclerosis, or ADPKD (25). An individual patient’s suspected kidney disease etiology was determined by the nephrologist most familiar with his or her course. Additionally, the baseline data correlate with what would be anticipated findings with these etiologies (e.g., patients with diabetic kidney disease having higher levels of albuminuria, patients with hypertensive nephrosclerosis being older, etc.). Second, the sample size for certain disease etiologies (e.g., ADPKD) was limited particularly at the 5-year time point. Thus, we cannot definitively conclude that the KFRE has comparable accuracy across the different kidney disease etiologies included in this study. Third, compared with prior demographic data on the advanced CKD population across Canada (26,27), our study population had a higher proportion of patients with diabetic kidney disease. This may be related to the design of the Multi-Care Kidney Clinic, where referral is at the discretion of the primary nephrologist; this may result in a selection bias in terms of which patients are referred, which may differ from the advanced CKD population at large. This feature, along with the single-center study design and geographic differences in racial diversity, may limit the generalizability of the results to other clinical settings.
In conclusion, the KFRE remains a clinically useful prediction tool for progression from CKD to kidney failure in patients with advanced CKD due to heterogeneous etiologies of kidney disease. These findings highlight the utility of the KFRE among diverse populations of patients with CKD. However, the predicted risk of kidney failure at both 2 and 5 years was generally higher than the observed risk, which may at least partly relate to the competing risk of death within this population that is not accounted for within the KFRE. Nephrologists should be aware of this feature of the KFRE because it may affect advanced care planning.
Disclosures
M.M. Sood reports receiving personal fees and other from AstraZeneca during the conduct of the study. N. Tangri reports receiving personal fees from Roche Inc. and other from ClinPredict Inc. during the conduct of the study. He also reports receiving grants and personal fees from AstraZeneca Inc. and Janssen; receiving personal fees from Boehringer Ingelheim/Eli Lilly and Otsuka Inc.; receiving grants, personal fees, consulting fees, and stock options related to Veverimer from Tricida Inc.; receiving consulting fees and stock options from Mesentech and PulseData; and co-ownership of ClinPredict, outside the submitted work.
All remaining authors have nothing to disclose.
Funding
G.L. Hundemer is supported by Kidney Foundation of Canada Kidney Research Scientist Core Education and National Training (KRESCENT) New Investigator Award (2019KP-NIA626990).
Acknowledgments
The authors acknowledge the individuals who assisted with data management at the Ottawa Hospital: Ms. Suzanne Jackson and Ms. Melanie Bujold.
Supplemental Material
This article contains the following supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.03940320/-/DCSupplemental.
Supplemental Figure 1. Competing risk analysis for 2-year KFRE.
Supplemental Figure 2. Competing risk analysis for 5-year KFRE.
Supplemental Material. References.
Supplemental Table 1. TRIPOD checklist for prediction model validation.
Supplemental Table 2. Baseline characteristics of 637 patients with advanced CKD referred to the Ottawa Hospital Multi-Care Kidney Clinic from 2010 to 2013.
Supplemental Table 3. Comparison of study cohort with the KFRE development cohort.
Supplemental Table 4. Kidney failure and death prior to kidney failure by kidney disease etiology.
Supplemental Table 5. Calibration data for 2-year KFRE by kidney disease etiology.
Supplemental Table 6. Calibration data for 5-year KFRE by kidney disease etiology.
Footnotes
Published online ahead of print. Publication date available at www.cjasn.org.
- Received March 26, 2020.
- Accepted August 3, 2020.
- Copyright © 2020 by the American Society of Nephrology