Abstract
Background and objectives: Abstracting information about comorbid illnesses from the medical record can be time-consuming, particularly when a large number of conditions are under consideration. We sought to determine which conditions are most prognostic and whether comorbidity continues to contribute to a survival model once laboratory and clinical parameters have been accounted for.
Design, setting, participants, & measurements: Comorbidity data were abstracted from the medical records of Dialysis Outcomes and Practice Pattern Study (DOPPS) I, II, and III participants using a standardized questionnaire. Models that were composed of different combinations of comorbid conditions and case-mix factors were compared for explained variance (R2) and discrimination (c statistic).
Results: Seventeen comorbid conditions account for 96% of the total explained variance that would result if 45 comorbidities that were expected to be predictive of survival were added to a demographics-adjusted survival model. These conditions together had more discriminatory power (c statistic 0.67) than age alone (0.63) or serum albumin (0.60) and were equivalent to a combination of routine laboratory and clinical parameters (0.67). The strength of association of the individual comorbidities lessened when laboratory/clinical parameters were added, but all remained significant. The total R2 of a model adjusted for demographics and laboratory/clinical parameters increased from 0.13 to 0.17 upon addition of comorbidity.
Conclusions: A relatively small list of comorbid conditions provides equivalent discrimination and explained variance for survival as a more extensive characterization of comorbidity. Comorbidity adds to the survival model a modest amount of independent prognostic information that cannot be substituted by clinical/laboratory parameters.
The extent of information on comorbid conditions that should be used to risk-adjust comparisons made across dialysis populations is unclear. In view of the high prevalence of comorbidity in this population (diabetes reported in nearly 50%, ischemic heart disease in 40%, and a history of congestive heart failure (CHF) in 50% [1–3]), simply accounting for a few broadly defined conditions may not be sufficient to control for case-mix differences across groups. Abstracting information from the medical record is time-consuming, which limits the number of conditions that would be practical to collect. The Charlson Comorbidity Index (CCI), which consists of 19 conditions, is frequently used for risk adjustment purposes. An increasing CCI score has been shown to correlate with mortality, reduced quality of life, and increased costs (4–7). Nevertheless, this instrument was designed for a general medical population, and it is unclear whether the most prognostic conditions are the same in a hemodialysis (HD) population. A previous study showed that the assigned weights do not generalize to a dialysis population (8). It is also unclear whether comorbidity adds independent prognostic information once routine laboratory and clinical parameters are accounted for. In previous studies, few case-mix factors other than demographics were considered. Laboratory and clinical parameters such as serum albumin and blood pressure (BP) are routinely measured and have consistently been shown to be strong predictors of adverse outcomes.
The Dialysis Outcomes and Practice Pattern Study (DOPPS) is the largest HD population to date from which detailed comorbidity data have been collected through medical record review. A previous study from DOPPS I described differences in the prevalence of comorbid conditions and their associations with mortality (3), but conditions were not ranked by their contribution to the survival model; neither was the prognostic significance of conditions individually or as a whole compared with other case-mix factors (9–12). The aim of this study was to identify the key comorbid conditions that are associated with survival through consideration of an extensive list of conditions and to compare the prognostic information provided by comorbidity with that provided by routinely measured laboratory and clinical parameters.
Methods
Study Population
Participants in the DOPPS are randomly selected from a representative sample of dialysis facilities within each country, as described previously (13). Our analyses were restricted to the DOPPS I (1996 through 2001), DOPPS II (2002 through 2004), and DOPPS III (2005 through present) US dialysis populations because between-country differences in prevalence and relationships of comorbid conditions with mortality were noted previously (3).
Collecting Information about Comorbid Conditions
Dialysis unit staff abstracted information about comorbid conditions from the medical record using a standardized questionnaire with 45 items that denoted the presence or severity of medical conditions at study entry. Comorbid conditions were recorded as absent, present, or suspected. “Suspected” was infrequent and was combined with “present.” Conditions were excluded from model building when they were believed not to be prognostic for survival on the basis of clinical judgment, were correlated with other conditions, or were defined subjectively (see Appendix 1). A CCI and Centers for Medicare and Medicaid Services (CMS) Medical Evidence Form 2728 (14) were scored using the equivalent item on the DOPPS Medical Questionnaire. Identical definitions were available for all items except congenital anomaly and toxic nephropathy on the Form 2728 and leukemia, lymphoma, and metastatic cancer on the CCI.
Other Case-Mix Factors
Several laboratory and clinical measurements were compared with comorbidity for their contribution to the survival models. Predialysis systolic BP and postdialysis weight were defined by the average of the three most recent measurements before or on the date of enrollment to DOPPS. Laboratory parameters were defined by the most recent value before or on the date of enrollment. The inability to walk independently and the inability to transfer independently were recorded by the patient's nurse or the dialysis unit social worker.
Statistical Analyses
One half of the combined DOPPS I, II, and III populations were randomly selected within each facility for model development. The other half were used to validate the models. Patients were followed until death, transplantation, transfer out of a participating unit, or study end, whichever came first. Relationships of comorbid conditions with mortality were assessed using Cox proportional hazards regression stratified by study phase.
The SCORE selection method (SAS/STAT 9.2) was used to fit successive models adding one comorbid condition at a time to a demographics-adjusted (e.g., age, gender, race, vintage) model until all 45 were incorporated, in the order of most to least contributory, the latter based on the score χ2 value. Forward stepwise selection was then used to build a demographics- and comorbidity-adjusted model, retaining variables that were significant at P ≤ 0.01. Laboratory and clinical parameters were added in subsequent models. An attributable fraction (AF) was calculated for each condition left in the model to rank its importance. The AF can be thought of as the reduction in mortality risk that would occur if no one had a particular comorbid condition. AF was calculated as the prevalence of the condition (pd) times the whole quantity of adjusted relative risk minus 1 divided by the adjusted relative risk: pd[(RR − 1)/RR] (15).
Assessing Predictive Accuracy in the Validation Sample
Predictive accuracy was based on the c statistic, a measure of discrimination, and the generalized Nagelkerke R2 (16), a measure of explained variance. The c statistic represents the number of times the model correctly assigns a lower predicted probability of 3-yr survival to a person who dies sooner than one who survives at a given time point or vice versa. A value of 0.5 indicates that the model performs no better than chance at predicting who, of two individuals with different death dates, will die earlier; a value of 1.0 represents perfect discrimination. Predictors were divided into four domains: Demographics, comorbid conditions, physical impairments, and laboratory/clinical factors. The increase in the c statistic and R2 upon addition of a variable or domain to a model indicates the increase in explained variance and discrimination provided by the variable or domain. Analyses were conducted using SAS 9.2 (Cary, NC).
Results
Baseline Characteristics
The development data set consisted of 7685 US HD patients who were followed for a mean of 1.5 yr (median 1.3 yr), during which 2505 (33%) died. The mean age was 61, 55% were male, 33% were black, 38% had diabetes as the cause of ESRD, 41% had a history of coronary heart disease, 44% had a history of CHF, 18% had a history of cerebrovascular disease, and 24% had a history of peripheral vascular disease (PVD). The prevalence of comorbid conditions was in keeping with other US dialysis populations on which detailed comorbidity information has been collected (15,16). The median duration of dialysis at the time of DOPPS enrollment was 2 yr. The validation data set consisted of 7696 individuals who were followed for a mean of 1.5 yr (median 1.3 yr), during which 2536 (33%) died. There were significant differences between the development and validation samples for only two factors: HIV/AIDS and malignancy (Table 1).
Characteristics of the development and validation study populations
Comorbid Conditions Related to Survival
The results of successively adding one comorbid condition at a time to a demographics-adjusted survival model in the development and validation data sets until all 45 conditions under consideration (Appendix 1) were incorporated is shown Figure 1. Nineteen conditions were significant in the development data set, two of which (systemic lupus erythematosus/vasculitis and cardiac arrest) were NS in the validation data set. Because of concerns that results would not generalize, the 17 conditions that remained significant in both the development and validation data sets were used for subsequent analyses (Table 2). There was no degeneration of model performance in the validation compared with the development data set (c statistic 0.67 versus 0.69, respectively, and R2 = 0.14 for both). The 17 conditions explained 96% of the total explained variance that would be found if all 45 comorbid conditions were added to a demographics-adjusted survival model (Figure 1).
Figure 1 shows the increase in R square with the successive addition of each of 45 comorbid conditions to a demographics-adjusted survival model. Demographics (age, gender, race and dialysis vintage) plus the 17 most prognostic comorbid conditions (see Table 2) account for 96% of the explained variance that would result if all 45 conditions were incorporated in the model.
Mortality risk attributable to comorbid conditions
Table 2 shows results of a stepwise model that excluded absence of hypertension, depression in the past 12 mo, and home oxygen use, conditions that are available from medical record review but may not be reliably captured in an administrative database. As expected, the c statistic (95% confidence interval [CI]) and R2 in the validation data set for a model with 12 conditions was less (0.65 [0.64 to 0.66], R2 = 0.08) than with 17 conditions (0.67 [0.66 to 0.69]; R2 = 0.10). The 17 conditions (referred to as the DOPPS short list from here onward) are used in subsequent analyses.
Predictive Accuracy of Models Based on Case-Mix Factors and Combination of Case-Mix Factors in the Validation Data Set
As shown in Table 3, the accuracy of the DOPPS short list in predicting 3-yr survival was higher (c statistic 0.67) than predictions that were based on age alone (0.63), demographics alone (0.64), serum albumin alone (0.60), and physical impairments (0.62) and was equivalent to a combination of prognostic laboratory/clinical parameters (0.67). Predictive accuracy increased as case-mix domains were combined. The increase in c statistic and R2 that results upon addition of comorbid conditions to a demographics-adjusted model was of the same magnitude as with the addition of laboratory/clinical parameters to a demographics-adjusted model; however, there is a further increase in the c statistic and R2 from 0.69 (95% CI 0.68 to 0.71) to 0.72 (0.71 to 0.73) and from 0.13 to 0.17, respectively, when comorbidity is added to the model adjusted for demographics and laboratory/clinical parameters.
Predictive accuracy of different risk factors for 3-yr survival in the validation data set
Comparison of the DOPPS Comorbidity Index with Other Indices
Table 4 shows similarities and differences in the conditions included in the DOPPS short list, the CCI, and the Form 2728. As shown in Table 5, when a comorbidity index plus demographics were used to predict survival, the DOPPS short list had increased discriminatory ability (c statistic 0.70) over the CCI (c statistic of 0.67 or 0.69, as a five-level score or as a continuous variable, respectively [4]) or the Form 2728 (0.68). The c statistics and R2 of models that were composed of the three comorbidity measures (DOPPS short list, CCI, and Form 2728) were no longer different when laboratory/clinical parameters were added.
Comparison of comorbid conditions on the DOPPS short list, CCI, and Form 2728
Comparison of various comorbidity measures
Discussion
Seventeen comorbid conditions plus demographics accounted for 96% of the total explained variance that would be found if a total of 45 conditions, each of which had face validity as a prognostic factor and was significant in univariate analyses, were related to survival. Stated another way, after the 17 most prognostic comorbid conditions were accounted for, the addition of 28 more conditions contributed only an additional 4% of the total explained variance of a survival model. Comparing the prognostic information provided by comorbidity with other case-mix factors, we found that the comorbid conditions had more discriminatory power for survival (c statistic 0.67) than age (0.64), serum albumin (0.60), or physical impairments (0.62) and was equivalent to a combination of laboratory/clinical parameters (0.67); however, a few comorbid conditions clearly provided additional prognostic information that was not found in demographics and laboratory/clinical parameters, as evidenced by the increase in c statistic (95% CI) and R2 value from 0.69 (0.68 to 0.71) to 0.72 (0.71 to 0.73) and from 0.13 to 0.17, respectively, with the addition of comorbidity to the demographics and laboratory/clinical factor-adjusted model.
That so few comorbid conditions mattered is unexpected. Each of the conditions under consideration was expected to increase mortality risk, and the majority were significant in univariate analyses, yet only 17 remained. One explanation is the coexistence of comorbid conditions (e.g., the patient with atrial fibrillation had ischemic heart disease and CHF and the risk of the former is captured in the latter conditions). In addition, although the number of comorbid conditions under consideration and the study population are larger than those in previous studies, the statistical power to detect significant relationships for low-prevalence items was limited. Failing to account for a rare but highly prognostic condition will have little impact on risk measurement in a large group of patients but will matter with smaller groups or for an individual. To some extent, the risk may be captured in nonspecific markers of illness, such as a reduced serum albumin or body mass index, although our study shows that these do not fully capture the prognostic information provided by comorbidity. To identify all possible conditions that influence survival and to determine the weight that each carries, however, would require hundreds of thousands of patients with even more detailed comorbidity information than was collected in this study. Even if they could be identified, the large number of items that would need to be considered during chart review would likely be impractical even for clinical trials. The Index of Coexistent Disease (63 items) was considered burdensome in the Hemodialysis (HEMO) Study (15). This study shows that a limited number of key conditions provide the bulk of the prognostic information that would be found with a more extensive comorbidity instrument.
Is it worth the time and expense to collect comorbidity information? A study that assessed the contribution of comorbidity to a survival model in 15,571 incident patients from five European countries suggested that comorbidity does not add much (17). The total explained variance of a model that was composed of age, gender, primary renal disease, treatment modality, and country increased from 14.4 to 16.3% with the addition of five comorbidities. We question whether some of the prognostic information that would have been provided by comorbidity was captured in treatment modality and country, which may not be widely applicable. In our study, comorbidity increased model R2 from 0.13 to 0.17, which represents nearly a 25% increase in explained variance over a model that was composed of demographics and laboratory/clinical parameters. Whether it is worth collecting depends on its use. Case-mix adjusting is most often performed to reduce bias that is introduced because of confounding—a factor of interest is associated with group assignment and with outcomes (18). There are clear examples in the dialysis literature in which the results of observational studies (e.g., improved survival with higher hemoglobin values or with higher Kt/V values) have been disproved by randomized clinical trials (19,20). The more accurate the baseline risk measurement, the less likely results will be biased. This study shows that a few key comorbid conditions provide additional prognostic information that is not found in other routinely available parameters. Given this and the consequences of not adequately controlling for differences in baseline risk, we believe that comorbidity should be included in defining case mix. Documentation about comorbid illnesses is also likely to improve in practice in response to revised CMS regulations that require a more detailed care plan to be maintained in the dialysis unit records (21).
In comparing comorbidity instruments, the DOPPS short list provided increased discriminatory ability and explained variance compared with the CCI and the Form 2728 when used as the sole predictor of mortality, although the advantage was no longer present once other case-mix factors were added. This suggests that a low predialysis systolic BP or serum albumin or other significant laboratory/physiologic parameter confers prognostic information that is comparable in degree to defining higher severity levels for CHF or PVD. Depending on the study question or the source of data, case-mix factors other than comorbidity may not be available, or it may be inappropriate to adjust for these as baseline risk factors. For example, in a quality-of-care analysis, it would be inappropriate to adjust for certain laboratory/clinical parameters because these may reflect care. Administrative databases will not have all of the laboratory/clinical parameters accounted for in these analyses. Comorbidity will play a greater role in defining baseline risk if fewer case-mix factors are incorporated. Our analyses show essentially equivalent prognostic information provided by the DOPPS index, the CCI, and Form 2728, as long as the data are collected as in this study—through dedicated review of the medical records (considered the gold standard for comorbidity information [22]). If administrative data are to be used, then this study identifies 12 key conditions that associate with survival.
The strength of this study is that it is based on the largest collection of comorbidity information in an HD population of this size. This enables the key comorbid conditions to be identified and affords greater precision in estimating the relative weight that each carries. The study population is drawn randomly from practice and is more representative of the general dialysis population than participants of clinical trials. There are also limitations. The risk measurement is based on factors that are measured at a single point in time, yet new conditions may develop and/or existing conditions may worsen with time. Although this might improve prognostication, the case-mix severity measurement serves as a baseline risk measure and accounting for changes in health after baseline would underestimate the effects of a treatment or quality improvement initiative. Another limitation is that the conditions that are most strongly associated with mortality may not be the same as for other outcomes, which were not studied. In addition, the CCI and Form 2728 were scored using data collected with the DOPPS Medical Questionnaire. We believe that results would not change, because the Medical Questionnaire has identical or nearly identical items as the CCI and the Form 2728. Moreover, the original Form 2728 submitted to CMS (which was unavailable because DOPPS is de-identified) does not provide a fair comparison, because this information reflects comorbidities at the start of dialysis, which may have changed at the time of enrollment in DOPPS. The analyses that were conducted to identify the key comorbid conditions that would be available using administrative data are based on data collected through chart review. Administrative data may not be as complete as the medical record, so model performance reported here may be better than would be found in practice. Finally, results were not validated in an external database. Unfortunately, the information collected on the DOPPS Medical Questionnaire is not available in another dialysis population.
Conclusions
Seventeen comorbid conditions contributed 96% of the total explained variance for survival that would be found if a full list of 45 conditions that had face validity as mortality predictors and were significant in univariate analyses were added to a demographics-adjusted survival model. This small number of key conditions contributes additional prognostic information over and above that provided by laboratory and clinical parameters.
Disclosures
Jennifer McCready-Maynes, an employee of Arbor Research Collaborative for Health, helped edit this article.
Acknowledgments
The DOPPS is administered by Arbor Research Collaborative for Health. DOPPS I, II, and III have been supported by research grants from Amgen Inc. and Kyowa Hakko Kirin Co., Ltd. As of January 2009, the DOPPS is additionally funded by Genzyme Corp. Support is provided without restrictions on publications. D.C.M. is supported by a career development grant from the National Institute of Diabetes and Digestive and Kidney Diseases (K23 DK066273).
Footnotes
-
Published online ahead of print. Publication date available at www.cjasn.org.
- Received January 28, 2009.
- Accepted August 11, 2009.
- Copyright © 2009 by the American Society of Nephrology
Appendix 1: Comorbid Conditions on the Medical Questionnaire
The conditions that were considered candidates in the survival models included “any history of coronary artery disease”; “angina associated with exertion or on dialysis within the past 12 mo”; “angina at rest within the past 12 mo”; “myocardial infarction ever”; “cardiac arrest ever”; “atrial fibrillation”; “history of congestive heart failure/ pulmonary edema”; “congestive heart failure requiring hospitalization within the past 12 mo”; “cardiomegaly by chest x-ray”; “left ventricular hypertrophy by electrocardiogram, echocardiography or unspecified but documented in the medical record”; “pericarditis”; “s/p heart valve replacement”; “diagnosis of hypertension (in the medical record, regardless of the cause of ESRD)”; “cerebrovascular accident without major neurologic deficit or transient ischemic attack(s)”; cerebrovascular accident with major residual neurologic deficit”; “prior diagnosis of peripheral vascular disease”; “aortic aneurysm”; “s/p arterial bypass surgery for PVD or surgical repair of aortic aneurysm”; “claudication”; “rest pain of extremities due to PVD”; “recurrent cellulitis, skin infection, gangrene due to PVD”; “amputation due to PVD”; “diabetes, regardless of whether diabetes was the attributed cause of ESRD”; “insulin therapy”; diabetic gastroparesis”; “history of chronic obstructive pulmonary disease”; “use of home oxygen”; “seizure disorder”; “dementia”; “cognitive impairment other than dementia”; “peripheral neuropathy (diabetic or other)”; “Parkinson's disease”; “depression (clinical diagnosis of depression) within the past 12 mo”; “substance abuse within the past 12 mo”; “alcohol abuse within the past 12 mo”; “other psychiatric disorder”; “gastrointestinal bleed within past 12 mo”; “ascites within the past 12 mo”; “diagnosis of hepatitis B”; “hepatitis C”; “history of cancer other than skin cancer within the past 2 yr”; “legally blind”; and “positive HIV or AIDS diagnosis.” The following conditions were combined in the analyses: “HIV” and “AIDS”; “any history of CHF” and “any history of pulmonary edema”; and “LVH by ECG, ECHO, and ‘unspecified’ ” and “stroke without deficit and TIA.”
The following conditions were excluded for the reasons listed: (1) Correlated with other covariates that were considered to be more prognostic: “History of angina,” “myocardial infarction within the past 3 mo,” “coronary angiography,” “coronary bypass surgery ever,” “coronary angioplasty ever,” “coronary angiogram normal,” “permanent pacemaker implanted,” “carotid endarterectomy,” “diabetes treated with oral hypoglycemic agents,” or “diabetic retinopathy”; (2) definition vague or subjective and prone to inconsistency in interpretation: “Other arrhythmias,” “valvular heart disease by echocardiogram,” or “dyspnea at rest or with minimal exertion”; (3) Condition not considered to be prognostic: “Hyperlipidemia,” “limb amputation for reason other than PVD,” “history of deep venous thrombosis,” “carpal tunnel syndrome,” “surgery for carpal tunnel syndrome,” “β-2 microglobulin disease,” “parathyroid surgery,” and “peptic ulcer disease.”
References
In this issue
Jump to section
More in this TOC Section
Original Articles
Epidemiology and Outcomes
Cited By...
- Development and Validation of the Modified Charlson Comorbidity Index in Incident Peritoneal Dialysis Patients: A National Population-Based Approach
- A Validation Study of the Canadian Organ Replacement Register
- Predicting potential survival benefit of renal transplantation in patients with chronic kidney disease