Renal Provider Recognition of Symptoms in Patients on Maintenance Hemodialysis

Study Participants

We conducted a cross-sectional, observational study of long-term hemodialysis patients and their renal providers from February 2006 to July 2006. Study patients and providers were recruited from three dialysis units in Pittsburgh, PA, including one university-affiliated and two community-based units. The university-affiliated unit provides care for a largely urban patient population and is staffed by academic nephrologists. One community-based unit serves an urban, predominantly black patient population, whereas the other services a largely suburban, white group of patients. Private practitioners treat patients at each of these community-based units. These three dialysis facilities were selected to recruit demographically diverse patients as well as a variety of renal providers from academic and community-based practice settings.

Provider and Patient Recruitment

At the initiation of the study, all physicians, physician assistants, and nurse practitioners caring for hemodialysis patients at each of the three participating dialysis units were approached and asked about their willingness to participate in a study of provider awareness of hemodialysis patients’ symptoms. We also approached nurse managers at one site in which patients were seen and evaluated by nurse managers before being evaluated by the renal providers. We included only providers who had experience treating dialysis patients for at least 3 mo and who routinely evaluated patients at the study dialysis units to help ensure the enrollment of clinicians who were familiar with the care of this patient population. Informed consent was obtained from providers who were willing to participate. Standard procedure at these three dialysis units was for the nephrologist to see patients once monthly and nurse practitioners/physician assistants to evaluate patients the alternate 3 wk.

After the enrollment of all providers, patient recruitment commenced. Dialysis technicians approached patients to inquire about their willingness to participate in a study of dialysis-related symptoms during a routine hemodialysis session immediately after they had been seen by an enrolled study provider during dialysis rounds. Patients who expressed interest were approached by the study coordinator to explain the study in detail and to obtain written informed consent. To help ensure the recruitment of patients who were capable of providing reliable responses to the study survey, we administered the Mini-Cog, a combined three-word recall and clock-drawing test, and excluded patients with scores that were indicative of cognitive impairment (18,19). No data were collected from patients who did not consent to participate in the study.

Survey Administration and Data Collection

After patient enrollment, the study coordinator collected basic demographic data from the patient and administered the Dialysis Symptom Index (DSI) during the dialysis treatment. The DSI contains 30 items, each of which targets a specific physical or emotional symptom (20). Patients were asked to identify symptoms that had been present at some time during the previous 7 d by responding “yes” or “no” and to describe the severity of symptoms that had been present using a five-point Likert scale (1 = not bothersome to 5 = bothers very much). Immediately after the administration of the DSI to the patient, the same study investigator located the provider who had just examined the enrolled patient in the dialysis unit to have that provider complete the DSI and identify the symptoms that he or she believed were present in that patient during the previous 7 d and for symptoms that they believed were present, the corresponding severity. The DSI that providers were asked to complete was modified slightly to allow a response of “don't know” for symptoms that he or she was not sure were present. For providers who were unable to complete the survey immediately after rounds, we requested that the survey be completed and returned within 24 h. This process helped ensure the temporal congruity between patient and provider completion of the DSI. Providers were not informed in advance regarding which patients would be approached for enrollment to limit the likelihood that they would alter their assessment of patients during dialysis rounds. Finally, the study investigator abstracted basic clinical and treatment-related data from the dialysis chart including hemoglobin, serum albumin, phosphorous, calcium, and intact parathyroid hormone concentrations and presence of the clinical conditions that comprise the Charlson Comorbidity Index.

Statistical Analyses

The primary analyses were based on assessing provider recognition of symptoms in their patients. To examine the accuracy and predictive value of provider responses for identifying symptoms in their patients, we calculated sensitivity, specificity, and positive (PPV) and negative predictive values (NPV) of provider responses using patient reports as the gold standard and considering provider “don't know” responses as a “no” response. Sensitivity denotes the proportion of patients who have a symptom and are correctly identified by providers as having the symptom, whereas specificity represents the proportion of patients who do not have a symptom and are identified as such by providers. PPV represents the probability that a patient has a symptom if their provider indicates its presence, whereas NPV describes the probability that a patient does not have a symptom if their provider indicates its absence. To examine agreement between patients and providers on the presence/absence of individual symptoms on the DSI, we calculated κ scores. These scores were based on agreement between patient and provider on individual symptoms and do not reflect the total number of symptoms correctly identified by the provider. The κ statistic defines the level of agreement above what is attributable to chance alone. A κ score of 0 to 0.2 signifies poor agreement, 0.2 to 0.4 indicates fair agreement, and >0.4 denotes moderate to good agreement (21). Negative κ scores represent less agreement than would be expected purely by chance. Because the DSI that was administered to patients and providers differed slightly in that providers were allowed to record “don't know” responses for the presence of symptoms, we assessed patient–provider agreement on the presence of symptoms in three different ways. The first method of calculating κ scores, which we considered to be the most clinically informative, was based on considering provider “don't know” responses as disagreement. We also calculated κ scores by considering “don't know” responses as a “no” response for providers and, last, by excluding items for which providers’ responded “don't know” and using only items for which providers responded “yes” or “no” to the presence of a symptom. To assess patient–provider agreement on the severity of those symptoms that both the patient and provider indicated were present, we used weighted κ scores, which give partial credit for provider responses that were close to but not identical to patients’ severity rating.

Multiple logistic regression was used to explore the effects of provider type, years of experience treating dialysis patients, and dialysis unit on the outcome variable of patient–provider agreement on the presence of symptoms (considering “don't know” responses as indicating disagreement). In these analyses, we accounted for clustering of responses by providers and controlled for type of symptom and provider age. We defined statistical significance using a two-tailed P = 0.05. All analyses were conducted using Stata 9 (Stata Corp., College Station, TX). All study procedures were approved by the University of Pittsburgh institutional review board.

Study Population

Overall, 18 of 19 providers and 75 of 79 patients were enrolled in the study; one patient did not pass the Mini-Cog test and was excluded. Ten (56%) providers were from the university-affiliated dialysis unit, and the remaining eight (44%) providers cared for patients at the community-based units. Nine providers were nephrologists, five were nurse practitioners or physician assistants, and four were nurse managers. The mean age of the providers was 41 ± 9 yr (range 27 to 56), and the mean number of years of experience treating dialysis patients was 9 ± 9 yr (range 1 to 27). The median number of surveys completed by providers was 3 (range 1 to 20). Sixteen providers were white, and two were Asian. Of the 75 patient participants, 33 (44%) were from the university-affiliated unit and the remaining 42 (56%) patients received dialysis at one of the two community-based clinics. None of the patients had been on dialysis for <6 mo. Table 1 illustrates the clinical characteristics of study patients.

Accuracy and Predictive Value of Provider Responses

The prevalence and severity of symptoms reported by patients and providers are depicted in Table 2. Providers underreported the presence of 29 (97%) of 30 individual symptoms and, compared with patient-reported severity, underestimated the severity of 19 (63%) of 30 symptoms (Table 2). The frequency of provider “don't know” responses ranged from 3% for “shortness of breath” to 76% for “difficulty becoming sexually aroused.” The sensitivity of provider responses for all symptoms other than “fatigue,” “nausea,” and “shortness of breath” was <50% (Table 3). The PPV of provider reports ranged from 0% for “chest pain” and “difficulty concentrating” to 100% for “dry mouth.” PPV were <75% for 25 individual symptoms (Table 3).

Patient–Provider Agreement on the Presence and Severity of Symptoms

Agreement between patients and providers on the presence and severity of symptoms is displayed in Table 4. Overall, κ scores were lowest when provider “don't know” responses were considered a disagreement and slightly higher when “don't know” responses were considered as a “no” response or excluded. Considering the model in which “don't know” responses were counted as a disagreement, there was poor concordance (κ < 0.2) on the presence of 25 (83%) individual symptoms. Patient–provider agreement on the 12 symptoms that patients described as being most severe (mean severity >3.0) was universally poor (κ < 0.15). Agreement on the severity of individual symptoms was lowest for “feeling sad” (κ = −0.17) and “headache” (κ = −0.15) and highest for “dry mouth” (κ = 0.56). Weighted κ scores for the severity of five symptoms were less than zero, including those for “trouble falling asleep,” “feeling sad,” and “restless legs.”

Associations of Provider Type, Years of Experience, and Dialysis Unit with Symptom Recognition

In the multiple logistic regression model, nurse practitioners and physician assistants were less likely than nephrologists to agree with patients on the presence of symptoms (odds ratio [OR] 0.28; P < 0.001), and although not statistically significant, nurse managers tended to be more likely than nephrologists to agree with patients (OR 2.8; P = 0.21). Each year of providers’ experience was associated with a greater likelihood of agreeing with patients on the presence of symptoms (OR 1.1; P = 0.008). Relative to providers at the university-affiliated dialysis unit, providers at the two community-based clinics were less likely to agree with patients (OR 0.38 and 0.59 respectively; P = 0.01).