Abstract
Background and objectives: Because there is wide variation in case-mix adjusted outcomes across dialysis facilities, it is possible that top-performing facilities use practices not shared by others. We sought to catalogue “best practices” that may account for interfacility variations in outcomes.
Design, setting, participants, & measurements: This multidisciplinary study identified candidate best practices in dialysis through a staged process, including systematic review, cognitive interviews, and a national “virtual focus group” of dialysis providers. The resulting candidate practices were rank-ordered by perceived importance as determined by mean RAND Appropriateness Scores from a national survey of nephrologists, nurses, and opinion leaders.
Results: A total of 155 candidate best practices were identified. Among these, respondents believed dialysis outcomes are most strongly related to 1) characteristics of multidisciplinary care conferences, 2) technician proficiency in protecting vascular access, 3) training of nurses to provide education in fluid management, vascular access, and nutrition, 4) use of random and blinded audits of staff performance, and 5) communication and teamwork among staff. In contrast, there was wide disagreement about the importance of facility-based health maintenance practices, optimal staffing ratios, frequency of dialysis-based physician visits, and optimal frequency of multidisciplinary care.
Conclusions: This study provides a “conceptual map” of candidate dialysis best practices and highlights areas of general agreement and disagreement. These findings can help the dialysis community think critically about what may define “best practice” and provide targets for future research in quality improvement.
In 2005, the U.S. Center for Medicare and Medicaid Services announced a pioneering initiative to provide reimbursement based on the quality of care provided to its beneficiaries. Although not yet fully enacted, this “pay-for-performance” (P4P) initiative (1) may redefine how U.S. providers, including nephrologists and dialysis facilities, are reimbursed for their services. In end-stage renal disease, P4P aims to track provider and facility achievement of key evidence-based clinical performance measures (CPMs), including optimization of dialysis dose, achievement of phosphate control, effective anemia management, correction of serum albumin levels, and use of fewer intravascular catheters (2–10).
Yet despite the dissemination of practice guidelines (11), considerable variation remains in achievement of CPM targets and overall mortality across facilities (12–16). Although outcome variation is a natural byproduct of variations in patient populations, data indicate that outcome variation persists even after case-mix adjustment (12,14,16). Because these variations exceed what would be expected by chance alone, there may be other unmeasured facility-level factors or practices affecting these outcomes. These factors may be procedural (e.g., implementation patterns of policies in a facility), attitudinal (e.g., staff morale), interpersonal (e.g., staff communication), belief-oriented (e.g., beliefs about the role of facility-based health maintenance), or structural (e.g., layout of facilities), among others. In short, there may be facility-level “best practices” that explain the variation in outcomes between facilities.
We initiated the Identifying Best Practices in Dialysis (IBPiD) study in preparation for P4P. IBPiD is a multidisciplinary research effort that aims to identify “best practices” that may improve facility-level achievement of CPMs and overall mortality in dialysis patients. IBPiD is organized into three phases (Figure 1). Phase I identifies and catalogues the array of candidate best practices proposed to drive facility-level outcomes in dialysis. This includes a systematic literature review and cognitive interviews of dialysis personnel. Phase II prioritizes these practices by quantifying perceptions of their relative importance. This paper provides the results of phase I and II. Phase III will subsequently test whether the candidate best practices predict CPM achievement and overall outcomes across a range of dialysis organizations.
Three phases of Identifying Best Practices in Dialysis.
Materials and Methods
Phase I: Identifying Candidate Best Practices
Systematic Literature Review.
As a first step toward cataloguing candidate best practices, we performed a systematic review of MEDLINE to identify studies that contain information linking patient-level, physician-level, and facility-level variables with dialysis outcomes. Systematic review is a necessary first step to develop a conceptual map of potential targets for “best practice” and is a technique used throughout other areas of medicine when developing candidate best practices (17–22). Appendix A provides the search strings for the review. Based on this review, we developed a taxonomy to categorize predictors of dialysis outcomes, with the aim of providing a qualitative framework to guide development of our subsequent cognitive interviews.
Demographic and practice-pattern information of “Virtual Focus Group” and phase II national survey respondents
Top-30 “Best Practices” in terms of perceived impact on overall outcomes in dialysis
Perceptions regarding “Best Practice” patient:nurse, patient:technician, and patient:dietician ratios, stratified by provider type
Systematic review search strategy
Dialysis Staff Cognitive Interviews.
Following the literature review, we expanded our conceptual framework by conducting detailed cognitive interviews with dialysis personnel, including technicians, nurses, dieticians, social workers, nephrologists, medical directors, and nurse managers, to elicit perceptions regarding best practices in dialysis care. Cognitive interviews are a technique to explore content in a structured yet flexible format (23,24). We conducted 9 one-on-one interviews in 2 university-based dialysis units, including 2 medical director/nephrologists, 2 nurse managers, 2 floor nurses, 1 dialysis physician assistant, 1 dialysis technician, and 1 dietician. The UCLA Institutional Review Board approved the protocols. Each interview began with an open-ended probe using the “think aloud” technique of cognitive interviewing (23,24). The interviewer then focused the respondent with “scripted probes” (24). The scripted probe technique allowed the interviewer to explore the basis of a response and to apply the a priori structure of the interview to the open-ended format.
National Online “Virtual Focus Group.”
We supplemented the interviews with a national online “virtual focus group” of a random sample of 250 nurse and nurse manager members of the American Nephrology Nurses' Association (ANNA), 250 community nephrologists from the AMA Masterfile, the 2000 clinical members of the Renal Physician Association, and 50 academic opinion leaders in dialysis care. The online “virtual focus group” asked respondents to “think of any processes of care that might distinguish ‘high performing’ dialysis facilities from ‘low performing’ dialysis facilities.”
Cataloguing Candidate Best Practices.
After abstracting the interviews and national online elicitations, we compiled an inventory of candidate best practices and created a series of questionnaires with Likert response scales to operationalize the practices. We designed these questionnaires to provide dialysis administrators with a standardized approach to measure staff perceptions about adherence with candidate best practices in their units.
Phase II: Quantifying Perceptions of Appropriateness and Importance of Candidate Best Practices
Overview of National Perceptions Survey.
After abstracting the cognitive interviews and national online elicitations, we compiled an inventory of candidate best practices. We then rank-ordered these practices in a national provider survey on the basis of: 1) perceived appropriateness, 2) perceived importance, and 3) presence and magnitude of variation in provider beliefs both within and between groups (25–27).
Sampling frame and Survey Distribution.
We surveyed dialysis opinion leaders and randomly selected community nephrologists and dialysis nurses from the RPA, AMA, and ANNA. The survey was mailed and administered on the Internet. Using baseline data from the RPA, ANNA, and AMA, we compared responders to nonresponders across key covariates.
Analyses.
Respondents rated the perceived importance of each candidate best practice using a standard 9-point RAND/UCLA Appropriateness Scale (RAS) with the following interpretation: 1 to 3 = “generally not appropriate”; 4 to 6 = “moderately appropriate”; 7 to 9 = “generally appropriate” (26).
We rank-ordered the perceived importance of the candidate best practices by mean RAS. In addition, we conducted analysis of variance to compare the mean RAS scores across groups for each practice. We defined “endorsement” of a candidate best practice with a score of 7 to 9 on the RAS, and we conducted χ2 tests to compare endorsement across the provider groups, a technique used in provider surveys in other areas of medicine (27). We then performed multivariate regression to determine if any provider or practice-type characteristics were associated with endorsement of individual practices. To quantify the level of agreement, we calculated the RAND/UCLA Disagreement Index (DI) for each candidate best practice (26). A DI >1.0 indicates “extreme variation” in beliefs; DI <1.0, including negative values, reflect increasing levels of agreement (26).
Results
Phase I: Identifying Candidate Best Practices
Figure 2 lists the complete inventory of candidate best practices resulting from phase I. These 155 candidate practices are categorized into to 8 “major domains,” including: 1) physician practices, 2) staff working climate, 3) facility characteristics and amenities, 4) facility-based health maintenance, 5) technician practices, 6) nursing practices, 7) general dialysis care practices, and 8) miscellaneous practices. These are subdivided into 23 “minor domains.” The candidate best practices have been structured into questionnaires for potential use with facility staff. The full questionnaires can be viewed under the “surveys” tab at www.ResearchCORE.org.
Complete inventory of candidate best practices resulting from phase I.
Phase II: Quantifying Perceptions of Appropriateness and Importance of Candidate Best Practices
Table 1 displays the characteristics of the survey respondents. There were 342 respondents, including 42% of opinion leaders (21 of 50), 24% of AMA nephrologists (60 of 250), 10% of RPA nephrologists (200 of 2000), and 38% of ANNA nurses (95 of 250). There were no statistically significant differences in mean age, gender, years in practice, proportion in management, or region of practice between responders and nonresponders.
To help establish a prioritization among the 155 candidate best practices identified during phase I, the phase II survey asked respondents to measure perceived importance of each candidate best practice vis-a-vis its potential impact on improving overall facility outcomes. Table 2 lists the Top-30 candidate best practices perceived by respondents as being most impactful, as rank-ordered by mean RAND Appropriateness Scores.
In addition to rank-ordering candidate practices, we also focused phase II on the candidate best practices that were characterized by disagreement and controversy in the cognitive interviews. Using this criterion, we identified 4 categories for evaluation: 1) facility-based health maintenance practices, 2) staffing ratios, 3) frequency of dialysis-based physician visits, and 4) frequency of multidisciplinary patient care conferences. Because of strong disagreement about “best practices” in these domains, the phase II national survey focused in these areas of contention to better quantify perceptions.
Perceptions Regarding Facility-based Health Maintenance Practices
There is debate about what role dialysis facilities should play in providing health maintenance services (e.g., immunizations, referral for age-appropriate cancer screening). Because disagreement was also evident in our phase I focus groups, we measured provider beliefs about the appropriateness of offering health maintenance services in dialysis facilities. Thirty-six percent of respondents reported that routine health maintenance is “generally appropriate.” This belief did not vary significantly between provider groups. However, within groups, opinion leaders were internally consistent in their perception (DI = −6.2), whereas nurses (DI = 1.7), medical directors (DI = 1.6), and nephrologists (DI = 3.1) exhibited “extreme variation” in their opinion of facility-based health maintenance services (i.e., DI >1).
Perceptions Regarding “Best Practice” Staffing Ratios
Because phase I revealed disagreements regarding best practice staffing ratios, we formally quantified opinions regarding staffing ratios in our phase II national survey. We elicited perceptions regarding patient-to-nurse ratios, patient-to-technician ratios, and patient-to-dietician ratios. For each pairing, we asked: “What do you think is the optimal ratio to provide quality care in a resource-limited environment?” Table 3 displays the results. Nurses were willing to maintain a higher patient-to-nurse ratio (9.5:1) than community nephrologists (8:1). Dialysis managers and opinion leaders were willing to maintain a higher patient-to-dietician ratio (70:1) than nurses (48:1) and community nephrologists (56:1). All providers agreed that the best practice patient-to-technician ratio should be between 3 and 4:1.
Perceptions Regarding Dialysis-based Physician Visits
Because phase I revealed disagreement about the “best practice” frequency of dialysis-based nephrology visits, we posed the following question in the phase II survey: “Recognizing that physicians are busy, how frequently do you think physicians should routinely visit stable hemodialysis patients in the dialysis facility?” Forty-six percent believed nephrologists should visit their patients at least weekly, whereas 25% believed physicians should visit only monthly or less frequently. The remainder believed nephrologists should visit every other week (30%). Nurses were more likely to endorse frequent (i.e., ≥weekly) visits versus physicians (P = 0.04). Similarly, physicians with larger patient loads (>50 patients/wk) were more likely to endorse frequent visits versus less busy physicians (P = 0.001).
Perceptions Regarding “Best Practice” Multidisciplinary Conference (MDC) Practices
The essential elements for a successful MDC have not been well described, and participants in phase I revealed disagreement regarding “best practice” frequency of MDCs. To measure perceptions about best practice frequency, the phase II survey posed the following question: “Recognizing the concerns of very busy staff and scheduling conflicts, how frequently should multidisciplinary care rounds be held for an ‘average,’ stable hemodialysis patient?” Five percent believed MDCs should be held at least every other week, whereas 33% believed every 6 mo is appropriate. The remainder believed MDCs should be held monthly (35%), every other month (7%), or every third month (20%). Nurses were more likely to endorse frequent (i.e., <monthly) MDCs compared with physicians (P = 0.0004). Similarly, providers with patients in large facilities (>20 chairs) were more likely to endorse frequent MDCs versus providers in smaller facilities (P = 0.001).
Discussion
It is important to formally understand how dialysis facilities can improve their outcomes. Despite the dissemination of K/DOQI guidelines (11), data indicate that adjusted outcomes vary by up to 30% between U.S. dialysis facilities (12–16). It is possible that top-performing facilities enact specific practices that bottom-performing facilities do not enact. We sought to identify candidate “best practices” that may account for variation in outcomes between top and bottom performing centers.
In reviewing Figure 2, it becomes clear that dialysis personnel believe that facility-level outcomes are related to a wide range of factors. In particular, respondents believe that outcomes are related to a range of issues pertaining to interpersonal relationships within the dialysis facility (e.g., level of mutual respect, open communication, staff empowerment). Similarly, we found general consensus that structural characteristics of dialysis facilities may be important predictors of facility-level outcomes. Specifically, respondents perceived that even mundane factors such as the position of nurse workstations, location of patient weigh-in scales (to avoid “cheating”), and configuration of pods may partly dictate outcomes. Respondents also believed that facility-level outcomes may be related to a series of procedural best practices, including physician-level practices (e.g., rapidly returning pages), nurse-level practices (e.g., creating nursing teams that optimally match experience), staff enhancement practices (e.g., appointing a dedicated anemia manager), and dialysis scheduling practices (e.g., developing formal systems to correct short-staffing), among many others.
It is unlikely that all 155 candidate “best practices” in Figure 2 are equally important. To help determine which proposed best practice are most important for future research and targeting, we rank-ordered the perceived impact of the candidate practices based on rankings from a national survey nephrologists, dialysis nurses, and dialysis managers (Table 2). The results demonstrate several trends, including: 1) a successful MDC is critical to ensure optimal patient outcomes, and it is particularly important that the attending nephrologist attend all conferences for their own patients; 2) it is critical to audit technician proficiency in obtaining vascular access, with special attention paid to using aseptic technique, consistently using a tourniquet, and assessing the bruit and thrill of fistulae; 3) nurses must be trained and highly proficient in educating patients across a range of topics, including fluid management, vascular access care, and nutrition; 4) dialysis units should employ random and blinded reviews of nurse and technician performance; and 5) communication and teamwork among the nursing staff should be enhanced with various maneuvers, including maintaining an intershift “communication log book,” creating nursing teams so that less experienced nurses are paired with senior nurses, using a “team nursing” approach instead of one-on-one nursing, and matching nurses' cultural and language backgrounds to individual patient cultural and language backgrounds.
Although we found consensus regarding many candidate best practices, we also found wide disagreement about several aspects of dialysis care, including: 1) facility-based health maintenance practices, 2) staffing ratios, 3) frequency of dialysis-based physician visits, and 4) frequency of multidisciplinary patient care conferences.
There is debate about what role dialysis facilities should play in providing health maintenance services. Because patients are a “captive audience” in the dialysis facility, there may be opportunities to provide healthcare maintenance and preventive care (28–30). However, many providers believe these services are more appropriately delivered outside of the dialysis facility. We found that one-third of providers in our survey believed that routine health maintenance is appropriate, whereas the rest were either neutral or believed these services are inappropriate. This suggests that the dialysis community lacks clarity about the role of dialysis facilities in providing routine health maintenance services. The “extreme variation” in beliefs (as measured by a DI >1.0) indicates that future research should formally measure the relationship between facility health maintenance activities and overall facility outcomes.
Staffing of dialysis clinics has changed considerably over the past two decades. Changes include relying more heavily on technicians to provide care and increasing the panel size of patients cared for by dieticians and social workers. Although minimum staffing ratios exist, it remains unclear what defines a “best practice” ratio given the often resource-limited environment of dialysis care. We found wide variation in perceived best practice staffing ratios, particularly for patient-to-nurse and patient-to-dietician ratios. Future research should measure the effectiveness and cost-effectiveness of varying staff ratios on patient and facility outcomes.
We found significant disagreement about the best practice frequency of dialysis-based physician visits. Whereas one-half of respondents believed nephrologists should visit their dialysis patients at least weekly, the other half believed less frequent visitation is appropriate. Although the focus has been on quantity of visits, the quality of the visits may be even more relevant. The semiprivate environment and limited accessibility of vascular access arm may reduce the value of mandatory income-generating weekly visit, whereas less frequent but more in-depth visits outside the dialysis clinic (e.g., nephrologist's office) are reimbursed at a lower level. Further research should explore the effectiveness of varying physician visitation schedules/venues in stable dialysis patients before establishing a best practice frequency of dialysis-based visits.
Finally, we found striking variation in the perceived optimal frequency of MDCs for stable hemodialysis patients. Nurses appear to value frequent MDCs more than physicians, as do providers with patients in large facilities. MDCs are an essential component of end-stage renal disease patient care and are required for dialysis facility reimbursement by CMS. Although the enhanced coordination of care is a touted benefit of these conferences, it is unclear what aspects of an MDC influence outcomes. Future research should measure the effectiveness of varying MDC schedules in stable dialysis patients before developing any best practice criteria.
Although we used quantitative methods in this study, the candidate best practices were largely derived through a qualitative approach. Qualitative research has pros and cons. In this instance, the “pro” is that we have systematically surveyed a range of dialysis providers to identify candidate best practices based on their experience and knowledge of the field, an approach that has become standard for developing initial candidate best practices in other areas of medicine (17–22). Moreover, we have used quantitative methods to rank-order candidate practices as a form of prioritization. The “con” of this type of qualitative research is that, although the process identifies novel ideas about how best to improve outcomes, it does not prove that these ideas are necessarily valid. Although the next research step is to empirically measure the link between these candidate practices and outcomes, this could not occur in the absence of a conceptual map. The current study provides the map. The next step is to empirically determine what really matters and what is mere “noise” now that the conceptual map has been laid out.
In the meantime, the data from this study can help dialysis organizations think critically about what may define “best practices” and provide a roadmap, based on current provider perceptions, for how to improve overall outcomes of dialysis. Moreover, it provides data regarding specific areas of controversy, including facility-based health maintenance practices, staffing ratios, frequency of dialysis-based physician visits, and characteristics of MDCs. As part of quality improvement efforts to standardize care, future research should be targeted in these areas of variation.
Disclosures
None.
Acknowledgments
B.M.R.S. was supported by VA HSR&D Research Career Development Award RCD 03 to 179–2 and by the CURE Research Center (NIH 2P30 DK 041301–17). A.A.D. was supported by a training grant from the Agency for Healthcare Research and Quality. M.G. is an employee of Amgen, Inc. Support for this investigator-initiated study was provided by a research grant from Amgen, Inc.
Footnotes
Published online ahead of print. Publication date available at www.cjasn.org.
- Received October 18, 2007.
- Accepted February 26, 2008.
- Copyright © 2008 by the American Society of Nephrology
References
In this issue
Jump to section
More in this TOC Section
Cited By...
- Clinical Factors and the Decision to Transfuse Chronic Dialysis Patients
- Reimbursement of Dialysis: A Comparison of Seven Countries
- Processes of Care and Reduced Mortality among Hemodialysis Patients in the United States
- Dialysis Practices That Distinguish Facilities with Below- versus Above-Expected Mortality
- Is there "Cherry Picking" in the ESRD Program? Perceptions from a Dialysis Provider Survey