The Dialysis Facility Compare Five-Star Rating System at 2 Years

The DFC Star Rating Methodology

The DFC website was the second Compare site to have a summary star rating. After the initial rollout in January of 2015, CMS updated the DFC star rating in October of 2015 and again, in October of 2016 with more recent data. CMS plans regular updates—currently scheduled updates include early 2018 and October of 2018 (2). The DFC star rating combines information from multiple quality measures and assigns a rating between one and five stars. The DFC star rating includes nine of the 18 quality measures on the website with empirically defined domains and weights using principal components analysis.

• (1) Risk-adjusted measures of mortality, hospitalization, and transfusion.

• (2) The percentage of patients dialyzing with a fistula or a catheter for 90 days.

• (3) Three measures of dialysis adequacy (adult hemodialysis, pediatric hemodialysis, and adult peritoneal dialysis) and the percentage of patients with hypercalcemia (2).

The star rating excludes the six publicly reported CAHPS measures and three clinical measures with limited public reporting experience (hospital readmission, pediatric peritoneal dialysis adequacy, and bloodstream infections). Except for the bloodstream infection measure, CMS plans to include most of these measures in the October of 2018 update.

The initial DFC star rating methodology held constant the percentage of dialysis facilities in each category (Figure 1). This fixed distribution was in part a response to the previous experience with Nursing Home Compare, where the use of fixed categories defined originally in 2008 led to 50% of nursing homes receiving four or five stars by 2014 (5).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Upward shift in distribution of dialysis facilities' star rating after updated methodology.

Some stakeholders questioned whether the use of a “bell curve” distribution was appropriate and noted that holding the distribution constant over time obscures industry-wide quality improvement. CMS responded to concerns about the fixed distribution in the methodology update in October of 2016 by allowing the distribution of facilities to change over time (2,6). In the October 2016 release with the updated methodology, the distribution shifted consistent with national improvements in the quality measures (Figure 1).

DFC Star Ratings Compared with Other CMS Star Ratings

In addition to examining how DFC’s methodology has changed, it is also informative to examine how DFC’s current star ratings and underlying methodology compares with the other Medicare Compare websites. Since the initial rollout of the DFC star rating, CMS has continued to issue star ratings for a variety of provider types and measure sets. There are notable differences in star rating methodology across the sites (additional information about the Medicare Compare sites is available at https://www.medicare.gov/) that explain differing distributions along with differences in underlying quality measure data.

• The Nursing Home Compare site includes separate star ratings for state health inspections, staffing, and clinical quality. An overall star rating combines information from these three domains, giving most weight to a within-state ranking on the basis of health inspections.

• The Home Health Compare clinical star rating ranks each home health agency on individual quality measures, with adjustment for statistical significance; the summary star rating is the average of the adjusted rankings of the individual quality measures.

• The Hospital Compare overall star rating combines data from 75 measures through iterative averaging and uses a statistical clustering technique that maximizes differences between groups and minimizes differences within groups of hospitals to define the star rating categories.

• Both home health agencies and hospitals have separate star ratings for patient experience measure sets derived from the CAHPS surveys. For these star ratings, CMS linearizes survey responses on a 0–100 scale and uses the same statistical clustering technique described above.

• The Physician Compare star ratings in development will use an achievable benchmark of care methodology, which will compare each physician with the top 10% of physicians.

Future Directions

Much like the DFC website has evolved since 2001, the DFC star rating will continue to evolve. Updates planned for October 2018 stem from an expert panel in early 2017 that made several recommendations, including the following.

• Add two established measures: pediatric peritoneal dialysis adequacy and the standardized readmission ratio.

• Add a separate star rating for measures from the In-Center Hemodialysis CAHPS.

• Consider when and how to update the star rating categories or “rebaselining” (2,7).

As CMS continues to develop new dialysis quality measures, they are likely to consider these new measures for inclusion in star ratings. Some measures may be quite innovative; indeed, a recent expert panel focused on patient-reported outcome measures (8).

To be useful to patients and caregivers, star ratings must accurately reflect quality. Although star ratings conveniently summarize information from multiple measures, they share many limitations of these measures. Therefore, CMS measure development process is critical to ensuring the quality of star ratings, including regular external review by the National Quality Forum to ensure that measures are important and evidence based, scientifically acceptable (i.e., valid and reliable), usable, feasible, and not duplicative (9). This measure development process also provides a roadmap for CMS to maintain the star ratings.

Other potential efforts include alignment with other CMS star ratings and other public reporting initiatives and increasing patient engagement with quality information. Patients increasingly see multiple types of health care providers, with no one central source for quality information. Variation in star rating methodologies poses challenges to patients using them to make informed decisions. Historically, surveys report that fewer than one in ten patients on dialysis uses quality information (10); however, this is likely to change as the dissemination of health care quality information is increasing in both quantity and quality. The continued expansion of the CMS Medicare Compare websites and star ratings as well as other public reporting efforts reinforce the need for continued research and careful policy in this area that allows flexibility to accommodate the context of each provider, while also ensuring that the information is useful to patients. Regular monitoring should include collecting website usage data and systematic assessments of how patients and caregivers use the information so that CMS can optimize the star ratings.

Disclosures

None.