Visual Abstract
Abstract
Background and objectives Patients with kidney failure experience depression at rates higher than the general population. Despite the Centers for Medicare and Medicaid Services’ ESRD Quality Incentive Program requirements for routine depression screening for patients with kidney failure, no clear guidance exists.
Design, setting, participants, & measurements For this systematic review, we searched MEDLINE, PsycINFO, and other databases from inception to June 2020. Two investigators screened all abstracts and full text. We included studies assessing patients with kidney failure and compared a tool to a clinical interview or another validated tool (e.g., Beck Depression Inventory II). We abstracted data related to sensitivity and specificity, positive and negative predictive value, and the area under the curve. We evaluated the risk of bias using the Quality Assessment of Diagnostic Accuracy Studies 2.
Results A total of 16 studies evaluated the performance characteristics of depression assessment tools for patients with kidney failure. The Beck Depression Inventory II was by far the best studied. A wide range of thresholds were reported. Shorter tools in the public domain such as the Patient Health Questionnaire 9 and Geriatric Depression Scale 15 (adults over 60) performed well but were not well studied. Short tools such as the Beck Depression Inventory–Fast Screen may be a good option for an initial screen.
Conclusions There is limited research evaluating the diagnostic accuracy of most screening tools for depression in patients with kidney failure, and existing studies may not be generalizable to US populations. Studies suffer from limitations related to methodology quality and/or reporting. Future research should target widely used, free tools such as the Patient Health Questionnaire 2 and the Patient Health Questionnaire 9.
Clinical Trial registry name and registration number: Systematic Review Registration: PROSPERO CRD42020140227.
Introduction
The incidence and prevalence of kidney failure in the United States have increased steadily over the past 4 decades (1). Patients with kidney failure experience major depressive disorder at three to six times the rate of the general US population, depending on the method of assessment (2,3). Comorbid depression is associated with treatment nonadherence, poorer quality of life, worse sleep, more frequent emergency department visits, hospitalizations, suicide, and all-cause mortality (4⇓⇓–7).
The Centers for Medicare and Medicaid Services requires routine depression screening for patients with kidney failure as part of their ESRD Quality Incentive Program (ESRD-QIP) (8). However, there is no system-wide screening protocol, leading to wide variation in the way depression is assessed. Established evidence and guidelines suggest that screening for depression in the general population is both accurate and can improve outcomes (9). However, screening may lead to false positives and concomitant iatrogenic harm from unnecessary pharmacotherapy or resource-heavy psychotherapy. Screening may also lead to false negatives in which depression goes untreated. Patients with kidney failure differ from the general population both because they experience higher rates of comorbid depression, and because they often have symptoms related to their underlying diagnoses and treatments that mimic the somatic symptoms of depression.
Given the wide variation in depression screening options and lack of a gold standard assessment tool for patients with kidney failure, a clear understanding of the validity of the available screening tools is vital. The purpose of this review is to identify depression screening tools (and/or thresholds) appropriate for patients with kidney failure, and to better understand the effect of depression screening in this population.
Materials and Methods
This is part of a larger systematic review commissioned by the Veterans Health Administration that examined both screening and the effectiveness of interventions for patients with kidney failure and comorbid depression (10). The protocol, which follows PRISMA guidelines (11), was registered to PROSPERO before study initiation (CRD42020140227).
Data Sources and Searches
We searched Ovid MEDLINE, PsycINFO, Elsevier EMBASE, and Ovid EBM Reviews Cochrane Database of Systematic Reviews (Database of Abstracts of Reviews of Effects, Health Technology Assessment Database, Cochrane CENTRAL, etc.) from database inception through June 2020. We reviewed the bibliographies of relevant articles and contacted experts to identify additional studies. Search strategies were developed in consultation with a research librarian, and were peer reviewed by a second research librarian using the instrument for Peer Review of Search Strategies (12; Supplemental Material). All studies identified were completed before the onset of the COVID-19 pandemic.
Study Selection
Studies were eligible if they: (1) assessed depression in patients with kidney failure or stage 5 CKD; (2) compared an index (examined) tool to a “gold standard” clinical interview or another well-validated tool (e.g., Beck Depression Inventory II [BDI-II] [13]); (3) were published in English; and (4) examined tools based on criteria from the Diagnostic and Statistical Manual of Mental Disorders IV (DSM-IV) or higher (Supplemental Table 1). Studies were independently reviewed by at least two reviewers. Discordant results were resolved through consensus or a third reviewer.
Data Abstraction and Quality Assessment
From each study, we abstracted details related to sample size, setting, population, inclusion and exclusion criteria, administration and timing of depression screening, the index and reference standard (comparison), and findings. Data were abstracted by one investigator and confirmed by a second. Two reviewers independently assessed study risk of bias using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). The QUADAS-2 is a commonly used 17-item tool for assessing the risk of bias across four domains: (1) flow and timing, (2) application of the reference standard, (3) application of the index test, and (4) patient selection (14). See Supplemental Table 2 for a list of all items. Disagreements were resolved by consensus or a third reviewer.
Data Synthesis
We qualitatively synthesized findings, organized them in tables, and present forest plots of the summary measures (e.g., sensitivity, specificity). The data did not allow for quantitative pooling of results.
Results
We reviewed 8050 titles and abstracts and 189 full text studies. A total of 16 studies were included (Figure 1). Nine studies examined the performance of the BDI-II (13). Other tools include the Cognitive Depression Index (CDI) (15), the Center for Epidemiologic Studies–Depression Scale (CES-D) (16), the Hospital Anxiety and Depression Scale–Depressive Subscale (HADS-D) (17), the Geriatric Depression Scale 15 (GDS-15) (18,19), the Hamilton Depression Rating Scale (20), the Patient Health Questionnaire 9 (PHQ-9) (21), and others. We identified only one study of a depression tool specifically targeting patients on maintenance dialysis (Depression Inventory–Maintenance Hemodialysis [DI-MHD]) (22). Supplemental Table 3 and Table 1 provide study characteristics and Table 2 provides a brief description of the included tools and the gold standard interviews used as reference standards.
Literature flow chart. *After deduplication. CDSR, Cochrane Database of Systematic Reviews; DARE, Database of Abstracts of Reviews of Effects; EBM, Evidence-based Medicine; HSR&D, Health Services Research and Development; HTA, Health Technology Assessment Database; ICTRP, International Clinical Trials Registry Platform; VA, Veterans Affairs; WHO, World Health Organization.
Study characteristics
Characteristics of included screening tools and gold standard semi-structured diagnostic interviews examined
There were five US studies (24,33,35,37,46). Other studies were located in Australia (25), Canada (27), China (22), Italy (29), The Netherlands (30,36), Norway (34), Saudi Arabia (23), Turkey (28), and the United Kingdom (26,30).
Most studies included only patients undergoing hemodialysis. Only four studies also included participants undergoing peritoneal dialysis (32,34,35,37). Across studies reporting time on dialysis, the minimum (mean) months was 8.5 (interquartile range, 4–22) (34) and the maximum was 72.2 (SD=11.7) (28).
Of the 16 studies, 11 compared tools with a gold standard clinical interview (e.g., Structured Clinical Interview for DSM-IV [38]), and five used other established, validated assessment measures (e.g., BDI-II [13]) for comparison.
Seven studies examined thresholds for major depressive disorder (23⇓⇓–26,30,33,37), one of which also categorized less severe depression (23). The remaining nine studies did not describe differences between major depressive disorder, less severe depressive disorders (e.g., dysthymia, pervasive depressive disorder), and subclinical symptoms (Supplemental Table 3) (22,27⇓–29,31,32,34⇓–36).
The 16 studies were relatively similar in quality, with the risk of bias largely unclear for patient selection, the index test, and the reference standard. Figure 2 summarizes the number of studies rated as low, unclear, and high risk of bias across the four QUADAS-2 domains (Supplemental Table 2 reports individual study ratings).
QUADAS-2 risk of bias summary. Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2), independently assessed by two investigators.
Screening Tools Compared with a Gold Standard Clinical Interview
Included studies compared nine tools to clinical interviews, across a range of thresholds. Figure 3 illustrates the performance characteristics by tool and threshold. Supplemental Table 4 and Table 1 provide detail.
Performance characteristics of tools compared with a gold standard clinical interview. BDI-II, Beck Depression Inventory–II; CES-D, Center for Epidemiologic Studies–Depression Scale; CI, confidence interval; DI-HMD, Depression Inventory–Maintenance Hemodialysis; FN, false negative; FP, false positive; GDS-15, Geriatric Depression Scale-15; HADS-D, Hospital Anxiety and Depression Scale–Depressive Subscale; Ham-D, Hamilton Depression Rating Scale; SRQ, Self-Reporting Questionnaire; TN, true negative; TP, true positive.
Beck Depression Inventory II.
Five studies examined the accuracy of the BDI-II in diagnosing major depressive disorder compared with a gold standard clinical interview (24⇓–26,30,37). Sample sizes ranged from 40 (26) to 96 (24). Two were conducted in the United States (24,37). One was a small, multicenter study (n=62) that reported an optimal BDI-II cutoff of ≥16. Sensitivity was 0.91 and specificity was 0.86, with an area under the curve (AUC) of 0.94 (37). The second was a multicenter study of adults 65 and older (n=96). At a cutoff of ≥10, sensitivity was 0.68, specificity was 0.77, and reported AUC was 0.73 (24).
One study examined a range of thresholds (30). The optimal threshold was ≥15, with a reported AUC of 0.93. Another study reported a much lower AUC (24). This study’s population was limited to older adults, and age differences may have contributed to the difference in performance (24).
Among studies screening for depressive symptoms and disorders ranging from subclinical to major depressive disorder (22,31,32,34), sample sizes ranged from 43 (26) to 319 (22). Only one study (n=98) was conducted in the United States (31). At a threshold of ≥14, sensitivity was 0.62, specificity was 0.81, and AUC was 0.77 (31). The largest study (n=319), conducted in China, compared the BDI-II (≥19) to the Structured Clinical Interview for DSM-IV as part of a development and validation study for a depression screen designed specifically for patients undergoing maintenance hemodialysis (22). Sensitivity, specificity, positive predictive value, negative predictive value, and AUC were 0.83, 0.86, 0.63, 0.94, and 0.84, respectively.
Cognitive Depression Index.
Four studies compared the CDI to a gold standard clinical interview (25,26,31,34), of which only two screened specifically for major depressive disorder (25,26). One study (n=45) examined a threshold of ≥11, with a sensitivity, specificity, and AUC of 0.79, 0.81, and 0.94, respectively (25). The second study identified an optimal threshold of ≥10. Sensitivity, specificity, and AUC were 0.78, 0.81, and 0.94, respectively (26).
The two studies screening for the range of depressive symptoms and diagnoses examined thresholds of ≥8 (n=98) (31) and ≥11 (n=109) (34). Sensitivity was 0.50 (31) and 0.82 (34), specificity was 0.83 (31) and 0.93 (34), and AUC was 0.76 (31) and 0.89 (34).
Of note, two studies (26,34) compared the BDI-II and the CDI to a clinical interview and both concluded that the BDI-II performed better.
Center for Epidemiologic Studies–Depression Scale.
A multisite study (n=98) (31) compared the CES-D (≥18) to the Structured Clinical Interview for DSM-IV for depressive disorders and subclinical symptoms. Sensitivity, specificity, and AUC were 0.69, 0.83, and 0.89, respectively.
Depression Inventory–Maintenance Hemodialysis.
A single validation study (n=319) conducted in China compared the Structured Clinical Interview for DSM-IV to both the BDI-II and the DI-MHD and found that at a cutoff of ≥25, the DI-MHD performed better than the BDI-II. Sensitivity, specificity, and AUC were 0.95, 0.93, and 0.94, respectively (22).
Hamilton Depression Rating Scale
A single study (n=45) conducted in Turkey compared the Hamilton Depression Rating Scale (≥10) to the Structured Clinical Interview for DSM-IV and screened for the range of depressive symptoms and disorders. Reported sensitivity was 1.00, specificity was 0.80, and AUC was 0.85 (28).
Hospital Anxiety and Depression Scale–Depression Subscale.
Two studies examined the performance characteristics of the HADS-D (32,34). Both studies screened for depressive disorders and subclinical symptoms. One study (n=62; ≥6) reported sensitivity, specificity, and AUC values of 0.91, 0.76, and 0.89, respectively (32). The other (n=109; ≥8) reported sensitivity, specificity, and AUC values of 0.73, 0.87, and 0.91, respectively. Of note, this study also examined the BDI-II (≥16), and concluded it performed better than the HADS-D (34).
Geriatric Depression Scale 15 (adults aged 60+).
A single study (n=96) compared the GDS-15 (≥5) to a gold standard interview for major depressive disorder. Sensitivity was 0.62, specificity was 0.82, and AUC was 0.81 (24).
Patient Health Questionnaire 9.
A small multisite study (n=62) compared the PHQ-9 (≥10) to the Structured Clinical Interview for DSM-IV for major depressive disorder. Sensitivity and specificity were both 0.92, and AUC was 0.94 (37).
Self-Reporting Questionnaire.
A single small study (n=26) conducted in Saudi Arabia compared the Self-Reporting Questionnaire (≥13) to Structured Clinical Interview for DSM-IV for major depressive disorder. Sensitivity, specificity, and AUC were 1.00, 0.82, and 0.96, respectively (23).
Screening Tools Compared with Other Tools
Five studies compared other, generally short, tools to established, validated tools (i.e., BDI-II, HADS-D; see Figure 4, Supplemental Table 5, Table 1) (27,29,33,35,36). One study screened for major depressive disorder specifically, and evaluated the BDI–Fast Screen (40). It had high sensitivity and specificity compared with the BDI-II (≥16) (33). The GDS-15 (≥6) screened for a range of depression diagnoses and symptoms and appeared to perform well (29).
Performance characteristics of tools compared with a gold standard clinical interview. BDI-FS, Beck Depression Inventory–Fast Screen; ESAS, Edmonton Symptom Assessment System; MHI5, Mental Health Inventory 5.
Timing of Screening
A small (n=43) multisite UK study in outpatient hemodialysis units compared depression screening (BDI-II, CDI) completed on and off dialysis (26). Findings indicated a high level of agreement among patients who were depressed. However, patients who were not depressed had higher mean overall BDI-II (9.6 [6.2] versus 7.3 [5.7], P=0.007) and somatic symptom item scores (4.4 [2.5] versus 3.3 [2.1], P=0.01) on assessments completed while undergoing dialysis (Supplemental Table 3).
Discussion
We identified 16 studies examining the performance characteristics of depression screening tools in patients with kidney failure, and found depression can be accurately diagnosed. By far, the strongest body of evidence addressed the use of longer screening tools and those that require a per-administration cost (e.g., BDI-II), which are not common in medical settings. We found promising evidence that shorter instruments in the public domain such as the PHQ-9 and GDS-15 (for older adults) performed well, but only one study compared each of these instruments to a clinical interview (24,37).
Across studies, sample sizes were small, and studies examined a wide (and inconsistent) range of thresholds. In addition, methodologic details, particularly related to the selection of patients and the conduct and/or interpretation of both the index and reference tests, were generally poorly reported. We identified few studies conducted in the United States, or countries with similar health systems, raising concerns about the generalizability of findings. Except for the BDI-II, the evidence base is quite limited due to few studies examining each tool. There was heterogeneity in how depression was operationalized across studies. Half of the studies evaluated the performance characteristics associated with thresholds intended to screen for major depressive disorder, whereas the other half defined depression broadly, some including subclinical depressive symptoms.
Figure 5 illustrates the effect of sensitivity and specificity across different population-based depression rates. We used data from a US study comparing the PHQ-9 to a gold standard interview (37) that screened specifically for major depressive disorder. At a threshold of ≥10, both sensitivity and specificity were 0.92. Holding these constant, we compared positive and negative predictive values across reported major depressive disorder prevalence rates for (1) general US populations (7.1%) (2); (2) US patients with kidney failure, diagnosed using a gold standard interview (22.8%) (3); and (3) US patients with kidney failure, diagnosed using a screening tool (39.3%) (3). Across populations, the negative predictive values, or accuracy of eliminating depression, are generally high, and false negatives are unlikely. However, the positive predictive values, or accuracy of correctly diagnosing depression, range from 0.47 to 0.88, suggesting in this example that for populations with a lower prevalence of depression, the potential for false positives may be high (Figure 5). Providers should keep these factors in mind when using the results of depression screening tools to guide treatment decisions.
PHQ-9 ≥10 positive and negative predictive values: Three US subpopulations. Both sensitivity and specificity are held constant at 0.92 based on findings from Watnick’s 2005 study comparing the PHQ-9 to a gold standard clinical interview (37). Prevalence data for the general US population comes from the 2017 National Survey on Drug Use and Health (2), and prevalence for patients with kidney disease is reported in a meta-analysis of 249 unique populations (3). NPV, negative predictive value; PHQ-9, Patient Health Questionnaire 9; PPV, positive predictive value.
Among the studies evaluating the BDI-II as a tool to identify major depressive disorder, the threshold that best optimized the balance between sensitivity and specificity for patients with kidney failure was ≥16. In fact, in some studies, the BDI-II performed reasonably well when compared with a gold standard clinical interview. The caveats are the heterogeneity in how tools were administered, and that very few studies contributed data for the same thresholds. Interestingly, two studies found that, compared with a clinical interview, the BDI-II performed better than the CDI, a subset of the BDI without items related to somatic symptoms.
Shorter screening tools compared with established, validated tools performed well overall. Because the ESRD-QIP requires a follow-up after an initial positive screen, these short tools may be appropriate for an initial depression screen of all patients with kidney failure. In particular, the BDI–Fast Screen performed well when compared with the BDI-II. Of note, we identified no studies evaluating the PHQ-2, arguably the most commonly used short screen for depression in US medical settings.
One study examined differences in performance based on the timing of screening and found that participants who were not depressed reported significantly more somatic symptoms when they were screened during dialysis sessions versus off dialysis. Not only were scores on somatic items significantly higher, but BDI-II scores were significantly higher as well. This has implications for dialysis units working to streamline processes, as it illustrates the potential for overdiagnosis and overtreatment.
There are several important limitations; notably, small sample sizes and few studies examining specific tool thresholds. Many studies were conducted outside of the United States and examined participants and health systems that differ from US populations and settings. In addition, the lack of methodology detail reported in many of the studies contributed to uncertainty about study processes and poor or unclear quality ratings. The definition of depression varied widely, which hampered our ability to synthesize the body of research for each tool. Future studies should use standardized language and diagnostic criteria (e.g., DSM-5) (47).
As described above, future research examining the diagnostic accuracy of depression tools in US populations is needed. In addition, although the PHQ-9 is used widely in medical settings, current research in kidney failure populations is extremely limited. Similarly, despite wide use of the PHQ-2 as an initial screen, no studies were identified. Research evaluating performance characteristics of both tools in this population are warranted. There are a handful of studies supporting the use of the BDI-II as a screening tool for major depressive disorder in this population. However, the BDI-II requires a per-use fee, is more commonly used in research and mental health than in medical settings, was developed to align to the out-of-date DSM-IV, and may be less informative for screening and assessment due to its reliance on somatic symptoms. Future research should target free tools (e.g., CES-D, PHQ-9, PHQ-2) that are widely used in US medical settings.
Short, population-targeted tools may be appropriate as an initial screen for depression in dialysis settings, and we identified several high-performing tools that used the BDI-II as a reference standard. However, more research is needed to validate existing findings. Given the ESRD-QIP requirement of both an initial and follow-up screen (if warranted), future research should evaluate both quick screens and those that are more comprehensive. Finally, the DI-MHD appears to be the only screening tool for depression designed specifically for patients with kidney failure. It performed well in a large sample in China. Additional research validating the DI-MHD generally and in English-speaking patients has the potential to affect screening practices in this population.
We identified no studies examining the effect of screening on health outcomes, and only one study that examined differences in implementation. It compared differences in both overall BDI-II scores and somatic item scores when completed on versus off dialysis and touches on only one of many important depression screening implementation issues (e.g., timing, location, administration). Implementation is an important issue, as patients’ responses may differ based on setting and timing, due not only to the experience of somatic symptoms, but also the perception of privacy and other factors. Future implementation research may help to better identify depression in patients with kidney failure and minimize overtreatment. We identified no studies examining potential harm associated with the lack of a standardized depression screening tool (e.g., false positive or negative depression screens contributing to over- or undertreatment). Harm research is especially important in patients with chronic conditions such as kidney failure, given that the physical symptoms of chronic illnesses and their treatment can mimic the somatic symptoms of depression. Also important, but missing, is evidence of potential demographic and clinical differences. Research in these areas will help decision makers to implement screening processes that are not only evidence based, but also the best fit for patient populations.
Our findings have implications for the selection and implementation of depression screening in patients with kidney failure, and highlight the moderate positive predictive values in this population. Clinicians should be prepared to validate positive screens before making treatment decisions that may be burdensome or introduce the possibility of harm.
There is limited research evaluating the diagnostic accuracy of most screening tools for depression in patients with kidney failure, and existing studies may not be generalizable to US populations. Studies suffer from limitations related to methodological quality and/or reporting. Future research should target widely used, free tools such as the PHQ-2 and the PHQ-9.
Disclosures
All authors have nothing to disclose.
Funding
This research was funded by the US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research & Development.
Acknowledgments
The authors wish to thank Ms. Robin Paynter for developing the search strategy and running electronic searches.
The findings and conclusions in this document are those of the authors who are responsible for its contents, the findings and conclusions do not necessarily represent the views of the Department of Veterans Affairs or the US government.
Ms. Chelsea K. Ayers, Dr. Devan Kansagara, Dr. Jennifer R. Antick, Dr. Karli Kondo, and Dr. Pavan Chopra were responsible for the research idea and study design; Dr. Jennifer R. Antick and Dr. Karli Kondo were responsible for study selection and data extraction; Dr. Jennifer R. Antick and Dr. Karli Kondo were responsible for data analysis and interpretation; Ms. Chelsea K. Ayers, Dr. Devan Kansagara, Dr. Jennifer R. Antick, Dr. Karli Kondo, and Dr. Pavan Chopra were responsible for evaluation of study quality and strength of evidence; and Dr. Devan Kansagara, Dr. Jennifer R. Antick, and Dr. Pavan Chopra were responsible for supervision and mentorship. Each author contributed important intellectual content during manuscript drafting or revision, accepts personal accountability for the author’s own contributions, and agrees to ensure that questions pertaining to the accuracy or integrity of any portion of the work are appropriately investigated and resolved.
Supplemental Material
This article contains the following supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.05540420/-/DCSupplemental.
Supplemental Material. Search strategies (parent VA systematic review) and supplemental references.
Supplemental Table 1. PICOTS by key question.
Supplemental Table 2. QUADAS-2 risk of bias assessment.
Supplemental Table 3. Characteristics of studies examining the diagnostic accuracy of depression screening tools in patients with kidney failure.
Supplemental Table 4. Findings of studies examining the diagnostic accuracy of depression screening tools in patients with kidney failure compared with a gold standard diagnostic interview.
Supplemental Table 5. Studies comparing a depression tool to another validated depression tool.
Footnotes
Published online ahead of print. Publication date available at www.cjasn.org.
See related Patient Voice, “Depression: A Side Effect of CKD,” and editorial, “Screening for Depression in People with Kidney Failure,” on pages 1692–1693 and 1702–1704, respectively.
- Received April 20, 2020.
- Accepted October 19, 2020.
- Copyright © 2020 by the American Society of Nephrology
References
In this issue
