Visual Abstract
Abstract
Background and objectives Many aspects of CKD management rely heavily on patient self-care, including medication and dietary adherence, self-monitoring of BP, and daily physical activity. Growing evidence suggests that incorporating smartphone-based applications can support self-care in CKD and chronic disease more generally.
Design, setting, participants, & measurements We identified applications targeting patients with CKD by conducting a search of the US Apple App Store (iOS) and Google Play Store (Android) using the following four phrases: “kidney disease,” “renal,” “dialysis,” and “kidney transplant.” We considered the first 50 applications for each search term on each application store. We adapted a previously described framework for assessment of mobile health applications to account for kidney disease–specific content areas and evaluated applications on their types of patient engagement, quality, usability, and safety. Engagement and quality were assessed by both a patient and a nephrologist, usability was assessed by a patient, and safety was assessed by a nephrologist. Overall, two patients with CKD and three nephrologists performed the evaluations. We examined pairwise correlations between patient, nephrologist, and consumer ratings of application quality.
Results Our search strategy identified 174 unique applications on Android and 165 unique applications on iOS. After excluding applications that were not related to kidney disease, were not patient facing, or were last updated before 2014, 12 Android-only applications, 11 iOS-only applications, and five dual-platform applications remained. Patient and nephrologist application quality ratings, assessed by the net promoter score, were not correlated (r=0.36; P=0.06). Consumer ratings on the application stores did not correlate with patient ratings of application quality (r=0.34; P=0.18).
Conclusions Only a small subset of CKD applications was highly rated by both patients and nephrologists. Patients’ impressions of application quality are not directly linked to consumer application ratings or nephrologist impressions.
- mobile health
- chronic kidney disease
- end stage kidney disease
- Smartphone
- Self Care
- Methyltestosterone
- Malus
- kidney transplantation
- Patient Participation
- blood pressure
- dialysis
- Mobile Applications
- kidney
- Renal Insufficiency, Chronic
- Chronic Disease
- Telemedicine
- Exercise
Introduction
Patient-centered care is at the forefront of CKD management. Nearly every aspect of CKD management relies heavily on patient self-management in collaboration with providers, including medication and dietary adherence, self-monitoring of BP, and daily physical activity. Patient education and self-monitoring have historically relied on face-to-face visits with physicians, handouts, and written diaries. There is growing evidence that incorporating technology, specifically smartphone-based applications (apps), can help support self-management in CKD and chronic disease more generally (1–3). Randomized, controlled trials support the use of mobile health apps in the self-management of diabetes and obesity, yet few studies have formally evaluated apps in the context of kidney disease (4,5). One study evaluating a smartphone-based self-management system in usual care for patients with CKD showed reductions in both systolic and diastolic BP in patients using the smartphone-based intervention, with 80% of patients self-monitoring their BP on a regular basis (3). Additionally, in this study, providers perceived that patients using the smartphone app were more empowered in their CKD care than patients who did not use it.
Patients are already using apps focused on their health. Over three quarters of Americans own smartphones, and smartphone ownership is rising among older adults (46%) and those with low household incomes (67%). The number of health-related mobile apps exceeds 318,000 (6,7). In a survey of United States adults conducted by Pew Research in 2013, 69% reported keeping track of at least one health indicator, such as weight, diet, exercise routine, or symptoms, through their phones (8). Individuals with chronic illnesses are more likely to track their health information and more likely to report that tracking has a direct effect on health compared with healthy adults.
Unfortunately, nephrologists face many challenges when guiding patients to apps appropriate for their CKD self-management needs. First, nephrologists may not be familiar with which apps to recommend and may not be engaged in supporting patients in the use of technology. Second, Pew Research also shows that patients and caregivers do not routinely interact with their providers using apps. Only 18% of individuals report that they share data that they are tracking through technology with their health care provider (8). Additionally problematic is that the apps described in the scientific literature are not usually available to patients on the app stores, which creates a large barrier to providing evidence-based care (9). Third, the majority of apps that track health information may not be safe. For example, some apps do not include alerts for patients when health parameters are dangerously out of range as in the case of extremely high BP (10). Fourth, clinician evaluations of app quality correlate poorly with those from nonclinicians and consumers more broadly, which leads to additional uncertainty about which apps providers should recommend (10). Two systematic reviews of more traditional patient-centered education and support interventions showed that most education/support interventions are heterogeneous and are not necessarily replicable, with clinical effect that is often equivocal (11,12). As such, it is critical that we explore any and all means to support patients’ learning about their CKD diagnosis and engagement in behaviors needed to stay healthy.
To date, there have been no systematic evaluations of CKD smartphone apps with respect to patient engagement, patient and provider evaluations of quality, usability, or safety. The purpose of this study is to identify and systematically evaluate patient-facing Apple iOS and Google Android smartphone apps across the spectrum of CKD care addressing these key areas. By including apps across the continuum of CKD care (predialysis, ESKD, and transplant), this study is designed to be an important step toward elucidating apps that can serve as additional resources for patients and providers to optimize patient support and kidney disease outcomes.
Materials and Methods
Identifying CKD Apps
We identified apps targeting patients with CKD by conducting a search of the US Apple App Store and Google Play Store. We searched for the following four phrases on both app stores: “kidney disease,” “renal,” “dialysis,” and “kidney transplant.” We considered the first 50 apps for each search term on each app store. The search was conducted on November 21, 2016.
Selecting Apps for Review
We removed duplicates when a single app was identified through multiple search terms. We then excluded apps that were not related to kidney disease, were not patient facing, or had been last updated before 2014. We considered apps to be patient facing if the description of the app identified patients as a potential target audience (even if the app also had a clinician-facing component). Both the iOS and Android operating systems underwent significant interface changes in late 2013 (iOS 7 and Android 4.4). We, therefore, chose to remove apps updated before 2014 to eliminate stagnant apps, consistent with prior research in this area (10).
Evaluating Apps
An assessment form was designed and user tested to collect quantitative data on each app in the areas of clinical focus, patient engagement, quality, safety, and cost. We adapted a previously described framework for assessment of mobile health apps to account for kidney disease–specific content areas (10,13). Each app was reviewed by a nephrologist (K.S., C.J.D., and J.W.-N.), an experienced patient with kidney disease (P.M. and J.W.), and a research staff member (S.R., N.A.B., J.R., and Srishti Agrawal). Apps available for both iOS and Android were reviewed on either device. Either iPhones or iPads were used for iOS reviews, and a second generation Google Nexus 7 was used for Android reviews.
Apps were reviewed in their initial downloaded form without making in-app purchases. In-app purchases consist of additional features that can be purchased from within the app. In addition to using the app, reviewers gathered information from the app store pages on the iOS and Android stores and from the app’s website. If these sources did not provide evidence for a particular feature, we assumed that the feature was absent. App evaluations were carried out between January 23, 2017 and August 22, 2017.
The complete evaluation forms are available here: https://kdpsingh.lab.medicine.umich.edu/datasets/ckd-apps-dataset. We ascertained information from apps related to the following areas (text in parentheses indicates which members of the research team evaluated each aspect).
General App-Related Information (Staff).
We ascertained app store ratings, type of developer (individual, nonprofit, for profit, or government agency), app cost, in-app advertising, privacy policy, whether a login was required to use the app, and connectivity with BP monitors and weight scales using Bluetooth technology.
General Medical Information (Nephrologist).
We determined the kidney disease population being targeted (CKD, hemodialysis, peritoneal dialysis, or kidney transplant recipient) and whether the app referenced scientific literature or clinical guidelines (e.g., Kidney Disease Outcomes Quality Initiative and Kidney Disease Improving Global Outcomes).
Patient Engagement (Areas 1–4: Patient and Nephrologist; Areas 5–7: Staff).
We evaluated the intended audience for the app (adults with CKD, children with CKD, caregivers, providers, or the general public) and how useful the app was for engaging patients in one of seven areas: (1) providing educational information; (2) reminding or alerting patients; (3) tracking, displaying, and summarizing health information; (4) providing guidance on the basis of information entered by the patients; (5) enabling communication of health information with others; (6) obtaining support from peers through social networks; or (7) providing rewards to motivate and support behavior change. For areas 1–4, we asked questions about specific CKD-related content areas. For instance, 19 questions assessed education related to CKD-specific dietary recommendations, hypertension, electrolyte abnormalities, and others (full list is in Supplemental Table 1). We considered functionality to be present for areas 1–4 only if patients and nephrologists agreed.
Quality (Patient and Nephrologist).
We assessed the quality of the app through a single question with slight variations for patients and nephrologists: “How likely is it that you would recommend this app to (another patient/your patient)?” This is derived from the net promoter score question, which was originally formulated as a way of measuring customer satisfaction or loyalty as it relates to a company or a product (14,15). The net promoter score is widely used by Fortune 1000 companies and has been used as a measure of health app quality in prior work (10,16).
Usability (Patient).
We asked patient reviewers to complete a system usability scale, which is a validated tool to evaluate usability (17).
Safety (Nephrologist).
For apps that track health information, we asked nephrologists to enter extremely abnormal values (e.g., extremely high BP) that would be indicative of danger to a patient’s health. We considered apps to be unsafe if they did not respond at all to entry of this information.
Statistical Analyses
We used descriptive summary statistics to evaluate the results. We evaluated relationships between clinician ratings, app store ratings, number of app store ratings, usability, and patient ratings using a Spearman rank correlation test. All analyses were performed in R 3.4.1 (R, Vienna, Austria). The study data are available here: https://kdpsingh.lab.medicine.umich.edu/datasets/ckd-apps-dataset.
Results
Overall, our search strategy identified 174 unique apps on Android and 165 unique apps on iOS (Figure 1). After excluding apps that were not related to kidney disease, were not patient facing, were last updated before 2014, or could not be downloaded, five multiplatform apps, 12 Android-only apps, and 11 iOS-only apps remained (Tables 1 and 2). The majority of iOS apps were not rated by consumers, and the largest group of Android apps was rated between four and five of five stars (Table 1). Two apps offered in-app purchases. Individual developers created the majority of the apps, and the majority of apps were free to download. The largest group of apps was aimed at nondialysis, nontransplant CKD care and intended for use primarily by adults or their caregivers (Table 3).
We identified 200 Android and 178 iOS apps using 4 search terms and narrowed this to 12 Android apps, 11 iOS apps, and 5 dual-platform apps after applying inclusion/exclusion criteria. Flowchart of how applications were selected for final evaluation by patients and providers.
Characteristics of applications evaluated in the study
All 28 applications evaluated in this study
Target population for applications
Patient Engagement
We found 15 apps that educate patients, one that reminds or alerts patients, seven that track health information, one that guides patients in response to tracked information, four that enable sharing of health information, five that connect patients to peers via social networks, and two that reward patients for positive health behaviors. We identified seven apps that were deemed to be high performing (greater than or equal to four of five) by both nephrologists and patients on at least one functionality related to how the app engaged patients (Table 4). The high-performing apps covered the entire spectrum of CKD, including two apps focused primarily on earlier stages of CKD (Kidney Disease Assistant and My Kidneys, My Health Handbook), one app focused on hemodialysis (Record Weight of Patients on Dialysis), two apps focused on peritoneal dialysis (C.A.P.D. Helper and Our Journey with Peritoneal Dialysis), and two apps focused on kidney transplant recipients (Syck and Transplant Hero).
Applications rated greater than or equal to four of five on functionality by patients and nephrologists in any kidney disease content area
Quality
Patient ratings of app quality as assessed by the net promoter score question were not correlated with nephrologist ratings (r=0.36; P=0.06). Patient ratings of quality did not correlate with consumer ratings (r=0.34; P=0.18) or the number of consumer ratings (r=−0.10; P=0.61) as shown in Supplemental Figure 1.
Four apps were rated greater than or equal to eight (of ten) on the net promoter score question by patients (How to Prevent Kidney Stones, Transplant Hero, Dialysis Maps, and Dialysis Dictionary). Of these, only one was rated greater than or equal to eight by a nephrologist (Transplant Hero).
Usability
Usability was moderately correlated with patients’ overall rating of the app’s quality (r=0.46; P=0.01) as shown in Supplemental Figure 2.
Safety
Of the seven apps focused on tracking health information, only two (C.A.P.D. Helper and Score Index for Haemodialysis) generated an alert or some form of response on entry of extremely abnormal values that may be considered dangerous or life threatening.
Discussion
We used an established app evaluation framework, adapted with a CKD focus, to assess the patient engagement, quality, usability, and safety of apps supporting self-management of patients with CKD (10). Our approach used multiple stakeholders, including patients with kidney disease, nephrologists, and staff with expertise in information technology, to evaluate different aspects of each app. We found that most of the 28 evaluated apps were designed by individuals, were free to download, targeted adult patients with CKD not on dialysis and their caregivers, and focused on patient education. Patients, nephrologists, and consumers generally did not agree on app quality, potentially reflecting competing perspectives on what makes an app useful. Patient usability was related to app quality, affirming the value of usability assessments of apps. We identified safety as a major concern, because only two of the seven apps with tracking functionality alerted users on entry of extremely abnormal values.
Whereas prior research has shown physician and consumer ratings of apps to be poorly correlated (10), this is the first study to show that patient ratings of apps correlate poorly with both physician ratings and consumer ratings. This is not surprising, because focus groups show that patients with CKD and nephrologists often disagree on which aspects of their care are most important (18–21). Disagreement on the quality of the apps may similarly reflect competing priorities for patients and providers. This has important implications, because national efforts to evaluate apps—led in part by the American Medical Association—have focused on the content quality and security rather than patient perceptions of usefulness (22). This is also relevant in CKD, where prior efforts to evaluate apps solely on the basis of content alone led to 67 apps being recommended, none of which were considered high quality by patients or providers in our study and only one of which was rated as having high patient engagement in at least one area (My Kidneys, My Health Handbook) (23). Some commercial efforts to evaluate health apps have included a patient-derived score, but one popular app review service only included a single app in the “kidney disease” category (24). Incorporating patient input into evaluation of apps may complement efforts to assess content as physicians and medical professional societies consider which self-management tools to recommend to patients with CKD.
Usability and user satisfaction are among the most common end points measured in the scientific literature, and this study affirms their importance in the assessment of apps (9). We found usability to be closely tied to patients’ overall assessment of app quality. This is not entirely surprising, because the first question on the system usability scale (“I think that I would like to use this system frequently”) has some similarities to the net promoter score question used to assess quality (“How likely is it that you would recommend this app to another patient?”). Given that usability is commonly measured and reported in the literature (25–27), it has the potential to serve as a surrogate marker for app quality as long as it is assessed by patients.
Safety has previously been identified as an area of concern for mobile health apps more generally (10), and our work largely confirms this to be true in CKD. Patients using apps to track important health measures may not always know which values are dangerous or require additional attention. When apps do not respond when patients enter abnormal values, this has the potential to severely erode the trust that providers may place in apps. Coupled with the fact that most of the apps that we evaluated were developed by individuals rather than well known organizations, physicians are unlikely to implicitly trust the quality of apps, which likely is in part responsible for the low rates of adoption of some of these apps. Although our initial search did capture several apps developed by the National Kidney Foundation, these were excluded either due to lack of updates or in one case, due to a reviewer being unable to download the app.
Our study has several limitations. First, it was not designed to comprehensively review all apps for kidney disease but rather, was designed to evaluate those apps most likely to have high effect on patients. In particular, our search strategy was not designed to identify apps solely focused on complications of CKD (e.g., hypertension) if the app description did not explicitly mention CKD. Second, the evaluative team was small and may not be reflective of all stakeholders who may interface with patients with CKD. Our methods included both patients and physicians in the review process, offering key perspectives from two of the most critical stakeholders involved in patient-centered CKD management. Third, our app search was conducted in late 2016, after which new apps have entered the market, and some have been removed from the app store. We have made our study data available to facilitate ongoing evaluation of apps in the face of these changes. Lastly, the study team was composed of individuals who likely have a higher degree of health literacy (i.e., patients who are considered advocates willing to participate in this research study) and e-literacy (i.e., patients and providers in this study with interest in apps evaluation) than the general population.
Despite the limitations, we believe that our study has important implications. Although the apps that we evaluated are not necessarily a representative sample of all CKD apps, the fact that the majority focused on predialysis CKD is encouraging given that the majority of Americans with CKD—approximately 30 million people—are not on dialysis. Focusing patient-facing tools on the early stages of kidney disease affords a potential to prevent progression to ESKD. We identified seven apps with at least one high-quality area of patient engagement as determined by both patients and nephrologists—these apps span the entire spectrum of kidney disease and may serve as a template for new apps targeting patients with CKD. There are a number of engagement areas for which we found no apps at all or no high-quality apps, suggesting ample opportunities for patient-facing engagement efforts through apps. One particularly surprising finding is that the majority of apps evaluated focus primarily on patient education rather than tracking of health information, which differs from prior findings in CKD (10). This may be explained by the fact that our study evaluated more CKD apps overall compared with prior work, thus casting a wider net for apps that is more representative of the current marketplace.
In conclusion, only a small subset of CKD apps is highly rated by both patients and nephrologists, and patients’ impressions of app quality are not directly linked to consumer app ratings or nephrologist impressions.
Disclosures
K.S. reports other funding from Blue Cross Blue Shield of Michigan outside the submitted work. S.R. reports other funding from Caravan Health during the conduct of the study. J.W.-N. serves as a nonpaid member of the board of trustees for the American Kidney Fund and intermittently advises work with the National Kidney Disease Education Program.
C.J.D., N.A.B., P.M., J.W., J.R., and T.W. have nothing to disclose.
Supplemental Material
This article contains the following supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.10370818/-/DCSupplemental.
Supplemental Figure 1. Correlation of evaluations (A) between nephrologists and patients, (B) between consumers and patients, and (C) between application store popularity and patients.
Supplemental Figure 2. Correlation of usability as assessed by patients and overall patient recommendation.
Supplemental Table 1. The κ and percentage agreement for application functionality (present or absent) as assessed by patient and nephrologist reviewers.
Acknowledgments
We thank Srishti Agrawal for her assistance with reviews.
K.S. received support from grant 5K12 DK111011 from the National Institute of Diabetes and Digestive and Kidney Diseases. C.J.D. received support from grants K23 DK099385 and R01 DK93938 from the National Institute of Diabetes and Digestive and Kidney Diseases. J.W.-N. received support from grants K23 DK097183 and R01 DK115844 from the National Institute of Diabetes and Digestive and Kidney Diseases.
The funding source had no role in the study design, conduct, analysis, or decision to submit the manuscript.
Footnotes
Published online ahead of print. Publication date available at www.cjasn.org.
See related Patient Voice, “Patient Perspective of Smartphone-Based Apps for CKD Self-Care,” and editorial, “Got CKD? There's an App for That!,” on pages 483–484 and 491–492, respectively.
- Received August 30, 2018.
- Accepted February 15, 2019.
- Copyright © 2019 by the American Society of Nephrology
References
In this issue
Jump to section
More in this TOC Section
Original Articles
Cited By...
- The Mobile Health Readiness of People Receiving In-Center Hemodialysis and Home Dialysis
- Mobile Health in Dialysis: The Best Engagement Medium Is the One thats with Patients
- The Role of Telenephrology in the Management of CKD
- Got CKD? There's an App for That!
- Patient Perspective of Smartphone-Based Apps for CKD Self-Care