Skip to main content

Main menu

  • Home
  • Content
    • Published Ahead of Print
    • Current Issue
    • Podcasts
    • Subject Collections
    • Archives
    • ASN Meeting Abstracts
    • Saved Searches
  • Authors
    • Submit a Manuscript
    • Author Resources
    • Reprint Information
  • Trainees
    • Peer Review Program
    • Prize Competition
  • About CJASN
    • About CJASN
    • Editorial Team
    • CJASN Impact
    • CJASN Recognitions
  • More
    • Alerts
    • Advertising
    • Reprint Information
    • Subscriptions
    • Feedback
  • ASN Kidney News
  • Other
    • JASN
    • Kidney360
    • Kidney News Online
    • American Society of Nephrology

User menu

  • Subscribe
  • My alerts
  • Log in
  • My Cart

Search

  • Advanced search
American Society of Nephrology
  • Other
    • JASN
    • Kidney360
    • Kidney News Online
    • American Society of Nephrology
  • Subscribe
  • My alerts
  • Log in
  • My Cart
Advertisement
American Society of Nephrology

Advanced Search

  • Home
  • Content
    • Published Ahead of Print
    • Current Issue
    • Podcasts
    • Subject Collections
    • Archives
    • ASN Meeting Abstracts
    • Saved Searches
  • Authors
    • Submit a Manuscript
    • Author Resources
    • Reprint Information
  • Trainees
    • Peer Review Program
    • Prize Competition
  • About CJASN
    • About CJASN
    • Editorial Team
    • CJASN Impact
    • CJASN Recognitions
  • More
    • Alerts
    • Advertising
    • Reprint Information
    • Subscriptions
    • Feedback
  • ASN Kidney News
  • Visit ASN on Facebook
  • Follow CJASN on Twitter
  • CJASN RSS
  • Community Forum
Perspectives
Open Access

SONAR

Do a New Design and Statistically Significant Results Translate to Reliability?

Michael Walsh
CJASN June 2020, 15 (6) 889-891; DOI: https://doi.org/10.2215/CJN.08540719
Michael Walsh
Population Health Research Institute, Hamilton Health Sciences and McMaster University, Hamilton, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Info & Metrics
  • View PDF
Loading
  • atrasentan
  • randomized controlled trials
  • diabetic nephropathy
  • humans
  • albuminuria
  • diabetic nephropathies
  • endothelin A receptor antagonists
  • type 2 diabetes mellitus
  • creatinine
  • nephrology
  • incidence
  • confidence intervals
  • endothelin A receptor
  • sample size
  • reproducibility of results
  • pyrrolidines
  • chronic renal insufficiency
  • chronic kidney failure
  • kidney

At the World Congress of Nephrology in 2019, two trials testing medications to reduce the risk of progressing diabetic kidney disease were presented. Almost unheard of in our field, both trials reported “positive” headline results, the Atrasentan and Renal Events in Patients with Type 2 Diabetes and CKD Trial (SONAR) and Canagliflozin and renal outcomes in type 2 diabetes and nephropathy (CREDENCE) trial (1,2). SONAR tested whether atrasentan, an selective endothelin A receptor antagonist, reduced the incidence of the primary outcome of a doubling of serum creatinine, ESKD, or kidney death in patients with diabetic kidney disease that experienced at least a 30% reduction in albuminuria when treated with open-label atrasentan during a run-in period (2). The trial was stopped early with only about half the planned sample size (and outcome events) because of slow recruitment, but ultimately the trial reported a significant lower risk of the primary (kidney) outcome in the atrasentan group (hazard ratio, 0.65; 95% confidence interval, 0.49 to 0.88; P=0.005). As with any trial, SONAR raises many questions, but in this case there are questions about the novel trial design and the credibility of the observed treatment effect. These questions merit further thought both because it is important to consider how atrasentan may play a role in the treatment of diabetic kidney disease and because innovative designs in randomized, controlled trials are likely to be more commonly used.

In any randomized, controlled trial, eligible participants should be those likely to benefit from the experimental treatment. Testing the effects of treatments in those unlikely to benefit exposes those participants to unnecessary risks and reduces the probability of identifying benefits. Unfortunately, medicine is still relatively crude in how “likelihood to benefit” is determined. Mostly, a range of pathophysiologies are assigned a diagnosis and, on the basis of this imprecise diagnosis, we consider patients eligible for a treatment (after excluding patients for whom the treatment would be likely dangerous). For example, in the treatment of myocardial infarctions with streptokinase, patients with coronary syndromes owing to vasospasm or dissection were unlikely to benefit from streptokinase, a drug for thromboses. However, it was difficult to differentiate these diagnoses from myocardial infarctions caused by plaque rupture and thrombosis within the time frame the drug was thought to be useful. Some patients with a pathology unlikely to benefit may therefore have been included but the trials still found streptokinase was, on average, effective presumably because such patients were a small proportion of the total (3).

Ideally, investigators pair pathophysiology with a treatment’s known mechanism(s) of action. This highlighted in modern oncology where malignancies have specific characteristics that make them susceptible to particular drugs. For example, the discovery that concomitant therapy with a mAb against HER-2 improved the survival of patients with HER-2–positive breast cancer (4). Now malignancies are being increasingly screened for markers that provide clues to their susceptibilities to particularly therapies. However, it is important to recognize that these approaches still have limitations and biomarker guided therapies, no matter how logical the treatment approach seemed, still do not always improve on less precise strategies.

SONAR was designed on the basis of the observation that risk of progressive kidney disease is correlated to albuminuria. Therefore, a drug that reduces albuminuria in some patients (a biologic response) but not in others may be more likely to exert a clinically important benefit in those with a biologic response. Enrolling only those with a biologic response makes sense as it would spare some the side effects and burden of a treatment unlikely to benefit them, while improving statistical power for the trial. There are caveats to the usefulness this design. Most importantly, the biologic response is most likely to predict a clinical treatment response if the biologic response lies on the causal pathway. History tells us that our assumptions that associations between biologic responses and clinical response, no matter how consistent and strong, are causal, are frequently incorrect (5). Second, if a biologic response is rare, the usefulness of the therapy to the population as a whole is likely limited. If a biologic response is almost universal, the extra step to determine a response is wasteful. If the biologic response is very delayed, participants are exposed to treatment-related harms that are not taken into account in the randomized part of the trial. Finally, if there is substantial measurement error in the biologic response, it will simply reduce the number of eligible patients rather than add statistical power. Regardless of these issue, the design limits the generalizability of the results to patients with a biologic response. In SONAR’s case, the results are only generalizable to treated patients whose albuminuria falls ≥30% after starting atrasentan (which is not the same as those in which atrasentan caused the reduction, because some changes are because of random variation).

None of these caveats fundamentally reduce the validity of the findings of the SONAR trial. The most important part of this strategy is that a biologic response is assessed before randomization. If the trial differentially treated or followed participants on the basis of response after randomization, the results would be at high risk of bias. So, the design is valid, but limits the application of the results to patients with ≥30% drop in albuminuria after starting atrasentan.

Importantly, we need to consider the effect of excluding participants that did not experience a reduction in albuminuria. The SONAR investigators, wisely, studied these patients and in their presentation but not in the article, the 1020 nonresponders did not have dissimilar results to the responders. This suggests that including only participants with an albuminuric response may not adequately identify those most likely to benefit.

Once we accept that the design of SONAR is valid, we need to decide if the observed difference in outcomes was truly the effects of the drug. This is conventionally done by ensuring the difference is unlikely attributable to the play of chance (i.e., statistically significant), that the risk of bias was reasonably low, and that the totality of evidence is sufficiently consistent (6).

From the perspective of ensuring the play of chance was unlikely, SONAR meets conventional criteria. The P value for the hypothesis test is low (not just <0.05 but <0.005!) and the 95% confidence interval excludes the null result. However, the fallibility of these metrics, and particularly treating them as binary is well documented (7). Simulation studies on the basis of cardiovascular outcome trials suggest treatment effects may be severely misestimated unless they use at least 600 outcome events (8). In the SONAR trial, there were 184 primary outcome events. Another way of thinking about this is with the Fragility Index, the number of events added to the intervention group that are required for the P value to slip from <0.05 to ≥0.05 (9). In the case of SONAR, only one extra event in the atrasentan group would potentially change the statistical inference in SONAR. A difference of one event is not difficult to imagine—it could happen by a random occurrence of a low eGFR, such as one patient taking extra nonsteroidal anti-inflammatory drugs or having a diarrheal illness in the placebo group, or one death occurring in the atrasentan group after a patient starts dialysis but before 90 days elapse. It is also notable that although the statistical significance would be lost by a different outcome in a single patient, 369 patients did not complete the study, of which 43 were lost to follow-up, and their outcomes could have easily contributed that one event difference. Similarly, RCTs stopped early for benefit, not the case with SONAR, consistently overestimate treatment effects (10). Taken together, these suggest there is insufficient data to make firm conclusions about whether atrasentan truly benefits patients or not.

In terms of risk of bias, SONAR had all the design elements to minimize bias: allocation was concealed, the investigators, patients, care providers and outcome adjudicators were all blinded, and follow-up and outcome ascertainment were excellent and similar in both groups.

Finally, consider the consistency of the results of SONAR, internally and with other trials. Internally, the results are broadly similar across the various definitions of the kidney outcomes and across the subgroups. Externally, there is little to compare with in terms of drugs in this class as they did not progress to the same stage of development. However, if we consider other effective drugs for preventing progression of diabetic kidney disease, both angiotensin receptor blockers and SGLT-2 inhibitors may give us an idea about expected treatment effects. Angiotensin receptor blockade reduces ESKD and doubling of creatinine by about 21% (11). Canagliflozin reduced a similar outcome by about 30%. In reality, the SONAR investigators powered the trial to detect (and therefore expected) a 27% reduction and the need for 425 outcome events. Although not unimaginable, the finding of a 35% reduction may stretch the limits of plausibility.

On the whole, the SONAR investigators should be commended for designing a trial that should have improved how we test and ultimately prescribe a new, promising treatment. However, because of its early termination, whether atrasentan truly reduces the risk of progressive diabetic kidney disease remains in question, as is whether future trials should use similar design. The use of a response in albuminuria may ultimately have created a logistical barrier to trial recruitment but without adding much in the way of power, and may pose a knowledge translation issue for future results. Although we certainly need to put careful thought into how we move forward with randomized trials, decades-old wisdom may still be very relevant today, and questions around common conditions are most reliably answered with a large, simple trial.

Disclosures

Dr. Walsh has nothing to disclose.

Funding

None.

Acknowledgments

Dr. Walsh is supported by a Mid-Career Research Award from McMaster University.

The content of this article does not reflect the views or opinions of the American Society of Nephrology (ASN) or CJASN. Responsibility for the information and views expressed therein lies entirely with the author(s).

Footnotes

  • Published online ahead of print. Publication date available at www.cjasn.org.

  • Copyright © 2020 by the American Society of Nephrology

References

  1. ↵
    1. Perkovic V,
    2. Jardine MJ,
    3. Neal B,
    4. Bompoint S,
    5. Heerspink HJL,
    6. Charytan DM,
    7. Edwards R,
    8. Agarwal R,
    9. Bakris G,
    10. Bull S,
    11. Cannon CP,
    12. Capuano G,
    13. Chu PL,
    14. de Zeeuw D,
    15. Greene T,
    16. Levin A,
    17. Pollock C,
    18. Wheeler DC,
    19. Yavin Y,
    20. Zhang H,
    21. Zinman B,
    22. Meininger G,
    23. Brenner BM,
    24. Mahaffey KW
    ; CREDENCE Trial Investigators: Canagliflozin and renal outcomes in type 2 diabetes and nephropathy. N Engl J Med 380: 2295–2306, 2019
    OpenUrlCrossRefPubMed
  2. ↵
    1. Heerspink HJL,
    2. Parving HH,
    3. Andress DL,
    4. Bakris G,
    5. Correa-Rotter R,
    6. Hou FF,
    7. Kitzman DW,
    8. Kohan D,
    9. Makino H,
    10. McMurray JJV,
    11. Melnick JZ,
    12. Miller MG,
    13. Pergola PE,
    14. Perkovic V,
    15. Tobe S,
    16. Yi T,
    17. Wigderson M,
    18. de Zeeuw D
    ; SONAR Committees and Investigators: Atrasentan and renal events in patients with type 2 diabetes and chronic kidney disease (SONAR): A double-blind, randomised, placebo-controlled trial. Lancet 393: 1937–1947, 2019
    OpenUrlCrossRefPubMed
  3. ↵
    Intravenous streptokinase in acute myocardial infarction. N Engl J Med 315: 1356–1357, 1986
    OpenUrlPubMed
  4. ↵
    1. Romond EH,
    2. Perez EA,
    3. Bryant J,
    4. Suman VJ,
    5. Geyer CE Jr.,
    6. Davidson NE,
    7. Tan-Chiu E,
    8. Martino S,
    9. Paik S,
    10. Kaufman PA,
    11. Swain SM,
    12. Pisansky TM,
    13. Fehrenbacher L,
    14. Kutteh LA,
    15. Vogel VG,
    16. Visscher DW,
    17. Yothers G,
    18. Jenkins RB,
    19. Brown AM,
    20. Dakhil SR,
    21. Mamounas EP,
    22. Lingle WL,
    23. Klein PM,
    24. Ingle JN,
    25. Wolmark N
    : Trastuzumab plus adjuvant chemotherapy for operable HER2-positive breast cancer. N Engl J Med 353: 1673–1684, 2005
    OpenUrlCrossRefPubMed
  5. ↵
    1. Baker SG,
    2. Kramer BS
    : A perfect correlate does not a surrogate make. BMC Med Res Methodol 3: 16, 2003
    OpenUrlCrossRefPubMed
  6. ↵
    1. Guyatt G,
    2. Rennie D,
    3. Meade MO,
    4. Cook DJ
    1. Walsh M,
    2. Perkovic V,
    3. Manns B
    : Therapy (randomized trials). In: Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice, edited by Guyatt G, Rennie D, Meade MO, Cook DJ, 3rd Ed., New York, NY, McGraw-Hill Education, 2015
  7. ↵
    1. Wasserstein RL
    : ASA Statement on Statistical Significance and P-Values, Alexandria, VA Amer Statistical Assoc, 2016
  8. ↵
    1. Thorlund K,
    2. Imberger G,
    3. Walsh M,
    4. Chu R,
    5. Gluud C,
    6. Wetterslev J,
    7. Guyatt G,
    8. Devereaux PJ,
    9. Thabane L
    : The number of patients and events required to limit the risk of overestimation of intervention effects in meta-analysis–a simulation study. PLoS One 6: e25491, 2011
    OpenUrlCrossRefPubMed
  9. ↵
    1. Walsh M,
    2. Srinathan SK,
    3. McAuley DF,
    4. Mrkobrada M,
    5. Levine O,
    6. Ribic C,
    7. Molnar AO,
    8. Dattani ND,
    9. Burke A,
    10. Guyatt G,
    11. Thabane L,
    12. Walter SD,
    13. Pogue J,
    14. Devereaux PJ
    : The statistical significance of randomized controlled trial results is frequently fragile: A case for a fragility Index. J Clin Epidemiol 67: 622–628, 2014
    OpenUrlCrossRefPubMed
  10. ↵
    1. Bassler D,
    2. Briel M,
    3. Montori VM,
    4. Lane M,
    5. Glasziou P,
    6. Zhou Q,
    7. Heels-Ansdell D,
    8. Walter SD,
    9. Guyatt GH,
    10. Flynn DN,
    11. Elamin MB,
    12. Murad MH,
    13. Abu Elnour NO,
    14. Lampropulos JF,
    15. Sood A,
    16. Mullan RJ,
    17. Erwin PJ,
    18. Bankhead CR,
    19. Perera R,
    20. Ruiz Culebro C,
    21. You JJ,
    22. Mulla SM,
    23. Kaur J,
    24. Nerenberg KA,
    25. Schünemann H,
    26. Cook DJ,
    27. Lutz K,
    28. Ribic CM,
    29. Vale N,
    30. Malaga G,
    31. Akl EA,
    32. Ferreira-Gonzalez I,
    33. Alonso-Coello P,
    34. Urrutia G,
    35. Kunz R,
    36. Bucher HC,
    37. Nordmann AJ,
    38. Raatz H,
    39. da Silva SA,
    40. Tuche F,
    41. Strahm B,
    42. Djulbegovic B,
    43. Adhikari NK,
    44. Mills EJ,
    45. Gwadry-Sridhar F,
    46. Kirpalani H,
    47. Soares HP,
    48. Karanicolas PJ,
    49. Burns KE,
    50. Vandvik PO,
    51. Coto-Yglesias F,
    52. Chrispim PP,
    53. Ramsay T
    ; STOPIT-2 Study Group: Stopping randomized trials early for benefit and estimation of treatment effects: Systematic review and meta-regression analysis. JAMA 303: 1180–1187, 2010
    OpenUrlCrossRefPubMed
  11. ↵
    1. Strippoli GF,
    2. Bonifati C,
    3. Craig M,
    4. Navaneethan SD,
    5. Craig JC
    : Angiotensin converting enzyme inhibitors and angiotensin II receptor antagonists for preventing the progression of diabetic kidney disease. Cochrane Database Syst Rev (4): CD006257, 2006
PreviousNext
Back to top

In this issue

Clinical Journal of the American Society of Nephrology: 15 (6)
Clinical Journal of the American Society of Nephrology
Vol. 15, Issue 6
June 08, 2020
  • Table of Contents
  • Table of Contents (PDF)
  • About the Cover
  • Index by author
View Selected Citations (0)
Print
Download PDF
Sign up for Alerts
Email Article
Thank you for your help in sharing the high-quality science in CJASN.
Enter multiple addresses on separate lines or separate them with commas.
SONAR
(Your Name) has sent you a message from American Society of Nephrology
(Your Name) thought you would like to see the American Society of Nephrology web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
SONAR
Michael Walsh
CJASN Jun 2020, 15 (6) 889-891; DOI: 10.2215/CJN.08540719

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Request Permissions
Share
SONAR
Michael Walsh
CJASN Jun 2020, 15 (6) 889-891; DOI: 10.2215/CJN.08540719
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like

Jump to section

  • Article
    • Disclosures
    • Funding
    • Acknowledgments
    • Footnotes
    • References
  • Info & Metrics
  • View PDF

More in this TOC Section

  • Incorporating Patient Preferences via Bayesian Decision Analysis
  • Legitimization and Incorporation of Patient Preferences
  • Integrating Patient Perspectives into Medical Device Regulatory Decision Making to Advance Innovation in Kidney Disease
Show more Perspectives

Cited By...

  • No citing articles found.
  • Google Scholar

Similar Articles

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Keywords

  • atrasentan
  • randomized controlled trials
  • diabetic nephropathy
  • humans
  • albuminuria
  • diabetic nephropathies
  • endothelin A receptor antagonists
  • type 2 diabetes mellitus
  • creatinine
  • nephrology
  • incidence
  • confidence intervals
  • endothelin A receptor
  • sample size
  • reproducibility of results
  • pyrrolidines
  • chronic renal insufficiency
  • chronic kidney failure
  • kidney

Articles

  • Current Issue
  • Early Access
  • Subject Collections
  • Article Archive
  • ASN Meeting Abstracts

Information for Authors

  • Submit a Manuscript
  • Trainee of the Year
  • Author Resources
  • ASN Journal Policies
  • Reuse/Reprint Policy

About

  • CJASN
  • ASN
  • ASN Journals
  • ASN Kidney News

Journal Information

  • About CJASN
  • CJASN Email Alerts
  • CJASN Key Impact Information
  • CJASN Podcasts
  • CJASN RSS Feeds
  • Editorial Board

More Information

  • Advertise
  • ASN Podcasts
  • ASN Publications
  • Become an ASN Member
  • Feedback
  • Follow on Twitter
  • Password/Email Address Changes
  • Subscribe

© 2021 American Society of Nephrology

Print ISSN - 1555-9041 Online ISSN - 1555-905X

Powered by HighWire