| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Special Features |
* Division of Nephrology, Departments of Medicine, Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California; Renal Section, VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania; and Department of Quantitative Health Sciences, Cleveland Clinic Foundation, Cleveland, Ohio
Address correspondence to: Dr. Glenn M. Chertow, University of California San Francisco, Department of Medicine Research, UCSF Laurel Heights Suite 430, 3333 California Street, San Francisco, CA 94118. Phone: 415-476-2173; Fax: 415-476-1700; E-mail: chertowg{at}medicine.ucsf.edu
 |
Introduction |
|---|
Several known mechanisms contribute to the development of AKI, including ischemia, vasoconstriction, toxic injury related to selected endogenous substances (e.g., myoglobin), radiocontrast and drugs (e.g., amphotericin B), and microcirculatory changes, as observed with sepsis and other inflammatory states (6). Given the dire consequences that are associated with AKI, efforts at prevention seem desirable and worthy of intense investigation.
Unfortunately, most episodes of AKI cannot be predicted readily, either from clinical criteria or from the timing of events. The vast majority of prevention trials in kidney disease have been conducted in the setting of radiocontrast exposure. Several recently published studies in AKI prevention have attracted significant attention and have changed practice considerably (7–11). In this report, we scrutinize three of these studies, focusing on issues related to effect estimates and statistical power, and incorporate principles of Bayes’ Theorem in our interpretation of study results. More general, we provide a cautionary note regarding the interpretation of "positive" studies with an insufficient sample size and a "significant" (P < 0.05) result.
|
Feasibility of AKI Prevention Strategies |
|---|
Determination of a study’s sample size requires multiple inputs, including an accurate estimate of the incidence rate in the placebo- or "usual therapy"–treated group, a reasonable estimate of the intervention’s effect, and an appropriate 
error rate (i.e., the rate of falsely concluding a treatment effect when in fact a treatment effect is absent). Most important, the sample size should lead to adequate power to detect between the treatment groups a difference whose magnitude is clinically relevant and biologically plausible. In the setting of AKI, the incidence rate depends greatly on both the AKI definition used and the specific characteristics of the population studied. More specific definitions (e.g., large nominal or percentage changes in serum creatinine or other biomarkers) would yield lower incidence rates. In most published studies of prevention of radiocontrast-associated AKI, only relatively small changes in serum creatinine concentrations were required to meet the AKI definition (e.g., 25 to 50% or 
0.5 mg/dl increase in serum creatinine).
Most published studies of AKI prevention failed to document the assumptions that were used in sample size calculations or failed to document sample size estimates altogether. However, estimates of effect size in excess of 20 to 40% of the control group event rate are unrealistic on the basis not only of the complexity of AKI but also of the proven efficacy of some of the most successful interventions to change medical practice (e.g., antibiotics for serious infections, lipid lowering in secondary prevention of cardiovascular disease). Ideally, the power of intervention trials to detect a biologically plausible treatment effect should be 90% or more, particularly when the intervention carries significant risk, although power estimates as low as 80% often are chosen. When the power is <80% (i.e., the ß error exceeds 20%), the likelihood that a truly effective intervention may be deemed ineffective (i.e., a false negative) generally is considered unacceptably high. Issues of "false positive" clinical trials rarely are raised, either by investigators or by journal editors.
|
Potential Models for AKI Prevention Trials |
|---|
|
Reasonable Sample Size Estimates |
|---|
error of 0.05 and a power of 90%, the required sample size would be 572 patients; with 80% power, the required sample size would be 438 patients. A robust but slightly more realistic effect estimate (40 rather than 50%) would require 928 patients. These estimates should be considered as we review the published literature on AKI prevention trials. |
Key Publications on AKI Prevention |
|---|
Marenzi et al. (10) published another high-profile paper on the use of hemofiltration for prevention of radiocontrast nephropathy. In this study, 114 consecutive patients who had moderate to severe CKD (serum creatinine > 2.0 mg/dl) and underwent percutaneous coronary intervention were randomly assigned to hemofiltration in an ICU versus 0.9% NaCl at 1 ml/kg body wt per h in a stepdown unit for 4 to 8 h before and 18 to 24 h after their procedure. An increase in serum creatinine of 25% or more was observed in 5% of hemofiltration-treated and 50% of saline-treated patients (P < 0.001). In-hospital and 1-yr mortality rates also were significantly reduced in the hemofiltration-treated group. In this study, a sample size calculation was provided. The authors estimated that radiocontrast nephropathy would develop in 40% of control subjects and in 30% of patients who were on hemofiltration (i.e., a 25% relative and 10% absolute risk reduction). The error was 0.05, and the desired power was 80%. Whereas the authors estimated a required sample size of 50 patients per group, our estimates on the basis of the published assumptions would have yielded a study of 752 patients, or 376 patients per group (14). As with the Tepel et al. (7) paper, Marenzi et al. (10) offered no discussion on the extraordinary treatment effects observed and no comment on the possibility that the findings may not have reflected the true effect(s) of the intervention (indeed, the hemofiltration itself would be expected to lower serum creatinine concentrations).
More recently, Merten et al. (11) published the results of a study that compared the relative efficacy of a sodium bicarbonate–based versus sodium chloride–based intravenous fluid strategy for prevention of radiocontrast nephropathy in patients who underwent cardiac catheterization, computed tomography, or other procedures that required radiocontrast administration. As designed, the investigators planned to enroll a total of 260 patients to detect a 10% absolute difference in the incidence of radiocontrast nephropathy (15% in the sodium chloride–treated versus 5% in the sodium bicarbonate–treated group) with an error rate of 0.05 and power of 80%. The study was halted at approximately the midpoint by a safety monitor because of a lower rate of radiocontrast nephropathy in the sodium bicarbonate–treated group. Eight (13.6%) of 59 patients in the sodium chloride–treated group had met the definition of AKI as compared with only one (1.7%) of 60 patients in the sodium bicarbonate–treated group, for a reduction in the incidence of radiocontrast nephropathy of 11.9% (95% confidence interval 2.6 to 21.2%). Although the reported P value was 0.02, it should be noted that if one additional patient who was treated with sodium bicarbonate had sustained an increase in serum creatinine of 25%, then the study would not have reached conventional levels of statistical significance. Although it could be argued that the study was terminated prematurely, to the authors’ credit, an additional 191 consecutive patients who were treated with open-label sodium bicarbonate (deemed a "registry phase") were evaluated. Among these 191 patients, there were three documented cases of radiocontrast nephropathy, yielding an incidence of 1.6%—a rate virtually identical to that observed in the randomized trial. Detailed characteristics of the "registry" population were not provided. These three studies highlight the medical community’s ongoing fascination with the P value; because all three studies were "positive," no objections were brought forward on the grounds of insufficient power.
|
Importance of Previous Information |
|---|
The Reverend Thomas Bayes put forward these principles in 1764, in "An Essay Toward Solving a Problem in the Doctrine of Chances" (15). Bayes’ theorem suggests that the appropriate estimate of the odds that a hypothesis is true, after having observed the results of a study, depends equally on the previous odds that the hypothesis is true and the "likelihood ratio," which reflects the compatibility of the data with the hypothesis being evaluated. From this perspective, the problem with underpowered studies is that the effect sizes that are necessary to achieve adequate power are implausibly large, suggesting that the previous odds of the research hypotheses that actually are being tested are small. If the previous odds of the research hypotheses are small, then Bayes’ theorem tells us that the probability that the hypotheses are true can remain low even in the presence of trends in the data that seem to support them. Despite the widespread application of Bayes’ theorem in multiple settings in medicine, including the evaluation of diagnostic studies with imperfect performance characteristics (i.e., sensitivity and specificity), few published clinical trials have been interpreted explicitly in this context.
Now imagine a bag of 10,000 nuts. Some are almonds and some are cashews, but the exact proportion of nuts is unknown. It is unnecessary to count all of the nuts to make some statement about this proportion. For example, a randomly acquired sample of 1000 nuts may be sufficient to make an inference about the proportion of almonds and cashews in the entire population. If almonds compose 40% of the 1000-nut sample, then we may be able to infer that approximately 40% of the population of nuts also is almonds. To laypersons (or to the editorial boards of some medical journals), this process may seem straightforward. In fact, it might seem that there is no need even to acquire a sample of 1000 nuts. A sample of 100 or even 10 nuts might do. However, as the sample size becomes smaller, the potential for error grows. For this reason, inferential statistics has developed numerous techniques for stating the level of confidence that can be placed on these inferences. The confidence in the estimate of the proportion of almonds (or cashews) will be higher with sample of 1000 > 100 > 10.
It is troubling that the most prominent studies of AKI prevention, with absent or incorrect power calculations but P < 0.05, generally have been accepted, despite that each study essentially is equivalent to bags of nuts that are too small for clear conclusions. As a result, there is so much variability in the data that these studies have adequate power to detect only very large, biologically implausible effects (which have low prior probabilities of being true). It is highly unlikely that N-acetyl cysteine or hemofiltration is so effective in such a complex disease as to exert a 90% treatment benefit. If one were to consider reasonable estimates of treatment effects given previous information—for example, a 10, 20, or even 30% treatment effect—then a larger bag of nuts would be required to provide a low enough error rate to yield confident conclusions. In other words, despite the significant P value, there is a relatively high likelihood that the study results cited above were in error (i.e., false positive results). For example, other clinical trials and meta-analyses on the effects of N-acetyl cysteine for prevention of radiocontrast nephropathy have yielded conflicting results; none as positive as those reported by Tepel et al. (7).
|
True versus False Positive Results: A Simulation |
|---|
|
|
Conclusion |
|---|
|
Acknowledgments |
|---|
|
Footnotes |
|---|
|
References |
|---|
This article has been cited by other articles:
|
|
 |
|
  S. S. Waikar, K. D. Liu, and G. M. Chertow Diagnosis, Epidemiology and Outcomes of Acute Kidney Injury Clin. J. Am. Soc. Nephrol., May 1, 2008; 3(3): 844 - 861. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. M. Chertow and S. S. Waikar Toward the Promise of Renal Replacement Therapy J. Am. Soc. Nephrol., May 1, 2008; 19(5): 839 - 840. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Top
Introduction
Feasibility of AKI Prevention...