CJASN
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Published ahead of print on April 5, 2006
Clin J Am Soc Nephrol 1: 563-574, 2006
© 2006 American Society of Nephrology
doi: 10.2215/CJN.00640805
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
CJN.00640805v1
1/3/563    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Goldfarb-Rumyantzev, A. S.
Right arrow Articles by Barenbaum, L. L.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Goldfarb-Rumyantzev, A. S.
Right arrow Articles by Barenbaum, L. L.

Renal Transplantation

Role of Maintenance Immunosuppressive Regimen in Kidney Transplant Outcome

Alexander S. Goldfarb-Rumyantzev*,{dagger},{ddagger}, Lonnie Smith{ddagger}, Fuad S. Shihab*,{ddagger}, Bradley C. Baird*, Arsalan N. Habib*, Shih-jui Lin§, and Lev L. Barenbaum||

* Division of Nephrology and Hypertension, {ddagger} Kidney Transplant Program, and § Department of Medical Informatics, University of Utah School of Medicine, {dagger} Veterans Affairs Salt Lake City Healthcare System, and || RenalService.com, Inc., Salt Lake City, Utah

Address correspondence to: Dr. Alexander S. Goldfarb-Rumyantzev, Division of Nephrology and Hypertension, University of Utah Health Sciences Center, 85 North Medical Drive East, Room 201, Salt Lake City, UT 84112. Phone: 801-585-9455; Fax: 801-585-3830; E-mail: alex.goldfarb{at}hsc.utah.edu


   Abstract
Top
 Abstract
Introduction
 Materials and Methods
 Results
 Discussion
 Conclusion
 References
 
Data of long-term immunosuppressive protocol comparison are lacking. The goal of this study was to compare kidney transplant outcome using three common immunosuppressive protocols. A retrospective study was performed of the graft and recipient survival using US Renal Data System data (n = 31,012) between January 1, 1995, and December 31, 1999, with the follow-up through December 31, 2000, on prednisone + cyclosporine + mycophenolate mofetil (PCM; n = 17,108), prednisone + tacrolimus + mycophenolate mofetil (PTM; n = 7225), or prednisone + cyclosporine + azathioprine (PCA; n = 6679). Compared with PCM, there is an increased risk for allograft failure associated with PTM (hazard ratio [HR] 1.09; P < 0.05) and PCA (HR 1.15; P < 0.001). Similar associations were demonstrated in the following subgroups: Early (before 1997) and late (in or after 1997) transplant periods, in living-donor transplants, and in adult and kidney-only recipients. This association also was found between PCA regimen and graft survival in the entire patient population (HR 1.15; P < 0.001) and in the studied subgroups. PCA (HR 1.15; P < 0.005), but not PTM (HR 1.01; P = 0.816), regimen was associated with increased recipient mortality in the entire patient population and in patient subgroups. Secondary outcomes (serum creatinine values at given time points, acute rejection rate, and posttransplantation malignancies) are also discussed. These data suggest that a PCM regimen is associated with lower risk for graft failure compared with a PTM regimen and with lower risk for graft failure and recipient death compared with a PCA regimen.


   Introduction
Top
 Abstract
 Introduction
Materials and Methods
 Results
 Discussion
 Conclusion
 References
 
Kidney transplant improves survival in recipients compared with patients who have ESRD and remain on the waiting list (1). The average deceased-donor graft fails at approximately 10 yr, with chronic allograft nephropathy being the main cause of failure other than patient death (2). In the past decade, the immunosuppressive armamentarium has increased substantially; several choices are now available for immunosuppression in kidney transplant recipients. Improvement in patient and graft survival over time has correlated best with the introduction of new, more effective immunosuppressive agents. These newer immunosuppressant medications have shown equal or superior short-term (1 yr) outcomes in comparison with the established immunosuppressive medications (3–5). We have seen improvement in short-term graft survival, as a result of the reduction in the incidence of acute rejection episodes or potentially a better use of existing medications (6). Unfortunately, these improvements in immunosuppression and reduced incidence of acute rejection episodes have had only minimal effects on chronic allograft nephropathy and late graft loss (2,7).

There is continuing shift in the calcineurin inhibitors that are used, from cyclosporine to tacrolimus, and in antimetabolites that are used, from azathioprine to mycophenolate mofetil (MMF). Both tacrolimus and MMF were introduced for kidney transplantation in the mid-1990s; since then, according to Scientific Registry of Transplant Recipients, in 2003, 67% of kidney transplant recipients received tacrolimus and 81% received MMF at the time of discharge (8).

Numerous clinical trials have compared the various immunosuppressive agents. These trials historically have evaluated the short 1-yr outcomes of graft and patient survival and have contained a relatively small number of patients. The results from clinical trials that have used tacrolimus and/or MMF have varied in their outcomes.

Clinical trials have shown superior graft survival with tacrolimus when compared with cyclosporine (9), whereas others have failed to show any significant differences in either graft or patient survival (3,10). Other trials have shown improved renal function for tacrolimus-based regimens when compared with cyclosporine-containing regimens but showed no significant differences in graft or patient survival (11). Conversely, in the brief report based on the United Network for Organ Sharing (UNOS) Scientific Renal Transplant Registry, analysis by Bunnapradist and Takemoto (12) demonstrated the better 3-yr graft outcome in patients who were on a cyclosporine + MMF regimen than in those who were on a tacrolimus + MMF regimen. Similar data for clinical trials that compared MMF with azathioprine can be found. Although MMF has been shown to decrease the incidence of acute rejections when compared with azathioprine in renal transplant recipients, the 3-yr follow-up of the United States and the Tricontinental studies did not show an increase in graft survival (13,14). Ojo et al. (15) subsequently showed an improved 4-yr graft survival in patients who were treated with MMF when compared with patients who were treated with azathioprine. To date, long-term outcome studies of various immunosuppressive regimens are lacking. The goal of this project was to analyze retrospective data provided by the US Renal Data System (USRDS) and to compare the graft and recipient outcomes of kidney transplants that were managed with the three most common maintenance immunosuppressive protocols over the last 5 yr of the last century.


   Materials and Methods
Top
 Abstract
 Introduction
 Materials and Methods
Results
 Discussion
 Conclusion
 References
 
Study Design and Source of Data
The study was performed retrospectively using a national data set of kidney and kidney-pancreas transplants. The data were collected by the USRDS and UNOS of all kidney allograft recipients (both pediatric and adult) who underwent kidney or kidney-pancreas transplantation during the period of January 1, 1995, through December 31, 1999. The follow-up period was extended through December 31, 2000. The reasons that we selected only patients who received a transplant in 1995 or later were (1) major changes in clinical practice and immunosuppressive regimens took place in the mid-1990s, and (2) in our experience with USRDS data analysis, the data were deemed much more reliable and complete starting in 1995. For recipients of multiple transplants, the most recent one was considered the target transplant (transplant of interest). Patient records with missing information regarding graft or patient survival were excluded from the study. Only patients who were treated with maintenance immunosuppression with one of three regimens were selected for the study: Prednisone + cyclosporine + MMF (PCM), prednisone + tacrolimus + MMF (PTM), and prednisone + cyclosporine + azathioprine (PCA). Patients who did not have the information regarding their maintenance immunosuppressive therapy were excluded from the study; therefore, the total number of patients who were included in the analysis was 31,012. The records that were used in this study were extracted from the data set that was previously analyzed by our group using approaches to data cleaning and validation described elsewhere (16–18).

Outcome Measures and Definitions
Primary Outcomes.  There were two primary outcomes in this study: (1) Graft survival, measured as the time between the most recent kidney transplant and the failure of the graft, and (2) recipient survival, measured as the time between the most recent kidney transplant and the patient’s death. Graft outcome was censored at the earliest of the following events and was analyzed as days to graft failure or censor: Loss to follow-up, patient death, or the study completion date (December 31, 2000).

We used death-censored definition of graft failure, i.e., graft failure definition did not include patient death with a functioning graft, the latter determined in the USRDS as a single binary variable. In case the value of this variable was missing and the patient’s death date was found to be equal to the graft failure date, we assumed that the patient died with a functioning graft, unless the cause of death was coded by the International Classification of Diseases, Ninth Revision as one of the following: 3200 (graft failure: primary failure), 3201 (graft failure: rejection), 3202 (graft failure: technical), 3299 (graft failure: other), or 3903 (miscellaneous: renal failure). Patient follow-up was censored at the earliest of loss to follow-up or study completion date and was analyzed as days to recipient death or censor.

Secondary Outcomes.  In addition, several secondary outcomes were compared between the study groups: Average creatinine value at 6 mo and 1, 3, 5, and 7 yr; the rate of acute rejection episodes; and the rate of posttransplantation malignancies. We used the variable MALIG from the TXFUUNOS file as a source of posttransplantation malignancy information.

Independent Variables
Primary Variable of Interest.  The primary variable of interest was the type of maintenance immunosuppressive regimen at the time of discharge from the hospital. Three regimens that were considered the most common maintenance protocols in the 1990s were selected for this analysis as indicated above: PCM, PTM, and PCA.

Covariates.  Cox models were adjusted for the covariates that were believed to present a potential source of confounding (i.e., potentially affecting the primary variables of interest and the outcome). The final decision of which covariates to include into the final models was based primarily on the known associations between variables that could cause confounding of the primary variables of interest; therefore, even variables that had no significant association with the outcome and were not selected by the stepwise regression but were deemed to be clinically significant or represent potential confounding were included in the final model. Results of the stepwise analysis were used only as a supportive tool.

To adjust for patients’ comorbidities, we calculated a comorbidity score similar to the one proposed by Davies et al. (19) but limited to comorbid conditions that were available in the data set and listed below and collected at the time of transplantation. The Davies score has been shown to be strongly associated with outcome (survival) in a prospective study of 97 peritoneal dialysis patients. The comorbidity score that was used in this study was calculated on the basis of the following coexisting conditions, each of them contributing one point to the score: Cardiovascular disease (defined in USRDS as symptomatic cardiovascular disease or angina/coronary artery disease), symptomatic peripheral vascular disease, diabetes, and hypertension. Information about coexisting conditions was obtained from the Transplant UNOS (TXUNOS) file, and comorbidity information was collected from the Transplant Candidate Registration Form. Therefore, the comorbidities that were used for this study are those that the patient had at the time of listing for the most recent transplant.

The covariates included a recipient comorbidity score, other recipient variables (recipient age, gender, race, height, weight; primary cause of ESRD; history of hypertension; diabetes; previous transplant; total duration of ESRD; total number of transplants; peak and mean panel reactive antibody [PRA] level; number of matched HLA antigens; number of pretransplantation transfusions; and fractions of the ESRD course spent on peritoneal dialysis and hemodialysis), donor variables (type of donor [deceased or living] and donor age, gender, race, height, weight), and cold ischemia time.

Unrealistic values of the independent variables that were used in the study were eliminated. In particular, for donors and recipients who were younger than 13 yr, the Centers for Disease Control and Prevention growth charts were used as a guide for determining valid ranges. The heights and weights of recipients and donors who were 13 yr and older were based on the acceptable ranges: height, 120 to 275 cm; and weight, 23 to 180 kg.

Statistical Analyses
Survival Analysis.  Kaplan-Meier survival curves were used to display hypothesized relationships, and Cox regression models were used to analyze time to event. Data were analyzed using SAS, version 9 (SAS Institute, Cary, NC). To reduce potential bias in the analysis, in addition to analyzing the whole patient population, we stratified Cox models by the transplant era (because different protocols were used mostly in different eras, with PCA being the earliest one and PTM being the latest). The time periods selected for this study were (1) early period (all kidney and kidney-pancreas transplants that were performed between January 1, 1995, and December 31, 1996) and (2) late period (transplants that were performed between January 1, 1997, and December 31, 1999). In addition, survival analysis was stratified by the type of kidney donor (living versus deceased), because living donors might have been considered to be at lower risk and therefore affect the choice of immunosuppression.

Secondary Outcome Analysis.  Study groups were compared with respect to secondary outcomes using ANOVA (creatinine concentration at different time points) and {chi}2 (acute rejection episodes and posttransplantation malignancies). In addition, secondary outcomes were evaluated in multivariate analysis using linear (creatinine values) or logistic (acute rejection episodes, posttransplantation malignancies) regression models. The models were adjusted for all of the covariates listed above and were evaluated for the role of immunosuppressive regimen (primary variable of interest) with respect to the secondary outcomes.


   Results
Top
 Abstract
 Introduction
 Materials and Methods
 Results
Discussion
 Conclusion
 References
 
Baseline Characteristics
Baseline characteristics of the patient population (n = 31,012) are presented in Table 1. Among recipients, the average age was 44.2 ± 14.3 yr, 39.6% were female, 68.7% were white, and 23.3% were black. A total of 55.2% (n = 17,108) of patients were receiving maintenance PCM, 23.3% (n = 7225) were receiving PTM, and 21.5% (n = 6679) were receiving PCA. Because different immunosuppressive drugs were introduced at different time periods, we evaluated the prevalence of different immunosuppressive protocols in transplants that were performed in two different time periods during the study: Before 1997 and during 1997 and later. Before 1997, 38.1% of the transplant recipients were placed on PCA maintenance at the time of discharge, 50.1% were placed on PCM, and 11.9% were placed on PTM. During and after 1997, 16.3% were placed on PCA maintenance, 56.8% were placed on PCM, and 26.9% were placed on PTM. We compared baseline characteristics of the patients in the three study groups using ANOVA for continuous variables and {chi}2 for the categorical variables (Table 2).


View this table:
[in this window]
[in a new window]

  		Table 1. Descriptive statistics of the study populationa

 

View this table:
[in this window]
[in a new window]

  		Table 2. Baseline characteristics of patients on three different maintenance immunosuppressive regimensa

 
Survival Analysis
Using the PCM group as a reference, the Cox model demonstrated the increased risk for allograft failure associated with the PTM (hazard ratio [HR] 1.08; P < 0.05) and the PCA (HR 1.14; P < 0.001) regimens (Table 3). The PCA (HR 1.15;P < 0.005) but not the PTM (HR 0.99; P = 0.9) regimen was associated with increased recipient mortality (Table 3). This association is illustrated by Kaplan-Meier survival curves for donor and recipient survival (Figures 1 and 2).


View this table:
[in this window]
[in a new window]

  		Table 3. Cox model: Graft and recipient survivala

 

Figure 1
View larger version (12K):
[in this window]
[in a new window]
[as a PowerPoint slide]
  		Figure 1. Kaplan-Meier curves illustrating the effect of the immunosuppressive protocol on the renal graft survival (A) and recipient survival (B). There is better graft survival in patients who were on the prednisone + cyclosporine + mycophenolate mofetil (PCM) regimen (P < 0.001 by log-rank). There is no difference in recipient survival on different immunosuppressive regimens (P = 0.8 by log-rank).

 

Figure 2
View larger version (41K):
[in this window]
[in a new window]
[as a PowerPoint slide]
  		Figure 2. Illustration of the results of Cox proportional hazard model. Hazard ratio (HR) of the graft failure and recipient death in the whole group of patients and in the subgroups based on the transplantation period (early versus late) and donor type (deceased versus living). Patients who were on the prednisone + cyclosporine + mycophenolate mofetil regimen were used as a comparison/reference group (HR 1.0; *P <0.05).

 
Survival Analysis in the Subgroups
Transplant Era.  Because clinical practice evolved during the follow-up period of our study, we analyzed the outcomes of the transplants that were performed during two time periods as described above: (1) From January 1, 1995, and through December 31, 1996 (early period), and (2) from January 1, 1997, through December 31, 1999 (late period). Among patients who received a transplant in the early period (n = 7448), 11.7% were on PTM, 38.1% were on PCA, and 50.2% were on PCM. Among those who received a transplant in the late period (n = 23,564) 26.9% were on PTM, 16.3% were on PCA, and 56.8% were on PCM.

Among the transplants that were performed in the early period, using PCM as a reference group, PTM was associated with the significantly greater risk for graft failure (HR 1.16; 95% confidence interval [CI] 1.01 to 1.33; P < 0.05). PCA was associated with nonsignificantly increased risk for graft failure (HR 1.06; 95% CI 0.96 to 1.17; P = 0.244). In the late period, both PTM (HR 1.10; 95% CI 1.01 to 1.19; P < 0.05) and PCA (HR 1.14; 95% CI 1.04 to 1.25; P < 0.01) regimens were associated with significantly greater risk for graft failure. Recipient survival had no significant association with either PTM or PCA regimens in the early or late periods. For the early period, HR was 1.04 (P = 0.693) and 1.10 (P = 0.15) for the PTM and PCA regimens, respectively. In the late period, HR was 1.003 (P = 0.96) and 1.14 (P = 0.057) for the PTM and PCA regimens, respectively (Figure 2).

Year of Transplantation.  To evaluate the outcome in patient cohorts with the same or similar duration of follow-up, we stratified the analysis by the year of transplantation, evaluating the patients who received a transplant in 1995 (n = 997), 1996 (n = 6451), 1997 (n = 8160), 1998 (n = 8379), and 1999 (n = 7025) separately. Patients who received a transplant in 1995 would have had 5 to 6 yr of follow-up during the study, those who received a transplant in 1996 would have had 4 to 5 yr of follow-up, those who received a transplant in 1997 would have had 3 to 4 yr of follow-up, etc. For the graft survival, using PCM as a reference group, the results were as follows: In the recipients who received a transplant in 1995, the PTM regimen was associated with an increased risk for graft failure (HR 1.49; P < 0.05), whereas the PCA regimen did not have any significant association. In the recipients who received a transplant in 1996, no significant associations between the graft survival and maintenance immunosuppressive regimen were found. For the recipients who received a transplant in 1997 and 1998, only PCA was associated with shorter graft survival (HR 1.14 and 1.25, respectively; P < 0.05). Finally, recipients of a kidney transplant in 1999 had an increased risk for graft failure associated with PTM (HR 1.17; P < 0.05), whereas PCA did not have any significant association with graft survival. We did not find any significant association between the maintenance immunosuppressive regimen and the recipient survival in the cohorts of patients who were stratified by the transplant year.

Living versus Deceased Donor.  Separate analysis was performed for the recipients of living- and deceased-donor transplants. Among the recipients of a living donor kidney (n = 8924), 19.6% were on PTM, 23.4% were on PCA, and the remaining 57% were on PCM. Among the recipients of a deceased-donor kidney (n = 22,088), 24.8% were on PTM, 20.8% were on PCA, and the remaining 54.4% were on PCM.

In the recipients of the deceased-donor graft, using PCM as a reference group, PTM had no significant association with the allograft outcome (HR 1.04; P = 0.284), whereas the PCA regimen was associated with an 18% increased risk (HR 1.18; 95% CI 1.1 to 1.27; P < 0.001). In the recipients of a living-donor kidney, PTM (HR 1.22; 95% CI 1.06 to 1.41; P < 0.01) but not PCA (HR 1.05; P = 0.51) was associated with a higher risk for graft failure as compared with PCM (Figure 2). Furthermore, because the analysis is somewhat suggestive of a potential superiority of PCA over PTM in the recipients of living-donor kidneys, we repeated the Cox analysis in living donors, with PTM group as a reference. That provides direct comparison between PCA and PTM regimens. Although, as expected, the PCM regimen had an advantage over the PTM regimen (HR 0.820; 95% CI 0.71 to 0.95; P < 0.01), the PCA regimen was not significantly different from the PTM regimen. Therefore, we concluded that the PCM but not the PCA maintenance regimen is superior to the PTM regimen in living-donor recipients.

In recipients of a deceased-donor kidneys, recipient survival had no significant association with the PTM regimen but was significantly associated with the PCA regimen (HR 1.17; 95% CI 1.05 to 1.29; P < 0.001). In recipients of a living-donor kidney, recipient survival had no significant association with either one of the maintenance immunosuppressive regimens (Figure 2).

Adult versus Pediatric Recipients.  Separate analysis was performed for the recipients who were younger than 18 yr (pediatric) and those who were 18 yr and older (adult). In pediatric patients (n = 1227), 18.3% were on PTM, 35.0% were on PCA, and the remaining 11.7% were on PCM. In adult recipients (n = 29,785) 23.5% were on PTM, 21.0% were on PCA, and 55.5% were on PCM. Among pediatric patients, we did not find any significant association between the drug regimen and the graft or recipient outcomes. In adults, the results were similar to those in the entire study population: PTM was associated with increased risk for graft failure (HR 1.08; P < 0.05) but not recipient survival, whereas PCA was associated with increased risk for both graft failure (HR 1.14; P < 0.001) and recipient death (HR 1.14; P < 0.01).

Kidney-Only versus Simultaneous Kidney-Pancreas Transplant.  We performed separate analyses of the recipients of kidney-only transplants and kidney-pancreas transplants (SPK). The information about SPK was missing in 542 patients; therefore, 28,404 patients with kidney-only transplants and 2066 patients with SPK were included in the analysis. In patients with kidney-only transplants, 20.9% were on PTM, 22.5% were on PCA, and the remaining 56.6% were on PCM. In SPK recipients, 55% were on PTM, 7.6% were on PCA, and 37.4% were on PCM. In SPK recipients, we did not detect any significant association between the drug regimen and either graft or patient survival. In the recipients of kidney-only transplants, results were similar to those in the entire patient population: In comparison with the PCM regimen, PTM was associated with increased risk for graft failure (HR 1.10; P < 0.01) but not recipient death, whereas PCA was associated with both increased risk for graft loss (HR 1.15; P < 0.001) and recipient death (HR 1.16; P < 0.005).

Effect of Induction Therapy.  Because the results of the survival analysis potentially could be confounded by induction therapy, we collected the data describing the induction regimen divided into the following groups: Muromonab-CD3 (OKT3), antithymocyte globulin (ATG), IL-2 receptor mAB (IL-2R mAb; daclizumab or basiliximab), and other or missing. The last category included patients with no induction, those in whom induction therapy information was missing, and those in whom regimens other than OKT3, ATG, or IL-2R mAb induction were used. Of the total patient population, ATG was used for induction in 5152 patients, OKT3 was used in 3904 patients, IL-2R mAb was used in 3337 patients, and the rest of the recipients (n = 18,619) either were on other induction therapy or had information missing. We included the information regarding the induction therapy into the Cox model and reanalyzed the data. The results were similar to those reported above. Using the PCM group as a reference, graft survival in the recipients who were on PTM (HR 1.07; P < 0.05) PCA (HR 1.15; P < 0.001) were associated with increased risk for graft failure. For recipient survival, only PCA regimen was associated with significant risk (HR 1.14; P < 0.01).

Secondary Outcomes
Serum Creatinine Levels in the Study Groups.  We compared average creatinine values at different periods in the three study groups. Because of the missing values of creatinine, we used only records with nonmissing information: 27,609 patient records were used for 6-mo, 26,748 were used for 1-yr, 21,243 were used for 3-yr, 7687 were used for 5-yr, and 218 were used for 7-yr analyses. Creatinine value in the PTM group seems to be consistently lower than in PCA or PCM groups, except for the 7-yr follow-up (Figure 3). We compared creatinine values among the three study groups using ANOVA and demonstrated significant differences (P < 0.001 for 6-mo, 1-yr, and 3-yr follow-ups; P < 0.01 for 5-yr follow-up), except for the 7-yr follow-up, for which the difference is NS. These associations were confirmed further by the linear regression model adjusted for the covariates listed in Materials and Methods.


Figure 3
View larger version (53K):
[in this window]
[in a new window]
[as a PowerPoint slide]
  		Figure 3. Results of mean serum creatinine concentration in the study groups at 6 mo, 1 yr, 3 yr, 5 yr, and 7 yr posttransplantation. Differences between the groups are significant by ANOVA for 6 mo, 1 yr, and 3 yr (P < 0.001) and for 5 yr of follow-up (P < 0.01). Differences are NS for 7-yr of follow-up.

 
Posttransplantation Malignancies in the Study Groups.  We identified 29,050 patients who had information on the presence or absence of the posttransplant malignancies: 4.9% of the patients who were on PCA, 3.8% of the patients who were on PCM, and 2.8% of the patients who were on PTM received a diagnosis of having posttransplant malignancies (P < 0.001 by ANOVA). Logistic regression confirmed that trend: Using the PCM group as a reference, PCA was associated with greater rate of posttransplantation malignancies (odds ratio [OR] 1.26; 95% CI 1.09 to 1.46; P < 0.005), whereas PTM was associated with reduced risk for posttransplant malignancies (OR 0.78; 95% CI 0.66 to 0.93; P < 0.01).

Because results potentially could be confounded by the induction therapy, we analyzed an additional model adjusted for the induction therapy as described above. The PTM regimen was associated with lower risk for posttransplantation malignancies as compared with PCM (OR 0.80; 95% CI 0.671 to 0.95; P < 0.05), whereas the PCA regimen demonstrated a greater risk (OR 1.24; 95% CI 1.07 to 1.44; P < 0.005). Finally, we analyzed the smaller data set limited only to patients who were on OKT3, ATG, or IL-2R mAb (n = 12,393). The direction of the association between the maintenance immunosuppressive regimen and posttransplantation malignancies was the same, although it lost statistical significance. It is interesting that in this small data set as compared with the IL-2R mAb group, both OKT3 (OR 1.60; 95% CI 1.21 to 2.12; P < 0.001) and ATG (OR 1.47; 95% CI 1.12 to 1.94; P < 0.01) had significant association with posttransplantation malignancies.

Acute Rejection Episodes.  We performed analysis to compare the rate of acute rejection in the study groups. Only 3595 patients had information regarding the acute rejection in the posttransplantation period. We compared the number of rejection episodes in the groups using ANOVA, as well as logistic regression adjusted for all of the covariates listed in Materials and Methods. Neither approach demonstrated a significant difference in the acute rejection rate among the groups.


   Discussion
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
Conclusion
 References
 
Multiple factors have been shown to affect the outcome of renal transplantation. These include demographic characteristics (20) such as race and ethnicity (21), pretransplantation dialysis course (17), and the timing of the transplantation (16). Patient response to the transplantation procedure (e.g., delayed graft function, acute rejection, acute tubular necrosis) is strongly associated with the long-term prognosis (22). The selection of the appropriate immunosuppressive regimen (including induction and maintenance) is without doubt one of the most important modifiable factors that might affect the short-term events, as well as long-term results of the transplantation.

Determining the optimal maintenance immunosuppressive regimen in kidney transplantation is an area of continued research. The immunosuppressive regimen that provides the best long-term outcome has yet to be defined. There are multiple factors to consider when choosing a maintenance immunosuppressive regimen, including the adverse-effect profile, cost, potency, and effect on allograft function (23). Because of the short-term outcomes and the low power of trials for new immunosuppressants, there is little information about the long-term graft outcomes that are associated with the various maintenance immunosuppressant regimens (24).

The short-term outcome of the kidney transplant, mostly expressed as an incidence of acute rejection, has been shown to be better with tacrolimus than with cyclosporine in both pediatric (25) and adult (26,27) kidney transplant patients as well as in heart transplant recipients (28). In lung transplant recipients, the episodes of acute rejections were similar in patients who received cyclosporine and those who received tacrolimus in combination with steroids and MMF (29). The data regarding long-term outcome are controversial. In the recipients of living-donor kidney transplants, cyclosporine + MMF had better long-term outcome in terms of all-cause graft failure and death-censored graft failure than tacrolimus + MMF (12). However, other authors reported better 3-yr outcome with tacrolimus-based regimens as compared with cyclosporine-based regimens (30,31) or no significant difference between the protocols (32,33). In a recent study that was based on 2-yr follow-up and compared tacrolimus- and cyclosporine-based regimens, the authors demonstrated no difference in the graft loss between the regimens; however, renal function was better in patients who received tacrolimus (26). Using the annualized change in GFR as an outcome, the superiority of the tacrolimus + MMF regimen in preserving renal function has been demonstrated in a study by Gill et al. (24). This study, however, was criticized for potential biases, including the nonuniform measurements of renal function between immunosuppressive regimens and between transplant centers (34). Despite the lack of long-term outcome data, the trend in recent years has been toward a shift from cyclosporine-based to tacrolimus-based and from azathioprine to MMF regimens (35).

We attempted the analysis of the kidney transplants that were performed during the last 5 yr of the last century to compare the long-term outcomes of the patients who received the three most frequently used drug protocols. Because the data are available, we considered extending the study period into the earlier years of 1990s but decided against it. Our study period covers the use of all three immunosuppressive regimens of interest, whereas comparing early 1990s (when azathioprine and cyclosporine mostly were used) with the later years (when tacrolimus and MMF started to dominate) would not be a fair comparison, because major changes in practice took place during the 1990s. Stratifying analysis by the transplantation era, as well as analyzing cohorts of patients who received a transplant in the same year to some extent should eliminate the confounding factor of time and the related issue of evolving clinical practice on the outcome.

This study demonstrates that the use of PTM and PCA as maintenance immunosuppression between 1995 and 1999 was associated with increased risk for graft failure by 9% (P < 0.05) and 15% (P < 0.001), respectively, as compared with the PCM protocol. Using the PCA regimen was associated with worsening of recipient survival by 15% (P < 0.005) as compared with PCM. Similar results were reported by other investigators who analyzed long-term outcome on the large kidney transplant data sets (12), in which, using data from the UNOS Scientific Renal Transplant Registry, authors demonstrated that the death-censored graft failure is 25% higher in the tacrolimus + MMF group as compared with cyclosporine + MMF. Woodward et al. at the recent American Transplant Congress presented results of the analysis in which graft survival in patients who were on the PCM regimen was superior to those who were on the PTM regimen in living-donor (36) and deceased-donor (37) kidney transplant recipients. In the analysis of the secondary outcomes, the results seemingly are pointing in another direction. Serum creatinine concentration values were consistently lower in the PTM group, except for the 7-yr follow-up time point. Results of this analysis should be interpreted cautiously. Serum creatinine has been used in numerous studies as a surrogate outcome, when the longer follow-up period to observe graft failure is not feasible. In this project, however, we believe that we have long enough follow-up to observe adequately the "hard" outcomes (graft failure and recipient death) to make a conclusion. In a way, a patient’s serum creatinine value at a given point in time probably is less important, as long as his or her graft is surviving longer. In addition, while interpreting these results, one must realize one very important potential flaw of this type of analysis. Comparing serum creatinine values among the groups at a given time point may be misleading because only patients who survived to that time point are included in this analysis. Therefore, if the PTM group were to have a higher graft failure rate, then creatinine might be artificially lower in the surviving patients as compared with the PCM group, in which fewer patients failed the transplant, and therefore average creatinine values might be artificially higher (survivor bias). For example, it is conceivable that if two cohorts of patients are compared and in the first one the graft failure rate is greater than in the second one, then the average creatinine also might be lower in the first cohort, because only the "healthiest" patients remain in the study. In the second cohort, in which the graft failure rate is slower, more patients with dysfunctional graft that has not failed yet would remain in the study, and therefore the average creatinine might be higher. This still would mean the better outcome (lower rate of graft failure) in the second cohort despite higher average creatinine concentration.

Another interesting finding of this study is a significantly lower rate of posttransplantation malignancies in the PTM compared with PCM and PCA groups. This phenomenon, however, did not translate into improved patient survival in the PTM group as compared with the PCM group. Contrary to our results, in a recent meta-analysis, no difference was demonstrated between tacrolimus-based and cyclosporine-based regimens (38). It has been suggested by other authors that induction therapy might be equally or even more important than the maintenance immunosuppression in development of posttransplantation malignancies (39). Adjusting our model for the induction therapy demonstrated results similar to those that we detected in the model that was not adjusted for induction, suggesting lower risk for posttransplantation malignancies with PTM regimen.

This study is a retrospective analysis that used data that were reported to the USRDS. There are limitations as well advantages to large renal transplant database analyses. Database analyses can show long-term differences in outcomes, but the results must be evaluated with caution. Using a database such as the USRDS provides the statistical power to help to determine the differences between current maintenance immunosuppressive regimens. However, because the database does not contain information about the dosage or the duration of therapy, caution must be taken when making conclusions from the data. Certain limitations should be considered when interpreting the results of this study. Selection bias is a common flaw of a retrospective analysis. On the basis of our results, the outcome of the cyclosporine-based regimen is superior to that of the tacrolimus-based protocol. We recognize that during the early use of tacrolimus, it was a tendency to use it mostly in the higher risk population, e.g., those with higher PRA levels, retransplants, black race, hence the selection bias. Indeed, we compared subgroups of patients who were on three immunosuppressive regimens of interest and found that there is a very small, although statistically significant, difference in the baseline characteristics among the study groups (Table 2). To reduce this potential bias, we adjusted the Cox model for the risk factors of premature graft failure: Recipient race, PRA levels, number of previous blood transfusions, number of previous transplants, and comorbidity index. Including these potential confounding factors in the multivariate model should reduce the bias considerably. In addition, we tried to reduce the selection bias by stratifying analysis by the transplantation era, as tacrolimus became more of a "mainstream" medication (as opposed to being used in high-risk patients only) in the later 1990s. Also, because living donors might have been considered at lower risk and therefore affect the choice of immunosuppression, we stratified the analysis by donor type. Finally, we analyzed pediatric and adult recipients and kidney-only and SPK recipients separately. The negative association between the PTM regimen and graft survival was observed in the entire patient population. Importantly, the observed negative association between the PTM regimen and graft outcome was observed not only in the early period but also in the late period, when the use of tacrolimus supposedly was not limited only to the high-risk patient population. The negative association between the PCA regimen and graft survival was observed in the entire patient population and only in late but not in early transplantation periods and only in deceased- but not in living-donor transplants (Figure 2). The described associations also were revealed in adult recipients and kidney-only recipients (but not in pediatric and SPK recipients). Specifically, there is no superiority of PTM as compared with PCM in SPK recipients. Subgroup analysis of the recipient survival did not demonstrate any association between PTM regimen and the recipient survival either in the whole patient population or in any of the subgroups. As kidney transplant recipient survival is relatively long, it is conceivable that a longer follow-up period is needed to observe enough events to demonstrate a difference between the PTM and PCM regimens. The negative association between the PCA regimen and recipient survival was demonstrated in the whole patient population, but in the subgroup analysis, only patients who received a deceased-donor kidney, adult recipients, and those who received a kidney-only (as opposed to SPK) transplant had increased risk for death on this regimen.

Unfortunately, any study that evaluates the long-term outcome has to deal with the fact that the practice evolves during the time of the study. We partially addressed this issue by performing analysis separately for early and late transplantation eras. However, the clinical practice even during the late era of the study (late 1990s) is different compared with the current practice in the early 2000s. In particular, the proportion of various immunosuppressive regimens during the study is different from that in the modern days. We tried to address that by performing additional analyses that compared centers that use a particular regimen (PCM, PCA, or PTM) exclusively or predominantly. However, the amount of missing data related to the immunosuppressive regimen evaluated separately by the transplant center prevented us from performing this analysis. We did, in fact, identify the centers that lean toward one regimen or another; however, in every case, we encountered a high amount of missing information that made the classification of the center into predominantly PCM, PCA, or PTM potentially erroneous. Therefore, the results of this analysis potentially might be misleading and are not presented here. Our study was performed in a retrospective manner and unfortunately does not provide the explanations for the mechanism of the observed associations. Future studies are needed to confirm this association and establish the mechanism of it. In general, the difference in graft survival between two regimens may be explained by the difference in nephrotoxicity, associated comorbidity, different adverse-effect profile, or center effect. Furthermore, some of the patients in the study had a relatively short follow-up because of censoring. Our study period included all transplants that were done since 1995, with the duration of follow-up through the end of 2000. Of all patients who were included in the study, 997 received a transplant in 1995 and therefore had 5 to 6 yr of follow-up, 6451 received a transplant in 1996 and had 4 to 5 yr of follow-up, and 8160 received a transplant in 1997 and had 3 to 4 yr of the follow-up; patients who received a transplant in 1998 (n = 8379) and 1999 (n = 7025) had relatively short follow-up (1 to 3 yr).


   Conclusion
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 Conclusion
References
 
Our data suggest that the PCM regimen is associated with lower risk for graft failure compared with PTM and lower risk for graft failure and recipient death compared with PCA.


   Acknowledgments
 
This study was funded in part by the Dialysis Research Foundation (Ogden, UT). The data reported here were originally supplied by the USRDS.


   Footnotes
 
Published online ahead of print. Publication date available at www.cjasn.org.

The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as official policy or interpretation of the US government.

Received August 3, 2005. Accepted February 13, 2006.


   References
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 Conclusion
 References
 

  1. Ojo AO, Hanson JA, Meier-Kriesche HU, Okechukwu CN, Wolfe RA, Leichtman AB, Agodoa LY, Kaplan B, Port FK: Survival in recipients of marginal cadaveric donor kidneys compared with other recipients and wait-listed transplant candidates. J Am Soc Nephrol12 :589 –597,2001[Abstract/Free Full Text]
  2. Halloran PF, Melk A, Barth C: Rethinking chronic allograft nephropathy: The concept of accelerated senescence. J Am Soc Nephrol10 :167 –181,1999[Free Full Text]
  3. Pirsch JD, Miller J, Deierhoi MH, Vincenti F, Filo RS: A comparison of tacrolimus (FK506) and cyclosporine for immunosuppression after cadaveric renal transplantation. FK506 Kidney Transplant Study Group. Transplantation63 :977 –983,1997[CrossRef][Medline]
  4. Sollinger HW: Mycophenolate mofetil for the prevention of acute rejection in primary cadaveric renal allograft recipients. US Renal Transplant Mycophenolate Mofetil Study Group. Transplantation60 :225 –232,1995[Medline]
  5. Shapiro R, Jordan ML, Scantlebury VP, Vivas C, Marsh JW, McCauley J, Johnston J, Randhawa P, Irish W, Gritsch HA, Naraghi R, Hakala TR, Fung JJ, Starzl TE: A prospective, randomized trial of tacrolimus/prednisone versus tacrolimus/prednisone/mycophenolate mofetil in renal transplant recipients. Transplantation67 :411 –415,1999[Medline]
  6. Hariharan S, Johnson CP, Bresnahan BA, Taranto SE, McIntosh MJ, Stablein D: Improved graft survival after renal transplantation in the United States, 1988 to 1996. N Engl J Med342 :605 –612,2000[Abstract/Free Full Text]
  7. Pascual M, Theruvath T, Kawai T, Tolkoff-Rubin N, Cosimi B: Strategies to improve long-term outcomes after renal transplantation. N Engl J Med346 :580 –590,2002[Free Full Text]
  8. Scientific Registry of Transplant Recipients, 2004 OPTN/SRTR Annual Report. Available: http://www.ustransplant.org. Accessed May 20,2005
  9. Gjertson DW, Cecka JM, Terasaki PI: The relative effects of FK506 and cyclosporine on short- and long-term kidney graft survival. Transplantation60 :1384 –1388,1995[Medline]
  10. Johnson C, Ahsan N, Gonwa T, Halloran P, Stegall M, Hardy M, Metzger R, Shield C 3rd, Rocher L, Scandling J, Sorensen J, Mulloy L, Light J, Corwin C, Danovitch G, Wachs M, van Veldhuisen P, Salm K, Tolzman D, Fitzsimmons WE: Randomized trial of tacrolimus (Prograf) in combination with azathioprine or mycophenolate mofetil versus cyclosporine (Neoral) with mycophenolate mofetil after cadaveric kidney transplantation. Transplantation69 :834 –841,2000[CrossRef][Medline]
  11. Ahsan N, Johnson C, Gonwa T, Halloran P, Stegall M, Hardy M, Metzger R, Shield C 3rd, Rocher L, Scandling J, Sorensen J, Mulloy L, Light J, Corwin C, Danovitch G, Wachs M, VanVeldhuisen P, Salm K, Tolzman D, Fitzsimmons WE: Randomized trial of tacrolimus plus mycophenolate mofetil or azathioprine versus cyclosporine oral solution (modified) plus mycophenolate mofetil after cadaveric kidney transplantation: Results at 2 years. Transplantation72 :245 –250,2001[CrossRef][Medline]
  12. Bunnapradist S, Takemoto SK: Controlling treatment allocation bias in a registry analysis when comparing calcineurin inhibitors. Transplant Proc35 :2407 –2408,2003[Medline]
  13. Mycophenolate mofetil in cadaveric renal transplantation. US Renal Transplant Mycophenolate Mofetil Study Group. Am J Kidney Dis34 :296 –303,1999[Medline]
  14. Mathew TH: A blinded, long-term, randomized multicenter study of mycophenolate mofetil in cadaveric renal transplantation: Results at three years. Tricontinental Mycophenolate Mofetil Renal Transplantation Study Group. Transplantation65 :1450 –1454,1998[Medline]
  15. Ojo AO, Meier-Kriesche HU, Hanson JA, Leichtman AB, Cibrik D, Magee JC, Wolfe RA, Agodoa LY, Kaplan B: Mycophenolate mofetil reduces late renal allograft loss independent of acute rejection. Transplantation69 :2405 –2409,2000[CrossRef][Medline]
  16. Goldfarb-Rumyantzev A, Hurdle JF, Scandling J, Wang Z, Baird B, Barenbaum L, Cheung AK: Duration of end-stage renal disease and kidney transplant outcome. Nephrol Dial Transplant20 :167 –175,2005[Abstract/Free Full Text]
  17. Goldfarb-Rumyantzev AS, Hurdle JF, Scandling JD, Baird BC, Cheung AK: The role of pretransplantation renal replacement therapy modality in kidney allograft and recipient survival. Am J Kidney Dis46 :537 –549,2005[Medline]
  18. Lin SJ, Koford JK, Baird BC, Hurdle JF, Krikov S, Habib AN, Goldfarb-Rumyantzev AS: Effect of donors’ intravenous drug use, cigarette smoking, and alcohol dependence on kidney transplant outcome. Transplantation80 :482 –486,2005[CrossRef][Medline]
  19. Davies SJ, Russell L, Bryan J, Phillips L, Russell GI: Comorbidity, urea kinetics, and appetite in continuous ambulatory peritoneal dialysis patients: Their interrelationship and prediction of survival. Am J Kidney Dis26 :353 –361,1995[Medline]
  20. Matas AJ, Gillingham KJ, Humar A, Dunn DL, Sutherland DE, Najarian JS: Immunologic and nonimmunologic factors: Different risks for cadaver and living donor transplantation. Transplantation69 :54 –58,2000[CrossRef][Medline]
  21. Isaacs RB, Nock SL, Spencer CE, Connors AF Jr, Wang XQ, Sawyer R, Lobo PI: Racial disparities in renal transplant outcomes. Am J Kidney Dis34 :706 –712,1999[Medline]
  22. Gjertson DW: Multifactorial analysis of renal transplants reported to the United Network for Organ Sharing Registry: A 1994 update. Clin Transpl519 –539,1994
  23. Danovitch GM: Immunosuppressive medications for renal transplantation: A multiple choice question. Kidney Int59 :388 –402,2001[CrossRef][Medline]
  24. Gill JS, Tonelli M, Mix CH, Johnson N, Pereira BJ: The effect of maintenance immunosuppression medication on the change in kidney allograft function. Kidney Int65 :692 –699,2004[CrossRef][Medline]
  25. Trompeter R, Filler G, Webb NJ, Watson AR, Milford DV, Tyden G, Grenda R, Janda J, Hughes D, Ehrich JH, Klare B, Zacchello G, Bjorn Brekke I, McGraw M, Perner F, Ghio L, Balzar E, Friman S, Gusmano R, Stolpe J: Randomized trial of tacrolimus versus cyclosporin microemulsion in renal transplantation. Pediatr Nephrol17 :141 –149,2002[CrossRef][Medline]
  26. Kramer BK, Montagnino G, Del Castillo D, Margreiter R, Sperschneider H, Olbricht CJ, Kruger B, Ortuno J, Kohler H, Kunzendorf U, Stummvoll HK, Tabernero JM, Muhlbacher F, Rivero M, Arias M: Efficacy and safety of tacrolimus compared with cyclosporin A microemulsion in renal transplantation: 2 year follow-up results. Nephrol Dial Transplant20 :968 –973,2005[Abstract/Free Full Text]
  27. Boots JM, van Duijnhoven EM, Christiaans MH, Nieman FH, van Suylen RJ, van Hooff JP: Single-center experience with tacrolimus versus cyclosporine-neoral in renal transplant recipients. Transpl Int14 :370 –383,2001[CrossRef][Medline]
  28. Meiser BM, Groetzner J, Kaczmarek I, Landwehr P, Muller M, Jung S, Uberfuhr P, Fraunberger P, Stempfle HU, Weis M, Reichart B: Tacrolimus or cyclosporine: Which is the better partner for mycophenolate mofetil in heart transplant recipients? Transplantation78 :591 –598,2004[Medline]
  29. Zuckermann A, Reichenspurner H, Birsan T, Treede H, Deviatko E, Reichart B, Klepetko W: Cyclosporine A versus tacrolimus in combination with mycophenolate mofetil and steroids as primary immunosuppression after lung transplantation: One-year results of a 2-center prospective randomized trial. J Thorac Cardiovasc Surg125 :891 –900,2003[Abstract/Free Full Text]
  30. Gonwa T, Johnson C, Ahsan N, Alfrey EJ, Halloran P, Stegall M, Hardy M, Metzger R, Shield C 3rd, Rocher L, Scandling J, Sorensen J, Mulloy L, Light J, Corwin C, Danovitch G, Wachs M, VanVeldhuisen P, Leonhardt M, Fitzsimmons WE: Randomized trial of tacrolimus + mycophenolate mofetil or azathioprine versus cyclosporine + mycophenolate mofetil after cadaveric kidney transplantation: Results at three years. Transplantation75 :2048 –2053,2003[CrossRef][Medline]
  31. Murphy GJ, Waller JR, Sandford RS, Furness PN, Nicholson ML: Randomized clinical trial of the effect of microemulsion cyclosporin and tacrolimus on renal allograft fibrosis. Br J Surg90 :680 –686,2003[CrossRef][Medline]
  32. Kaplan B, Schold JD, Meier-Kriesche HU: Long-term graft survival with neoral and tacrolimus: A paired kidney analysis. J Am Soc Nephrol14 :2980 –2984,2003[Abstract/Free Full Text]
  33. Irish W, Sherrill B, Brennan DC, Lowell J, Schnitzler M: Three-year posttransplant graft survival in renal-transplant patients with graft function at 6 months receiving tacrolimus or cyclosporine microemulsion within a triple-drug regimen. Transplantation76 :1686 –1690,2003[Medline]
  34. Mange KC: Challenges from bias when estimating change of renal allograft function. Kidney Int66 :463; author reply 463–464,2004[Medline]
  35. Shapiro R, Young JB, Milford EL, Trotter JF, Bustami RT, Leichtman AB: Immunosuppression: Evolution in practice and trends, 1993–2003. Am J Transplant5 :874 –886,2005[CrossRef][Medline]
  36. Woodward RS, Kutinova A, Brennan DC: Tacrolimus versus cyclosporine microemulsion: Immunosuppressive regimen makes a difference in living renal transplantation. American Transplant Congress, Seattle, Washington, May 21 to 25, 2005 [Abstract 1366]. Am J Transplant5[Suppl 11] :503 ,2005
  37. Woodward RS, Kutinova A, Brennan DC: Immunosuppressive regimen makes a difference in cadaveric renal transplantation: Tacrolimus versus cyclosporine microemulsion. American Transplant Congress, Seattle, Washington, May 21–25, 2005 [Abstract 974]. Am J Transplant5[Suppl 11] :404 ,2005
  38. Webster AC, Woodroffe RC, Taylor RS, Chapman JR, Craig JC: Tacrolimus versus ciclosporin as primary immunosuppression for kidney transplant recipients: Meta-analysis and meta-regression of randomised trial data. BMJ331 :810 ,2005[Abstract/Free Full Text]
  39. Bustami RT, Ojo AO, Wolfe RA, Merion RM, Bennett WM, McDiarmid SV, Leichtman AB, Held PJ, Port FK: Immunosuppression and the risk of post-transplant malignancy among cadaveric first kidney transplant recipients. Am J Transplant4 :87 –93,2004[Medline]




This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
CJN.00640805v1
1/3/563    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Goldfarb-Rumyantzev, A. S.
Right arrow Articles by Barenbaum, L. L.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Goldfarb-Rumyantzev, A. S.
Right arrow Articles by Barenbaum, L. L.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS