Abstract
Incorporation of the patient perspective into research and clinical practice will enrich our understanding of the status and management of patients with glomerular disease and may result in therapies that better address patient needs. In recent years, the importance of the patient experience of glomerular disease has become clear, and significant efforts have been undertaken to systematically capture and describe the patient’s disease experience. Patient–reported outcome instruments provide a means to assess the patient’s experience in a quantitative manner, thus enabling for comparisons within and between patients. Patient–reported outcome assessments are solely on the basis of a patient report about the status of their health without amendment or interpretation by a clinician or others. Patient–reported outcome assessments provide an opportunity to incorporate the patient perspective into clinical care, research, and clinical trials. Our paper provides an overview of terminology and development methods for patient-reported outcomes and reviews (1) currently available patient–reported outcome instruments appropriate for use in glomerular disease, (2) existing patient–reported outcome data in glomerular disease, and (3) opportunities for incorporating patient–reported outcome instruments into clinical care and research.
- quality of life
- kidney disease
- glomerulopathy
- patient-centered outcomes
- Disease Management
- Humans
- Patient Outcome Assessment
- glomerulonephritis
Introduction
Although the health of a patient and consequences of a disease can be assessed in a number of ways, the perceptions of a patient’s health and disease are primarily ones of an individual—the patient (Figure 1) (1,2). Subsequently, the outcome of a disease experience may be defined in patient-centered ways considering, for example, physical, mental, and social functioning, perception of wellbeing, and clinical outcomes, such as organ function and patient survival. In clinical practice, the patient voice is often sought and documented in qualitative ways that are difficult to track within groups of patients with a similar condition or longitudinally in the management of a single patient. Use of patient–reported outcome (PRO) instruments in clinical and research settings allows the clinician and researcher to capture the patient perspectives of health and disease in a quantitative way that can be readily incorporated into patient management and clinical research.
A comprehensive view of the health of a patient with glomerular disease may include clinical markers, physician determination of clinical disease severity, patient-reported outcomes, and hard end points. UPC, urine protein-to-creatinine ratio.
The value of high–quality PRO instruments has been recognized by clinicians, payers, and regulators in a variety of therapeutic areas. This manuscript contains a general discussion of PRO instruments and provides an overview of the use of PRO assessments in clinical trials, observational studies, and clinical care of patients with glomerular disease.
The key point is that PRO instruments provide a means to incorporate a quantitative assessment of the patient voice into clinical practice and research.
Glossary
Health–related quality of life (HRQOL) is a multidomain concept representing the patient’s general perception of the effect of illness and treatment on physical, psychologic, and social aspects of life.
Clinical outcome assessment (COA) is an assessment of a clinical outcome that can be made through report by a clinician, patient, or nonclinician observer or with a performance-based assessment. There are four types of COAs: PRO, observer-reported outcome (ObsRO), clinician-reported outcome (ClinRO), and performance outcome.
PRO is a measurement on the basis of a patient report that comes directly from the patient (i.e., study subject) about the status of his/her health condition without amendment or interpretation by a clinician or anyone else.
ObsRO is a measurement on the basis of a report of observable signs, events, or behaviors related to a patient’s health condition by someone other than the patient or a health professional.
ClinRO is a measurement on the basis of a report that comes from a trained health care professional after observation of a patient’s health condition. Most ClinRO measures involve a clinical judgment or interpretation of the observable signs, behaviors, or other manifestations related to a disease or condition. ClinRO measures cannot directly assess symptoms that are known only to the patient.
Content validity is the extent to which the assessment measures what it purports to measure in a specific context of use (1,3).
Terminology
In 2016, the US Food and Drug Administration (FDA) –National Institutes of Health (NIH) Biomarker Working Group published the Biomarkers, Endpoints, and Other Tools (BEST) Resource (3). The resource includes definitions of terms used in translational science and medical product development, including terminology related to PRO instruments. Although this paper discusses the use of PRO instruments in the clinical practice setting as well as the research setting, for the purpose of this paper, we provide definitions compiled from the BEST and PRO guidance for industry (1,3).
PROs are a type of COA; hence, to appreciate the meaning of a PRO, one needs to understand a COA. A COA is an assessment of a clinical outcome made through report by a clinician, patient, or nonclinician observer or with a performance-based assessment (3). It is important to distinguish among the types of COAs, particularly PROs, ObsROs, and ClinROs. A PRO is defined as a measurement on the basis of a patient report that comes directly from the patient about the status of his/her health condition without amendment or interpretation by a clinician or anyone else (1). A PRO instrument can be used to assess concepts that may be narrow (e.g., pain intensity) or broad (e.g., HRQOL). In contrast, an ObsRO is a measurement on the basis of a report of observable signs, events, or behaviors related to a patient’s health condition by someone other than the patient or a health professional (3). ObsRO can be used in settings where a patient is too young or otherwise unable to complete a self-reported instrument. Finally, a ClinRO is a measurement on the basis of a report that comes from a trained health care professional after observation of a patient’s health condition (3). Neither the observer nor the clinician can validly report information known only to the patient, such as an inability to perform a task because of pain interference.
Some PRO instruments may have an associated proxy version that allows a parent or other family member to complete questions about the physical, mental, or social health of the patient. These tools may include questions that require inference by the proxy reporter about how a patient feels and hence, should not be considered a PRO or an ObsRO. Parent report of a child’s HRQOL is an example of a proxy instrument that does not meet the definitions of a PRO or an ObsRO. This distinction has important implications when evaluating COAs in clinical trials, particularly those that are intended to support medical product approval as discussed.
The key point is that a PRO is defined as a measurement on the basis of a patient report about the status of his/her own health condition without amendment or interpretation by anyone else.
PRO Instruments
PRO instruments have been developed for generic and disease-specific use. Generic instruments, such as the Pediatric Quality of Life Inventory (PedsQL) (4), the Short–Form Health Survey (SF-36) (5), and the Patient–Reported Outcome Measurement Information System (PROMIS) (6), assess concepts that may be relevant across diseases or patient populations (Table 1). PRO measures differ in the quantitative approaches used in development, which include classic test theory and item response theory (including Rasch analysis or graded response model) (7,8). Detailed descriptions of each methodology and the associated statistical methods are beyond the scope of this review and can be found in the associated references (8–12). In practical terms, PRO instruments aim to balance response burden and precision by using high-quality items (questions), which measure a single concept across a range of responses. Administration of several items within a domain may improve the precision of the score. Figure 2 provides an example of item selection using item response theory and items with good versus poor discrimination properties.
Patient–reported outcome measures used in research in glomerular disease
Examples of item response theory item characteristics curves. These three-item information plots provide descriptive information for Likert scale items with five different response options (never, rarely, sometimes, often, always). Higher values for probability (y axis, left) and information (y axis, right), non-overlapping peaks, and a plot that is centered at θ=0 (x axis; θ values should be normally distributed) indicate better item characteristics. The dotted lines indicate overall item information. High and wide is better. Plot (A) provides an example of an excellent item. Plots (B) and (C) provide examples of poor items ([B] = poor discrimination; [C] = poor difficulty).
There are important differences between how PRO instruments that are developed using classic test theory and those that are developed using item response theory are used or administered. PRO instruments developed using classic test theory (e.g., SF-36 and PedsQL) require administration of the entire test with a prespecified set of domains. Conversely, PRO instruments developed using item response theory, such as the PROMIS, allow the selection of a single item or a subset of domains or items deemed to be of greatest relevance to the patient population under study or care. Indeed, the effective use of the PROMIS requires the selection of relevant domains from the library of 23 adult and 21 pediatric domains (www.nihpromis.org) (Figure 3). One approach to domain selection includes (1) concept elicitation from patients across a spectrum of disease severity, (2) selection of existing items that match the patient defined concepts, and (3) development and validation of items to capture high-priority concepts missing from available PRO resources. The goal of disease-specific tailoring of the PRO instrument is to reduce the burden of administering the PRO and the noise from domains that are irrelevant to the patient disease experience while preserving the signal from relevant domains (see the Outcome Measures in Rheumatology [OMERACT] example below).
The Patient–Reported Outcome Measurement Information System (PROMIS) was developed with item response theory methods. Selection and use of an individual or a subset of items or domains that are specifically relevant to a specified patient population can improve patient–reported outcome measurement precision and reduce question burden for irrelevant domains. (A) Adult domains for ages 18 years old and older. (B) Pediatric domains. Children can complete the PROMIS reliably at age 8 years old and older. Reprinted from the PROMIS Health Organization and the PROMIS Cooperative Group (http://www.nihpromis.org/measures/full_framework.aspx), with permission.
Regardless of how they are developed, a common limitation of generic PRO instruments is that they lack domains that capture unique disease experiences particular to specific conditions, such as glomerular disease. As a result of these limitations, researchers may choose to use disease-specific instruments alone or in combination with generic instruments.
The key point is that there are adult and pediatric PRO instruments currently available. However, generic instruments may lack the specificity needed to precisely capture the patient glomerular disease experience.
Incorporating PRO Instruments in Clinical Trials
In recent years, there has been increasing interest in incorporating the patient voice into drug development. PROs provide an important means to incorporate the patient perspective into clinical trials end points. According to the US Code of Federal Regulation (CFR), particularly 21 CFR 314.126, methods of assessing a patient’s response to a treatment should be “well defined and reliable” (13). In 2009, the FDA finalized a PRO Guidance for Industry to communicate how the FDA reviews PRO instruments to determine whether they meet this regulatory standard to support claims in medical product labeling (1). However, regulatory flexibility and judgment are necessary to meet the practical demands of drug development, and other approaches may also be acceptable.
From a regulatory perspective, a key aspect of a PRO instrument is its content validity (i.e., the extent to which the instrument measures what it purports to measure in a specific context of use). Establishing the content validity of an instrument is important for describing the benefit and effect of a treatment (i.e., how the drug affects how patients feel or function) in drug labeling. The process of establishing content validity includes generating evidence from qualitative studies that the items of an instrument are appropriate and comprehensive relative to what the instrument claims to measure in the targeted patient population and context of use. Quantitative evaluation of an instrument’s other measurement properties (e.g., reliability and ability to detect change) is important to establish but cannot replace evidence of content validity. In general, when developing or selecting a PRO instrument, it is important to include patient input and use qualitative research methodology to identify the most important and relevant aspects of the patient experience (1,14).
Another important aspect of capturing the patient voice in clinical trials is deciding what to measure. PROs and other COAs that aim to measure a broad range of concepts reported in a summary score are generally not optimal for showing the efficacy of a therapy in a clinical trial. Included in these instruments are often questions that are indirectly related to the potential therapeutic action of a drug, whereas others may not describe a treatment benefit at all (e.g., presence of family support) or may capture information on aspects of life that can be affected by many factors other than the underlying disease and treatment. Although such instruments can be important in clinical care from a drug development standpoint, it is important to use specific PRO instruments that focus on core, disease–related symptoms or functional effects of a disease, which may be affected by the experimental intervention. In general, the inclusion of attributes of wellbeing that are indirectly related to the potential therapeutic action of a drug (e.g., worry and hope) attenuates the overall ability of the instrument to detect a product’s efficacy and makes interpretation of any observed changes challenging. That is not to say that such multidomain PRO instruments are never useful in the clinical trial context, but careful thought and planning are important when selecting a PRO instrument to ensure that the effects of the treatment can be clearly described in product labeling.
Although many of the concepts related to PRO instruments apply to both adult and pediatric populations, one issue that arises when conducting pediatric trials is how to capture the response in children who are unable to respond for themselves. In such a setting, it may be reasonable to use ObsROs. Observer reports should only be used to capture events or behaviors that can be observed. In relation to glomerular diseases and the manifestations of nephrotic syndrome, certainly parents and caregivers can observe the presence of edema in a child. However, a key issue is whether the observed changes in edema are meaningful to the child (i.e., affect how the child feels or functions). Because edema can interfere with activities dependent on physical functioning, observer report of edema could be complemented by an observation-based instrument assessing the effect of the edema on daily life functioning or mobility. Furthermore, a pattern of improvement on both outcomes in a clinical trial could provide evidence of clinical benefit.
The key point is that PRO assessments may be used as end points in clinical trials to establish the effectiveness of medical products and support claims in medical product labeling. Guidance is available from the FDA on the development and use of PRO instruments in medical product development.
Existing PRO Studies in Glomerular Disease and Lessons Learned from Vasculitis
Lessons Learned from Vasculitis
Vasculitis provides an excellent example of the evolution of PRO instruments in a therapeutic area that overlaps with other glomerular diseases. Similar to many other glomerular diseases, the vasculitides are systemic in nature and affect multiple organ systems. Moreover, kidney dysfunction is also a critical component of many vasculitides and contributes significantly to patient morbidity. Thus, the strategies adopted and lessons learned from the development of PRO instruments for vasculitis serve as a useful model.
The OMERACT group is an international group of clinical investigators, patients, methodologists, and representatives from the biopharmaceutical industry and regulatory agencies focused on data-driven approaches to validating outcome instruments for use in clinical trials. The OMERACT Vasculitis Working Group implemented a multiyear research agenda, which led to the evaluation, validation, and endorsement of a “Core Set” of outcome measurements for use in clinical trials of ANCA-associated vasculitis (15). The Core Set includes the following domains: disease activity, disease damage, PRO instruments, and mortality. Before endorsement of the Core Set, each domain had to have at least one associated validated outcome assessment.
Early efforts to identify a PRO instrument to use as an outcome assessment examined the SF-36, and differences were detected in a manner comparable with the experience with other chronic diseases with modest correlation to physician-based instruments (16,17). Similarly, a simple Patient Global Assessment was useful in assessing disease activity in ANCA-associated vasculitis (18). Although these instruments had some use in vasculitis, it became clear that generic instruments lacked the ability to capture the full spectrum of outcomes important to these patients (19–22).
In response to the unmet need to develop PRO instruments, the OMERACT Vasculitis Working Group spearheaded parallel projects to advance the development and validation of PROs in vasculitis. Three initial projects focusing on PROs in ANCA-associated vasculitis were undertaken: (1) exploration of the utility and validity of different PROMIS domains and instruments in ANCA-associated vasculitis, (2) development of a disease–specific PRO instrument for ANCA-associated vasculitis (ANCA-associated vasculitis PRO), and (3) examination of outcome instruments in vasculitis according to the World Health Organization’s International Classification of Function (22). Through individual interviews with patients in three countries using standard qualitative methods, themes of importance to patients were extracted; these data guide all three projects. This collaborative effort, which represents a work in progress, provides a useful model for clinicians and researchers working in the arena of glomerular diseases.
To date, several large randomized trials in ANCA-associated vasculitis have included PRO instruments as secondary outcomes using the SF-36. These include the trials of etanercept as adjuvant to standard therapy and rituximab for induction of remission (23,24). In each of these studies, treatment was associated with an improvement in the PRO score. To date, trials using a disease-specific instrument as a protocol–defined outcome assessment have not been conducted in patients with vasculitis.
The medical literature in vasculitis has taught us several valuable lessons. First, generic PRO instruments in clinical trials provide insight into the patient experience and show that the ANCA-associated vasculitides significantly affect patients. Second and more importantly, the work of the OMERACT group has shown that generic instruments do not fully capture the relevant disease experience in these patients. Third, this group has created a path forward, which may be modeled by other study groups, that has resulted in the creation of a disease-specific instrument that, when validated, may serve to better characterize PRO measurement in these patients.
Generic PRO Instruments in Nephrotic Syndrome
Some studies have evaluated PRO instruments in adults and children with nephrotic syndrome. In the NIH–sponsored FSGS Clinical Trial, 138 patients with steroid-resistant FSGS were randomized to one of two treatment arms. PROs were assessed using the PedsQL in the 94 pediatric participants and the SF-36 in the 45 adult participants. At enrollment, trial participants had a worse PRO than healthy controls. Children with FSGS reported lower domain scores in physical, emotional, and school functioning and lower total score. Similarly, the adults reported lower domain scores than healthy controls in the SF-36 Mental Health and Physical Health composites and the domains of mental health, vitality, general health, role physical, and social functioning (25). This trial did not show a difference in proteinuria end points or PRO outcomes between the study arms. In a cross-sectional study of 151 children with nephrotic syndrome designed to validate the PROMIS instruments, children with active disease (edema) had worse scores on the anxiety, fatigue, pain interference, and mobility domains (26). In a prospective study of 127 children with active nephrotic syndrome, the PROMIS scores were worse in prevalent versus incident patients in the domains of peer relationship and pain interference. Within this same study, the PedsQL scores were also worse among prevalent patients in comparable domains of social functioning and school functioning (27). Finally, in a cross-sectional study of 99 patients with nephrotic syndrome, the SF-36 PRO scores were associated with proteinuria clinical outcomes (28).
In total, PRO studies have shown that adults and children with nephrotic syndrome have significant impairments detectable with generic PRO instruments compared with healthy controls and active nephrotic syndrome compared with inactive disease. However, similar to studies in vasculitis, generic instruments alone may lack the ability to capture the patient disease experience in nephrotic syndrome in a precise, sensitive, and patient-relevant manner. Specifically, generic instruments do not describe all of the disease characteristics, such as individual symptoms (e.g., edema), overall symptom burden, and the relapsing nature of the disease, that are relevant to this patient population. A single disease–specific PRO for adults with FSGS has been generated and is now undergoing testing within an ongoing clinical trial (29). No other disease–specific PRO instruments for adults or children with nephrotic syndrome or its associated conditions have been published.
PRO Instruments in Observational Studies
PRO instruments are important outcome assessments in observational studies of patients with glomerular disease for many of the same reasons that they are important for clinical trials. PROs can be informative for short-, intermediate-, and long-term studies. The importance of PRO assessments in cohort studies has been recognized by the NIH, and PRO instruments have been increasingly incorporated into longitudinal glomerular disease cohort studies, such as the Nephrotic Syndrome Study Network and CureGN projects as well as the those of the Vasculitis Clinical Research Consortium (30,31). Information obtained from these prospective observational studies may be used to help establish the validity and reliability of these PRO instruments and support the use of these assessments as end points in future drug development trials.
The key point is that PRO instruments have consistently shown that patients with a variety of glomerular diseases have impairments but that generic instruments do not fully capture the patient disease experience. Disease-specific instruments are needed, and they are being developed for glomerular diseases.
Implementing PROs in Clinical Trials and Clinical Care
PRO Instruments in Clinical Trials
Across the medical fields, PRO use has been increasing in clinical trials; currently, 27% of all clinical trials registered on Clinicaltrials.gov incorporate PRO instruments (32). Several drugs have been approved with FDA labeling describing the effect of the drug on a PRO instrument. An example of one such drug is ruxolitinib, an mAb kinase inhibitor that is used to treat severe myelofibrosis. Specifically, the product labeling notes a “a greater likelihood of a 50% or greater reduction in Total Symptom Score from baseline to Week 24 as measured by the modified Myelofibrosis Symptom Assessment Form.” As of the publication of this manuscript, there have not yet been any clinical trials in glomerular disease where PRO assessments have served as the primary outcome or where PROs have served an important role in drug approval or labeling.
Transitioning from Research to Clinical Care
Although PRO instruments have been administered in patients with glomerular disease in a number of research settings, incorporation of PRO assessments into clinical practice requires an implementation strategy and PRO results that are readily available, interpretable, and relevant to treatment decision making. In nephrology practice, PRO assessments have been added to the care of patients on dialysis in the United States on the basis of a mandate by the Center for Medicare and Medicaid Services (33). An equivalent mandate does not exist in glomerular disease, but PRO instruments are available within many nondialysis care environments and electronic health record systems.
Typically, PRO responses are transformed into continuous scores, often ranging from zero to 100. From a clinical standpoint, a clinician might ask “what does a mobility score of 55 indicate?” When the score is reported on a T scale normed to 50 with an SD of 10, we know that a value of 55 is five points or one half of an SD above the calibrated mean (34). Nephrologists already routinely interpret clinically meaningful strata and changes of other markers of glomerular disease, such as eGFR, serum albumin, and proteinuria. The interpretation of PRO scores may similarly use the concept of strata of scores and meaningful changes in scores. PRO score strata may be defined relative to a healthy population or relative to the range of scores observed from within the same disease population. In longitudinal single–patient follow-up, the determination of a score change of sufficient magnitude to consider a change in management may be informed by prior publications documenting the minimally important difference in score within the same population (35) statistically by, for example, a change of ≥0.5 SD or anchor-based methods. A minimally important difference of three points in the PROMIS instruments has been defined for children with nephrotic syndrome and other chronic health conditions through research including patients, parent caregivers, and physicians (35). The determination of the meaning of PRO scores assessed cross-sectionally and changes in scores assessed longitudinally will advance the clinical utility of these outcomes.
PRO instruments can include multiple or single domains. When patients are invited to identify the domains of relevance, physical, mental, and social health concepts may emerge as important. Incorporating a meaningful set of PRO domains in a single instrument as part of a clinical visit may help pinpoint which domains are problematic for an individual patient. Targeted intervention may then be prescribed to improve the physical, mental, or social health. For example, patients with active glomerular disease often report poor mobility and pain (26). Recognition of these PROs and targeted treatment may improve patient outcomes and satisfaction with care (36–38).
Administration of PROs in clinical settings may be delivered through electronic medical record patient portals, in paper format, or administered verbally. When incorporating PRO assessments into clinical care, a variety of options for administration of PRO instruments may be needed to assure assessment of all patients, regardless of access to computers, literacy, or functional differences in vision, hearing, and dexterity (39).
The key point is that PRO instruments are increasingly being incorporated into clinical trials. PROs can be used in medical product development to support claims in medical product labeling. The electronic medical record provides an opportunity for the incorporation of PRO instruments into clinical care.
Conclusions
Incorporation of the patient perspective into research and clinical practice will enrich our understanding of the status and management of patients with glomerular disease beyond the traditional clinical and laboratory-based assessments. Current methodologies for the development, validation, and selection of PRO instruments can be readily adopted by the research, clinical, and patient communities to accelerate the use of PRO instruments. The experience in the field of vasculitis provides a paradigm that can be applied in nephrology and in particular, to patients with other forms of glomerular disease. Incorporation of PRO assessments into clinical trials can provide end points that are meaningful and measureable, even in short-term studies. The importance of the patient experience has been recognized by the FDA with acceptance of PRO assessments as end points for clinical trials. The FDA provides guidance so that PRO instruments are developed in a way that supports their use in the generation of evidence to support medical product testing. Several generic PRO instruments exist for use by clinicians and researchers, but they may not adequately capture the patient experience with glomerular diseases. Disease-specific instruments are emerging that may, alone or with generic measures, augment our ability to capture the patient experience with greater precision.
Disclosures
D.T.S., A.T., S.K., E.J.P., N.E.C., and J.P.T. have no conflicts of interest to report. H.T. serves as a consultant to Otsuka and Kaneka. P.A.M. receives research support from Actelion, Bristol-Myers Squibb (Princeton, NJ), Celgene, ChemoCentryx, Genentech (South San Francisco, CA)/Roche (Basel, Switzerland), and GlaxoSmithKline (Brentford, United Kingdom) and provides consulting for Actelion, Alexion, Bristol-Myers Squibb, ChemoCentryx, Genentech/Roche, MedImmune/AstraZeneca Pharmaceuticals (Wilmington, DE), Prothena, and Sanofi US (Bridgewater, NJ). D.S.G. serves as a consultant for GlaxoSmithKline, Retrophin, Janssen Biotech (Horsham, PA), and Bristol-Myers Squibb (through University of Michigan) and receives research support from Retrophin, GlaxoSmithKline, and Bristol-Myers Squibb.
Acknowledgments
D.T.S., J.P.T., D.S.G., and the work described for pediatric nephrotic syndrome received support from National Institutes of Health, National Center for Research Resources (grant 5U01AR052181-05), National Institute of Arthritis and Musculoskeletal and Skin Diseases (grant 2U01AR05218106), and the NephCure Kidney International Foundation. The Nephrotic Syndrome Patient–Reported Outcomes Workshop was supported by the NephCure Kidney International Foundation. P.A.M. and the work described for vasculitis was supported by the Vasculitis Clinical Research Consortium, which has received support from National Institute of Arthritis and Musculoskeletal and Skin Diseases (grants U54 AR057319, U01 AR5187404, and R01 AR064153); the National Center for Advancing Translational Science; and the Office of Rare Diseases Research. Additional support for the work of the Outcome Measures in Rheumatology Vasculitis Working Group was received through a Patient–Centered Outcomes Research Institute Pilot Project grant. D.S.G. served as a coinvestigator in the development of the FSGS patient–reported outcome sponsored by GlaxoSmithKline (Brentford, United Kingdom).
This article reflects the views of the authors and should not be construed to represent the views or policies of the US Food and Drug Administration.
Footnotes
Published online ahead of print. Publication date available at www.cjasn.org.
- Copyright © 2016 by the American Society of Nephrology