FDA perspective on objective performance goals and clinical trial design for evaluating catheter-based treatment of critical limb ischemia
Article Outline
- Abstract
- Regulatory background
- Regulatory requirements for device evaluation
- The SVS proposal for critical limb ischemia trials
- Conclusion
- Author contributions
- References
- Copyright
The article by Conte et al.1 on behalf of the Society for Vascular Surgery (SVS) in this issue of the Journal of Vascular Surgery provides guidelines for improving the consistency and interpretability of clinical trials intended to evaluate treatment options for patients with critical limb ischemia (CLI). This article identifies a number of key challenges with conducting and comparing CLI trials, including the wide spectrum of clinical presentations that CLI encompasses, the use of disparate eligibility criteria and endpoint measurements, and logistical and economic considerations that can limit study initiation and completion. The authors propose definitions for a number of performance goals derived from historical surgical literature as a means of reducing the negative impact of these factors. The current editorial reviews aspects of this proposal from the perspective of the authors in terms of their understanding of the statutory obligations of the U.S. Food and Drug Administration (FDA) to regulate the marketing of cardiovascular devices based on valid scientific evidence.
Regulatory background
Since the passage of the Medical Device Amendments to the Food, Drug, and Cosmetic Act in 1976, FDA has been charged with regulating marketing approval of new medical devices.2 Permission to market a new device is based on a review of valid scientific evidence that provides reasonable assurance that the device is both safe and effective for use in an intended population for a specified indication, based on a determination that the probable benefits of device use outweigh the probable risks. This aspect of FDA's mission is limited to regulating medical device and their manufacturers; the Agency is explicitly prevented from any such regulation of practitioners or the practice of medicine by the FDA Modernization Act of 1997.3 Thus, physicians may use any currently marketed device in a manner they deem would be in the best interests of their patient. Such “off-label use” often occurs with the interventional treatment of peripheral vascular atherosclerotic disease, in that these devices are frequently used outside of their approved indications. One consequence of this practice is the reduced incentive for manufacturers to conduct pivotal clinical trials to support FDA approval for these indications if their devices are already marketed for other intended uses. It is also important to note that FDA does not regulate or approve types of medical procedures by themselves, only the devices used therein, and that labeled indications are limited to those allowed by FDA based on submitted information.
Regulatory requirements for device evaluation
There are three main pathways for medical devices to enter the market: Premarket Notification via demonstration of substantial equivalence to a similar marketed device (commonly known as 510[k], the section of the Medical Device Amendments that defines this program); approval of a Premarket Approval Application (PMA), which requires demonstration of safety and effectiveness for the stated intended use; and approval of a Humanitarian Device Exemption (HDE), which is based on demonstration of safety and probable benefit in lieu of effectiveness for devices intended to treat or diagnose a disease or condition that affects fewer than 4,000 patients annually. Most permanent implants used to treat peripheral vascular atherosclerotic disease, including vascular stents and endovascular grafts for treating aortic aneurysms, are regulated via the PMA pathway because they present the highest level of risk to the patient in event of failure, and because FDA believes that a complete set of special controls such as performance standards does not yet exist for allowing these devices to be cleared via 510(k). By contrast, many non-implant devices such as peripheral balloon angioplasty catheters and atherectomy devices reach the marketplace via the 510(k) pathway through a determination of substantial equivalence, due to the lower risk profile that these devices present.
For cardiovascular devices, demonstration of substantial equivalence or reasonable assurance of safety and effectiveness frequently involves the collection of clinical data to support the proposed intended use. FDA typically recommends that such data be collected through the conduct of randomized, controlled trials (RCTs), as such studies provide the highest levels of clinical evidence and patient comparability.4 However, in accordance with the “least burdensome” provisions specified in the FDA Modernization Act of 1997,5 FDA accepts data from alternative study designs, such as single-arm studies using historical controls, provided the data are scientifically sound and free of bias. Such studies have been used to support marketing clearance/approval when an RCT is not possible due to ethical concerns or challenges in achieving a satisfactory rate of subject enrollment. Acceptance of clinical data resulting from non-RCTs is facilitated when objective performance criteria (OPCs) have been developed, as they have for certain cardiovascular devices such as prosthetic surgical heart valves and ventricular assist devices. Development of robust OPCs generally requires relatively mature device technology and the availability of high-quality historical clinical evidence.
The Society for Vascular Surgery (SVS) authors propose a similar approach for evaluating new devices used in the treatment of critical limb ischemia (CLI). As part of this approach, SVS reviewed the available literature to identify suitable reports of RCTs involving endovascular, pharmacologic, and surgical treatments, and formulated sets of standardized endpoint definitions, entry criteria, and performance benchmarks for use in designing CLI studies. Because these historical studies vary greatly in their designs and definitions, the authors elected to define these standards as “objective performance goals,” rather than OPCs. Development of proper OPCs may be possible once additional, standardized clinical data from CLI studies are available.
In addition to their goal of enhancing comparability across different CLI studies, the SVS authors also believe that single-arm studies designed using their proposed definitions and metrics can facilitate the timely evaluation and adoption of new treatment options for CLI. From a regulatory standpoint, such an approach can provide a “least burdensome” pathway for device approval/clearance, provided the data collected are sufficiently robust and applicable to the relevant CLI population. Single-arm studies and other alternative study designs have been successfully used to support marketing submissions for other peripheral vascular devices, thus allowing physicians access to additional novel treatment options.
The SVS proposal for critical limb ischemia trials
SVS has conducted a laudable effort to facilitate the clinical investigation of devices for treating patients with CLI, a population sorely lacking evidence-based treatment strategies. The Society's research has identified useful parameters for designing new CLI studies, and culled out some of the challenges with leveraging historical data for use as a control for comparing diverse treatment options. The following are our comments on the SVS proposal with respect to the evaluation and subsequent approval/clearance of devices used to treat CLI, as part of our efforts to provide a transparent and least burdensome regulatory review process.
In their analysis, the SVS authors have analyzed and compiled the best available literature to provide a historical baseline for the evaluation of devices to treat CLI. They identified three trials with available patient-level data from control patients who underwent surgical bypass, the current gold standard for revascularization. Based on their goals for management of CLI patients, they formulated eight metrics of import to the assessment of revascularization in CLI, encompassing clinical as well as functional endpoints. Each provides an important assessment and can provide baseline data for comparison when evaluating novel devices.
CLI as a disease entity is heterogeneous, with a final common pathway of limb ischemia and tissue loss. The underlying pathology, comorbidities, patient demographics, and vascular anatomy vary among patients. Treatment goals vary across patients, and strategies for successful treatment are necessarily different depending on the individual patient scenarios and the techniques and devices used. As such, treatment of CLI often entails a multi-factorial treatment strategy, using multiple modalities at multiple anatomic levels. The treating physician thus needs an armamentarium of devices and procedures, and often employs several in an individual patient. From a regulatory standpoint, each “tool” in this “toolbox” must be shown to be safe and effective for its intended use prior to marketing clearance or approval.
Designing trials to evaluate each device poses challenges to standardization, as each fits a unique niche within the overall treatment paradigm. Devices may be indicated for specific instances, and in particular patient populations. As a result, each device may have its own set of relevant outcome metrics and significant adverse event types.6
This heterogeneity in patient demographics and treatment effects means that different devices may be best assessed using different metrics and endpoints. The use of historical controls in such a situation becomes challenging due to differences in clinical cohorts and in the action of the devices themselves.4 While there is some published data regarding treatment of patients with CLI, most of the data that exist are specific to the devices or procedures studied.
The SVS analysis can be seen as providing a starting point for designing prospective clinical studies involving CLI treatment. With this approach, sponsors can cull out historical outcomes for patients with characteristics that match the population to be treated to derive appropriate performance goals. Specifically, in addition to overall patient demographics and comorbidities, SVS has identified octogenarian status and the presence of tissue loss as being characteristic of a “Clinical High Risk” patient subgroup predictive of worse outcomes.1 The performance goals are therefore highly dependent on the proportion of patients with these high-risk features, and the portability of the performance goals to subsequent trial designs is therefore predicated on proper matching of study cohorts.
It is expected that different combinations of the eight identified metrics may be appropriate for comparisons of individual devices, based on the particular technology and patient population studied. In addition, we expect that functional, as well as clinical and anatomic, endpoints may be appropriate for the assessment of new device-based treatment options.
This ground breaking work by SVS can be used as the basis for clinical evaluation of the first generation of endovascular devices to treat CLI. We hope that as additional CLI studies are completed and the body of available clinical evidence becomes larger, the performance goals can be updated and further refined to reflect more recent standards of care, thus forming a clinical database for future treatment comparisons. In particular, we expect that, over time, robust data from endovascular approaches will be available, from which performance goals can be generated for use in evaluating the next generation of devices. Future study designs can incorporate these baseline data sets and facilitate the most suitable comparisons of subject devices based on their technological characteristics and indicated patient population.
It is important to note that certain limitations are inherent in the design of any single-arm study. The presence of confounding factors, such as patient demographics, anatomy, differences in comorbidities and the extent of disease, and the proportion of clinical high-risk features, can result in treatment and control groups that are not completely comparable, thus diminishing the interpretability of the resulting data.4 While the impact of known confounders can potentially be addressed through statistical techniques, unknown confounding factors may always be present and their effects cannot easily be quantified. In addition, certain novel device types, such as drug-device combination products, may not be suitable for evaluation in a single-arm study because historical data may not represent an adequate comparator due to differences in technology and clinical practice. RCTs may be necessary to evaluate these types of products.
Conclusion
Clinical trial designs always incorporate a delicate balancing act. Study sponsors must carefully define the appropriate target population and outcome measures, weigh the risks of the procedure versus the potential benefits, establish clinically meaningful goals for study success and design a study that can reasonably be completed in a timely fashion. Because the possible treatment options for CLI are diverse, much like the manifestations of the disease itself, a single paradigm for CLI study design may not be sufficient to evaluate all device types or patient populations in the most appropriate manner. Enhancing the design of clinical studies is an area that can be facilitated greatly through the collaboration of all stakeholders interested in learning how to better treat patients, including the clinical community, medical device manufacturers, and regulatory agencies.
FDA believes that the definitions and performance goals developed by SVS may help to define a starting point for industry sponsors considering their own CLI studies. We expect that modifications could be warranted, based on the specific devices and populations to be studied. These goals may aid the development of clinical trial designs capable of evaluating the safety and effectiveness of peripheral vascular devices used to treat CLI, keeping in mind the regulatory framework involved in these studies and the specific scientific considerations outlined above. We support the continued conduct and refinement of CLI studies, and would encourage interested study sponsors to discuss potential study designs with FDA in the early planning stages.
Author contributions
References
- Suggested objective performance goals and clinical trial design for evaluating catheter-based treatment of critical limb ischemia. J Vasc Surg. 2009;50:1462–1473
- . Washington, DC: Govt. Printing Office; 1976;43FR 32988.
- Food Drug and Cosmetic Act Amendment. Federal Register 1998;63(225):64617 (21U.S.C.396. Section 906).
- Statistical guidance for clinical trials of non-diagnostic medical devices. http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm106757.htmAccessed on September 3, 2009.
- Least burdensome provisions of the FDA Modernization Act of 1997: concept and principles; final guidance for FDA and industry. http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm085994.htmAccessed on September 3, 2009.
- 21 CFR 860.7 (c)(2) - Medical Device Classification Procedures. Washington, DC, 2008.
Competition of interest: none.
The views and opinions are those of the authors and do not necessarily reflect those of the U.S. Food and Drug Administration, the U.S. Department of Health and Human Services, or the Public Health Service.
The editors and reviewers of this article have no relevant financial relationships to disclose per the JVS policy that requires reviewers to decline review of any manuscript for which they may have a competition of interest.
PII: S0741-5214(09)01935-1
doi:10.1016/j.jvs.2009.09.045
Published by Elsevier Inc.
