Premarket assessment of devices for treatment of critical limb ischemia: The role of Objective Performance Criteria and Goals
Article Outline
- Abstract
- Defining performance goals for critical limb ischemia
- Differentiating performance goals from practice guidelines
- Conclusion
- Author contributions
- References
- Copyright
Medical devices are cleared for marketing approval through the Food and Drug Administration (FDA). Unique statutory requirements, such as the “least burdensome mandate,” have allowed the FDA to employ non-concurrent controls in its evaluation of prospective therapies. The use of Objective Performance Criteria and Goals (OPC and OPG) for the premarket evaluation of cardiovascular devices has become established as an alternative to randomized, controlled trials (RCTs). These single-armed comparisons may facilitate rapid entry of novel devices to the market. Unlike RCTs, they do not establish superiority or non-inferiority of the examined therapy, and study populations must be carefully inspected to ensure validity of comparisons to historical controls.
In their accompanying commentary, representatives of the Food and Drug Administration (FDA) Center for Devices and Radiological Health provide a concise overview of current methods of premarket assessment of medical device safety and efficacy. Kumar et al. note that randomized, controlled trials (RCTs) provide the highest scientific standard by which the FDA evaluates the entry of new drugs, devices, or other therapeutics into the marketplace, yet also acknowledge the financial, temporal, and subject recruitment challenges that accompany the design and implementation of RCTs.
The FDA's approach to regulatory processes is dictated by the statutory mandate under which the agency operates. In keeping with its statutory guidance, the FDA does not require proof of superiority of a proposed device to each similar existing device or therapy, nor does it seek to regulate the practice of medicine. Rather, the agency provides a timely premarket process to assure that proposed medical devices meet reasonable safety and efficacy benchmarks. Central to the execution of this mission is the following clause from the FDA Modernization Act of 1997: “The Secretary shall consider, in consultation with the applicant, the least burdensome appropriate means of evaluating device effectiveness that would have a reasonable likelihood of resulting in approval.” The “least burdensome” concept was subsequently defined as a successful means of addressing a premarket issue that involves the most appropriate investment of time, effort, and resources on the part of industry and FDA.1 The FDA is thus given the leeway to consider alternatives to randomized, controlled trials, while maintaining sound scientific practice.
The use of non-concurrent controls in well-defined clinical circumstances has therefore found acceptance within the regulatory framework of the FDA.2 The cardiac valve literature demonstrates the successful adaptation of historical datasets to formulate Objective Performance Criteria (OPC) for new devices.3 Established OPC permit comparison of new devices to validated historical controls in single-armed trials, and thus expedite the entry of innovative devices into the medical arena. When historic controls are less robust in nature, or are gathered from alternate treatment methodologies, Objective Performance Goals (OPGs) may be suggested, and developed into OPC as the supporting dataset is strengthened over time.4
Lastly, not all data collection occurs prior to marketing approval. Postmarket data collection provides valuable information regarding long-term outcomes, and may reveal unanticipated complications or late events that require further analysis.
Defining performance goals for critical limb ischemia
Critical limb ischemia (CLI) is a complex process, anatomically multilevel in nature, which affects several distinct high-risk subpopulations. The number, morphology, and anatomic location of infrainguinal lesions treated in CLI patients may differ dramatically from patient to patient. Moving from the traditional arterial-segment marketing approval to disease-state marketing approval places a stronger focus on patient-centered outcomes (survival, limb salvage, and burden of reinterventions) without abandoning the traditional surgical focus on patency of the treated vessel. The accompanying manuscript by Conte et al. summarizes the recommendations of the Society for Vascular Surgery (SVS) Working Group on performance goals for CLI. As noted previously, promulgation of these performance goals aims to benefit patients by providing a timely and cost-effective means of device evaluation. It is hoped that these goals will also be useful for the evaluation of non-implanted devices for CLI intervention, and biologic therapeutics for CLI treatment.
In addition to the patient benefits summarized above, avoidance of the expense and time required for randomized studies provides a clear financial incentive for medical device manufacturers to pursue single-armed trials. However, adoption of the OPG format is not without risk. Among the real-world concerns of the authors is that single-armed trials may allow subtle bias in patient selection to mask treatment futility. The “clinical high-risk (age over 80/tissue loss)” subgroup in the accompanying article represents a population that may well benefit from endovascular treatment. However, if investigators shy away from enrolling those patients with more substantial degrees of tissue loss (eg, Rutherford stage 6), the resulting cohort will outperform expectations, thus potentially validating a device that would not have demonstrated noninferiority in a randomized, controlled trial format. Therefore, ongoing collaborative efforts will focus on formulating a statistically valid approach to risk-adjusting the OPG targets for a given trial, based on characteristics of the enrolled study population. Post-enrollment adjustment of performance goals would enable future study sponsors to recruit reasonably heterogenous study populations, and yet still qualify for comparison to the historical surgical controls.
Accumulation of robust outcomes data for angioplasty-based CLI treatment should remain a priority. The accompanying SVS Working Group manuscript offers detailed recommendations for standardized trial design and outcomes assessment. Utilizing these guidelines, compilation of high-quality angioplasty data may yet be accomplished, and may permit development of complementary, angioplasty-based performance goals for CLI treatment. The OPG suggested by the SVS Working Group should be viewed as a starting point, with the expectation of further refinement over time by the accumulation of additional data from CLI trials.
Differentiating performance goals from practice guidelines
The proposed performance goals for CLI devices are the product of an intensive analysis of available historic controls from published randomized controlled trials. Although the resultant OPGs are based upon rigorous statistical evaluation, it is crucial that they not be mistaken for practice guidelines. The narrowly defined constructs of performance goals exist solely for the facilitation of medical device approval. Attainment of the OPG standards implies that an acceptable level of device safety and efficacy has been demonstrated, and thus permits introduction of the candidate device into the realm of clinical usage. It remains for robust clinical investigation, including RCTs, to subsequently provide appropriate scientific evidence to guide clinical decision-making and appropriate utilization.
In contrast to the carefully limited function of performance goals, practice guidelines seek to bring a data-driven sensibility to the expansive clinical scope of modern vascular surgery. The Society for Vascular Surgery maintains several committees charged with the exposition of practice guidelines for a variety of topics, such as thoracic aortic disease, infrarenal aortoiliac aneurysmal disease, lower extremity arterial disease, arterio-venous hemodialysis access, and venous disease. These writing groups evaluate the broad range of available clinical literature on the topic of interest, and aid the practicing surgeon by refining that evidence into succinct recommendations for clinical care. Strength of clinical recommendations and quality of the supporting evidence are graded on a standardized scale, with recognition that many areas will require further research before definitive clinical guidance can be provided.
Conclusion
The FDA plays a critical role in premarket assessment and approval of new medical devices, yet physician and societal participation in this process remains essential. Working in consultation with members of the FDA, the CLI Performance Goals Task Force has provided a carefully validated set of safety and efficacy criteria for evaluation of new treatment modalities. Broad adoption of the accompanying recommendations for CLI trial design will facilitate data sharing between investigator groups, and contribute to the periodic refinement of these performance criteria.
OPC and OPG do not supplant the role of properly conducted RCT in discerning superiority between different therapies, and should not be misinterpreted as practice recommendations. Although the FDA, by statute, does not consider cost of medical devices, this factor looms large in the deliberations of the Agency for Healthcare Research and Quality, the Centers for Medicare and Medicaid Services, private payers, and practicing physicians and surgeons. These entities may require a more rigorous proof of efficacy than that offered by single-armed comparisons. Moreover, the scope and emphasis of FDA device scrutiny, although defined by statute, remains open to reasoned debate. How shall we assess the safety and efficacy of combined pharmacologic and mechanical interventions? Combined biologic and mechanical interventions? Ongoing collaboration between the FDA, industry, and practicing physicians is needed to safely employ medical innovation for the benefit of all patients.
Author contributions
References
- The Least Burdensome Provisions of the FDA Modernization Act of 1997: Concept and Principles; Final Guidance for FDA and Industry. http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm085994.htmAccessed June 15, 2009
- . FDA perspective on clinical trial design for cardiovascular devices. Ann Thorac Surg. 2006;82:773–775
- . Prosthetic heart valves: Objective Performance Criteria versus randomized clinical trial. Ann Thorac Surg. 2006;82:776–780
- . FDA perspective on clinical trial design for femoropopliteal stent correction of peripheral vascular insufficiency cardiovascular devices. Catheter Cardiovasc Interv. 2007;69:920–921
Competition of interest: Dr Geraghty has received consulting fees from W. L. Gore and Cook Medical. Dr Matsumura has received research support, consulting fees, and/or training director fees from Abbott, Bard, Cook Medical, Cordis, Ev3, W. L. Gore, Lumen, and Medtronic.
The editors and reviewers of this article have no relevant financial relationships to disclose per the JVS policy that requires reviewers to decline review of any manuscript for which they may have a competition of interest.
PII: S0741-5214(09)02091-6
doi:10.1016/j.jvs.2009.10.041
© 2009 Society for Vascular Surgery. Published by Elsevier Inc. All rights reserved.
