Learning curve analysis of thoracic endovascular aortic repair in relation to credentialing guidelines

Methods 

Our vascular surgery database was reviewed retrospectively to identify all patients who underwent elective or emergent endovascular repair of thoracic aortic lesions by one of three vascular surgeons at our university affiliated medical center between 1998 and 2006. Demographic, procedural, and outcome data were identified and reported on an intent to treat basis. No patients were excluded from analysis.

The main outcome variable was primary technical success as defined by the Ad Hoc Committee for Standardized Reporting Practices of The Society for Vascular Surgery/American Association for Vascular Surgery.8 Briefly, primary technical success includes: successful access to the arterial system using a remote site, successful deployment of the endograft with secure proximal and distal fixation without type I or III endoleaks, and a patent endograft without significant kinks, stenoses, or obstruction on completion arteriograms.

The cumulative sum failure method (CUSUM) was then used to analyze each surgeon’s learning curve. This form of analysis permits the identification of improved or suboptimal results earlier than other statistical tools by recognizing the importance of time and experience as a clinical variable.7, 8, 9 Each surgeon’s patients were plotted consecutively using the following formula: Sn = Σ(Xi−Xo), where Xi = 0 for primary technical success and Xi = 1 for technical failure. A target, or acceptable, rate (Xo) of primary technical success was set at 95% based on the EUROSTAR and United Kingdom Thoracic Endograft Registry experience.11 Analysis of the resulting curves identifies the patient number at which the plot adopts a generally negative slope corresponding to the number of patients following which improved outcomes, compared with the target primary technical success rate, can be expected. Additionally, upper 80% alert and 95% alarm boundary lines and lower 80% reassurance lines were calculated using the target success rate of 95% and an alternative rate of 90% indicative of the level at which one wishes to be alerted if the success rate dropped to this level. The formula used to calculate these boundary lines are listed in the Appendix.

Group means (and standard deviations) and proportions were compared using unpaired t tests and Fisher exact tests with InStat version 3.06 (GraphPad Software, San Diego, Calif) and a P < .05 level of statistical significance. This study received approval from the University of Western Ontario Research Ethics Board for Health Sciences Research Involving Human Subjects.

Results 

During this 8-year period, 67 patients underwent endovascular repair of various thoracic aortic pathologies by one of three vascular surgeons. All repairs were performed in the operating room, under general anesthesia, using portable C-arm fluoroscopy. The most common presentation was asymptomatic thoracic aneurysms (46.3%), while similar numbers of patients were treated for ruptured thoracic aneurysms (16.4%), acute thoracic aortic dissections or penetrating ulcers (14.9%), and blunt thoracic aortic trauma (22.4%). Patients were treated with either Talent (Medtronic, Santa Rosa, Calif) or Zenith (Cook, Bloomington, Ind) thoracic endografts. This information is summarized in Table I. As previously reported, the 30-day, or in hospital, mortality rate was 10.4% (0% for elective cases and 19.4% for emergent cases), and postoperative paraplegia occurred in two patients (3%).12 There were no conversions to open repair and all deaths were the result of multisystem organ failure and not attributed directly to the endovascular repair itself.

Table I. Patient demographics and pathology (N = 67)

	Characteristic	% (Number)
	Mean age (years)	65(range: 20-90)
	Male gender	68.6(46)
	Elective thoracic aneurysms	46.3(31)
	Ruptured thoracic aneurysms	16.4(11)
	Aortic dissection or ulcer	14.9(10)
	Traumatic thoracic aortic injury	22.4(15)
	Talent	59.7(40)
	Zenith	40.3(27)

Each of the three surgeons treated 27, 23, and 17 patients, respectively with primary technical success, as previously defined,8 achieved in 92.5% of the entire patient cohort. The various thoracic aortic pathologies treated by the three surgeons are listed in Table II. Although there was a trend towards a difference in the proportion of different pathologies treated by each surgeon, this did not reach statistical significance (P = .06). Additionally, the rates of primary technical success (92.6%, 91.3%, 94.1%) were similar between surgeons (P = .9) with failure, as defined by the reporting standards, occurring in five patients. Reasons for failure included the presence of a type I endoleak on completion angiography in four cases, and in one case, persistent perfusion of the false lumen following endovascular repair of an acute type B thoracic aortic dissection.

Table II. Surgeon specific data

		Surgeon A	Surgeon B	Surgeon C	P value
	Cases	27	23	17
	Pathology:				.06
	TAA	18	7	6
	RTAA	3	2	6
	AD/AU	3	5	2
	BTAI	3	9	3
	Primary technical success (%)	92.6	91.3	94.1	.9

TAA, Elective thoracic aneurysm; RTAA, ruptured thoracic aneurysm; AD/AU, acute aortic dissection or penetrating aortic ulcer; BTAI, blunt thoracic aortic injury.

Cumulative sum failure analysis of each surgeon’s cases was performed and the results are summarized in the figure. These plots include upper 80% “alert” and 95% “alarm” lines and a lower 80% “reassurance” line constructed using the formulae described in the Appendix. The target success rate was set at 95% and derived from a review of European registry data.11 The upper boundary lines represent an unacceptably high failure rate, and the lower boundary line indicates improved results compared with the target success rate (95%). In the figure, each surgeon’s cases were plotted with the slope of the resulting curves being important during analysis. A flat or horizontal plot indicates satisfactory outcomes compared with the target success rate. A positively-sloped plot indicates an unacceptably high rate of failure compared with the target success rate, while a negatively sloped plot indicates improved results with respect to the target rate.

View Large Image Download to PowerPoint

Fig. 1. Cumulative sum failure plot of endovascular thoracic repairs.

In the figure, each surgeon’s cases are plotted sequentially. Initially the resulting curves remain horizontal with little deviation towards the boundary lines. All three plots then adopt a gradually negative slope following 5 to 10 cases, which indicates that improved results compared with the 95% target success rate are beginning to be achieved. In other words, although acceptable results, with respect to the achievement of primary technical success, are accomplished throughout each surgeon’s experience improved results compared with the target rate do not occur until 5 to 10 patients are treated.

Discussion 

The thoracic aorta poses several obstacles to endovascular surgeons that are not as prominent in the abdominal aorta. The wider caliber aorta has necessitated larger endografts and delivery systems that initially might be ill equipped to traverse the aortic arch and provide secure and durable fixation. Additionally, thoracic endografts are subjected to forces and stresses not seen in the abdominal aorta that can make accurate deployment and secure fixation problematic.13 In contrast to the abdominal aorta, where aneurysms are the primary pathology treated endovascularly, thoracic aortic lesions are more variable and include aneurysms, traumatic injuries, and dissections, each having its own unique features.1

Despite this hostile environment, favorable outcomes have been achieved and were facilitated by technological advances in endograft and delivery system design. Also, surgeons realized that experience with peripheral interventions and abdominal endografts is necessary to achieve optimal patient care and have considered this during the training and credentialing processes. In recognition of these requirements, two credentialing documents for thoracic endovascular repair were recently published by different groups of national societies.3, 4 Both sets of guidelines identify requirements for knowledge and experience regarding thoracic aortic pathologies, catheter based skills, endovascular abdominal aortic aneurysm repair, and adjunctive surgical skills including iliac, femoral and arch vessel exposures and repairs. Important differences between these documents include the requirement for open thoracic aortic surgery experience in the STS/AATS4 guidelines, and the case volume requirements for thoracic endovascular repairs (5 by the STS/AATS4, and 10 by the SVS led multispecialty consensus3). Additionally, as described previously, the SVS document includes more stringent guidelines regarding interventional experience and larger case volumes regarding abdominal endovascular repairs. Therefore, it is important to note that the present series involves surgeons whose interventional experience approximates the SVS guidelines, which greatly exceeds that recommended in the STS/AATS document.

It seems reasonable that surgical outcomes will improve with experience, but it is unclear as to whether 5 or 10 endovascular thoracic cases are sufficient to allow for optimal results. A learning curve analysis, which takes time and experience into consideration, is needed to further clarify this volume-outcome relationship. CUSUM analysis is just such an instrument that can be applied to any dataset that can be converted to a “success” or “failure” format and for which there is a target rate of performance.9 It repeatedly applies a sequential probability test and can identify results that deviate from expected prior to traditional post hoc analyses. Our group has had experience in applying this analytical tool to ruptured14 and elective10 open aneurysm repair and endovascular abdominal aortic aneurysm repair7 where 60 cases were initially determined to be necessary to achieve optimal results, although this learning curve can be shortened depending on the center’s overall experience.15 In the present study, “success” was defined as the achievement of primary technical success, as defined by the SVS Reporting Standards,8 while “failure” was the inability to do so. The target rate of success was set at 95% after reviewing large registry data. Specifically, the EUROSTAR and UK Thoracic Endograft Registry’s include information regarding 443 patients who underwent thoracic endovascular procedures.11 Primary technical success for thoracic aneurysms, dissections, and traumatic injuries was reported to be 87.1%, 88.6%, and 96%, respectively. As a result, a 95% target success rate was selected for this study.

Our group has extensive experience with endovascular aortic repairs allowing each surgeon to exceed both credentialing guidelines’ requirements including knowledge of thoracic aortic pathologies, experience with open repair and adjunctive surgical procedures, catheter and guidewire experience, and a large number of endovascular abdominal aortic aneurysm repairs. With this background, we attempted to analyze each surgeon’s learning curve with endovascular repair of thoracic aortic lesions with primary technical success designated as the main outcome variable. In the entire cohort, success was achieved in 92.5% of cases, which compares favorably with other series.11, 16 When each surgeon’s cases were plotted sequentially (Fig), the outcomes remained within an acceptable range as determined by the target success rate (95%). Failures occurred early in each surgeon’s experience, and improved results with respect to the target success rate were achieved following the completion of 5 to 10 cases. This serves to validate the catheter and thoracic endovascular case volume requirements of the SVS,3 while it is unclear whether these results could have been achieved with the less stringent STS/AATS4 credentialing guidelines.

The use of the CUSUM method in this series of thoracic aortic cases posed several challenges that were not apparent with previous applications with abdominal aortic cases.7, 10, 14, 15 Patients present with a variety of thoracic aortic pathologies, both emergent and elective, whereas those with abdominal aortic aneurysms tend to be more uniform with respect to pathology and urgency of treatment. The case mix reported in the present study was fairly consistent between the surgeons and when compared with the experience at other centers.11, 16 However, this range of pathologies results in a somewhat artificial situation when we try to apply one target success rate and one definition of success to the entire cohort. For instance, one would expect higher rates of primary technical success with more focal pathologies, such as blunt thoracic aortic injuries and aortic ulcers, where there would only be a small risk of endoleaks.17, 18 Anatomy and pathology are only two factors that contribute to operative risk with others including comorbidities and elective or emergent operative status. To consider these preoperative risk factors, a risk-adjusted model is necessary, and although, we have applied such models to elective19 and ruptured20 open aortic aneurysm repair, it remains an area of investigation regarding endovascular repairs and the thoracic aorta. The number of cases in the present study is not sufficient to permit a learning curve analysis concerning the role of different endografts, emergent versus elective repairs, and different thoracic aortic pathologies. Although a larger number of cases would allow for a more robust analyses, the cumulative sum failure method is hindered less by small numbers than the more commonly used statistical methods. Despite relatively small numbers, the CUSUM method allows for earlier recognition of improved or poorer results compared with a target rate of performance than is possible with standard post hoc analytical instruments.7, 9, 10, 14, 15

Despite these shortcomings, our surgeons’ experiences are typical of what one would expect in today’s endovascular era. This review confirms that with a sound background in catheter and guidewire manipulations and endovascular aortic surgery, optimal results can be obtained with endovascular thoracic aortic repair following the completion of 5 to 10 cases. This serves to validate the peripheral intervention and thoracic aorta case volume requirements of the recent SVS credentialing guidelines document.

Author contributions