| | The Glasgow Aneurysm Score as a tool to predict 30-day and 2-year mortality in the patients from the Dutch Randomized Endovascular Aneurysm Management trialReceived 22 June 2007; accepted 10 October 2007. ObjectiveRandomized trials have shown that endovascular repair (EVAR) of an abdominal aortic aneurysm (AAA) has a lower perioperative mortality than conventional open repair (OR). However, this initial survival advantage disappears after 1 year. To make EVAR cost-effective, patient selection should be improved. The Glasgow Aneurysm Score (GAS) estimates preoperative risk profiles that predict perioperative outcomes after OR. It was recently shown to predict perioperative and long-term mortality after EVAR as well. Here, we applied the GAS to patients from the Dutch Randomized Endovascular Aneurysm Repair (DREAM) trial and compared the applicability of the GAS between open repair and EVAR. MethodsA multicenter, randomized trial was conducted to compare OR with EVAR in 345 AAA patients. The GAS was calculated (age + [7 points for myocardial disease] + [10 points for cerebrovascular disease] + [14 points for renal disease]). Optimal cutoff values were determined, and test characteristics for 30-day and 2-year mortality were computed. ResultsThe mean GAS was 74.7 ± 9.3 for OR patients and 75.9 ± 9.7 for EVAR patients. Two EVAR patients and eight OR patients died ≤30 days postoperatively. The area under the receiver-operator characteristic curve (AUC) was 0.79 for OR patients and 0.87 for EVAR patients. The optimal GAS cutoff value was 75.5 for OR and 86.5 for EVAR. By 2 years postoperatively, 18 patients had died in both the EVAR and the OR patient groups. The AUC was 0.74 for OR patients and 0.78 for EVAR patients. The optimal GAS cutoff value was 74.5 for OR and 77.5 for EVAR. ConclusionThis is the first evaluation of the GAS in a randomized trial comparing AAA patients treated with OR and EVAR. The GAS can be used for prediction of 30-day and 2-year mortality in both OR and EVAR, but in patients that are suitable for both procedures, it is a better predictor for EVAR than for OR patients. In this study, the GAS was most valuable in identifying low-risk patients but not very useful for the identification of the small number of high-risk patients. Abdominal aortic aneurysms (AAAs) are considered for elective repair when they exceed the 55-mm threshold, above which the risk of rupture is considered higher than the mortality risk of surgery.1 Obviously, operative mortality is higher in patients with severe comorbidity. Endovascular aneurysm repair (EVAR) was originally designed for patients that are unfit to undergo invasive open repair (OR). However, because of the initial positive results, EVAR was quickly implemented in the clinical routine and was also frequently used for low-risk patients. A drawback of the EVAR treatment is the relatively frequent occurrence of complications and reinterventions that arise due to incomplete exclusion of the AAA.2 Furthermore, the devices are expensive and therefore cost-effectiveness is not obvious.3 Another disadvantage is the need for lifelong follow-up and the amounts of contrast and radiation involved. The Dutch Randomized Endovascular Aneurysm Management (DREAM) trial and the British EVAR-1 trial were initiated to determine which procedure is superior.4, 5 The trials had very similar perioperative and mid-term results. Although EVAR has an initial survival advantage,6, 7 this benefit disappears after 1 year, and the 2-year survival rate is approximately 90% for both procedures.8, 9 The late EVAR deaths were mainly related to cardiovascular comorbid conditions. It can be assumed that some AAA patients are better off with no surgery at all because of their high-risk profile. The EVAR-2 trial was designed to compare the outcome of EVAR vs an expectant policy in AAA patients who were unfit for the conventional OR treatment.10 In summary, after 4 years, based on intention-to-treat analysis, there was no difference in all-cause mortality, aneurysm-related mortality, or quality of life measures. The value of this trial has been criticized because it lacked an adequate definition of “unfit for surgery.” It therefore remains a big challenge to clearly define and identify the high-risk AAA patient group that presumably is better off with no surgery at all. Scoring systems are designed to predict the risk of specific events in individual patients. The Glasgow Aneurysm Score (GAS) is a simple prediction rule for AAA patients to determine their preoperative risk profile.11 Age and the presence of cardiac, cerebrovascular, renal disease, and shock are used to classify a patient as high or low risk. The GAS was originally designed to predict perioperative mortality and morbidity after OR, but a recent report from the European Collaborators on Stent-graft Techniques for Abdominal Aortic Aneurysm Repair (EuroSTAR) showed that the GAS may also be a good predictor of both perioperative and long-term results after EVAR treatment.12 Although the DREAM trial patients were considered to be suitable for both open and endovascular repair, this decision was not based on standardized risk-stratification systems. Consequently, it is conceivable that patients were included in the DREAM trial who actually had an elevated risk of dying in the first 2 years after surgery based on a scoring system like the GAS. This would make the indication for AAA repair questionable in these patients. Here, we applied the GAS system to the AAA patients from the DREAM trial to compare the predictive performance of the GAS of 30-day and 2-year mortality between randomized OR and EVAR patients. Methods  Patients The design and methods of the trial have previously been described in detail.4 In brief, patients with an AAA of ≥50 mm, and considered suitable for both operative methods, were referred to 24 centers in the Netherlands and four centers in Belgium. After obtaining a written informed consent, patients were randomly assigned to undergo OR or EVAR. All data were submitted to the trial coordination center (Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, the Netherlands). Follow-up visits were scheduled 30 days and 6, 12, 18, and 24 months after the procedure. The study was performed according to the Declaration of Helsinki, and the institutional review board of each participating hospital approved the protocol. Glasgow Aneurysm Score The GAS was calculated using the comorbidities described by Samy et al11: risk score = age + (7 points for myocardial disease) + (10 points for cerebrovascular disease) + (14 points for renal disease). The original GAS also adds 17 points for shock; however, because this analysis was done in elective patients, this did not apply to our study. Myocardial disease was defined as previously documented myocardial infarction or ongoing angina pectoris, or both. Cerebrovascular disease was defined as all grades of stroke including transient ischemic attack. Renal disease was defined as a serum creatinine level >150 μmol/L or a creatinine clearance <50 mL/min, or a history of acute or chronic renal failure, or both. The elements of the GAS were prospectively collected. Data analysis The association between the GAS and 30-day and 2-year mortality after OR and EVAR was analyzed using the χ2 test. To study the performance of the GAS, we assessed its calibration and discrimination. Calibration, or goodness of fit, refers to the agreement in the individual patients between the predicted risks as assigned by the prediction rule and the actual observed frequencies of mortality. This was evaluated with the Hosmer-Lemeshow test, where a significant test results indicates poor model fit. Discrimination is the ability to assign higher probabilities of the event (death) to the patients who will actually die in the observation period than to those patients who will survive. This was quantified using the area under the receiver-operator characteristic (ROC) curve (AUC). A value between 0.7 and 0.8 indicates acceptable discrimination, and a value >0.8 indicates excellent discrimination. From the ROC curve, we estimated the threshold that yielded the most optimal combination of high true-positives (sensitivity) and low false-negatives (1-specificity). For this analysis, we looked for the highest possible sensitivity while not accepting a specificity of <50%. Statistics from two-by-two tables were calculated using the Diagnostic and Agreement Statistics (DAG-Stat) calculator. All other statistical analysis was performed using SPSS 11 software (SPSS Inc, Chicago, Ill). P < .05 was considered statistically significant. Results Of the 351 randomized patients, two died before treatment (1 from a ruptured AAA before undergoing OR and 1 from a pneumonia before undergoing EVAR), and four refused treatment (3 were originally assigned to OR and 1 was originally assigned to EVAR). The remaining 345 patients composed the treatment groups: 174 OR patients and 171 EVAR patients. The medium GAS of OR patients was 74.7 ± 9.3 and of EVAR patients, 75.9 ± 9.7 (Table I). The 30-day outcome is known for all patients; however, in the subsequent 2 years, six patients were lost to follow-up (5 OR and 1 EVAR). The 2-year mortality rates therefore are based on 169 OR and 170 EVAR patients. Thirty-day mortality Two EVAR patients and eight OR patients died ≤30 days postoperatively (Table II, A). Univariate analysis of the total groups showed that the GAS was significantly associated with an increased 30-day mortality in OR patients (P < .003), but not in EVAR patients (P = .10), most likely due to the low number of events. The Hosmer-Lemeshow test was not significant for OR (P = .07) or EVAR (P = .55), indicating good calibration. The ROC AUC was 0.79 for OR patients and 0.87 for EVAR patients (Fig 1). The optimal GAS cutoff value for OR patients was 75.5, with a sensitivity of 75.0% (95% confidence interval [CI], 34.9%-96.8%) and a specificity of 54.2% (95% CI, 46.3%-62.0%; Table II, B). For EVAR patients, the optimal cut off value was 86.5, with a sensitivity of 100% (95% CI, 15.8%-100%) and a specificity of 85.2% (95% CI, 78.9%-90.3%; Table II, B). Using these thresholds, both treatments have a high negative-predictive value but a low positive-predictive value. | | |  | OR 30-day | GAS <75.5 | GAS>75.5 | Total | EVAR 30-day | GAS<86.5 | GAS>86.5 | Total | |
|---|
| Alive | 90 | 76 | 166 | Alive | 144 | 25 | 169 | | | Dead | 2 | 6 | 8 | Dead | 0 | 2 | 2 | | | Total | 92 | 82 | 174 | Total | 144 | 27 | 171 | | | | |
| | | | 30-day mortality | Open repair (95% CI) | EVAR (95% CI) | |
|---|
| Optimal cutoff point | 75.5 | 86.5 | | | Sensitivity, % | 75.0(34.9-96.8) | 100(15.8-100) | | | Specificity, % | 54.2(46.3-62.0) | 85.2(78.9-90.3) | | | Positive-predictive value, % | 7.3(2.7-15.3) | 7.4(0.9-24.3) | | | Negative-predictive value, % | 97.8(92.4-99.7) | 100(97.0-100) | | | Likelihood ratio positive test | 1.6(1.1-2.5) | 6.8(4.7-9.7) | | | Likelihood ratio negative test⁎ | 2.2(0.6-7.3) | DNC | | | | |
Two-year mortality Two years postoperatively, 18 patients had died in both the EVAR and the OR groups (Table III, A). Univariate analysis of both groups showed that the GAS was significantly associated with an increased risk of 2-year mortality in both OR (P = .001) and EVAR (P = .001) patients. The Hosmer-Lemeshow test was not significant for OR (P = .44) or EVAR (P = .58), indicating good calibration. The AUC was 0.74 for OR patients and 0.78 for EVAR patients (Fig 2). The optimal cutoff value for OR patients was 74.5, with a sensitivity of 77.8% (95% CI, 52.4%-93.6%) and a specificity of 52.3% (95% CI, 44.0%-60.5%; Table III, B). For EVAR patients, the optimal cutoff value was 77.5, with a sensitivity of 88.9% (95% CI, 65.3%-98.6%) and a specificity of 61.9% (95% CI, 53.6%-69.6%; Table III, B). As determined by these thresholds, both treatments have a high negative-predictive value, but a low positive-predictive value. | | | | OR 2-year | GAS<74.5 | GAS>74.5 | Total | EVAR 2-year | GAS<77.5 | GAS>77.5 | Total | |
|---|
| Alive | 79 | 72 | 151 | Alive | 94 | 58 | 152 | | | Dead | 4 | 14 | 18 | Dead | 2 | 16 | 18 | | | Total | 83 | 86 | 169 | Total | 96 | 74 | 170 | | | | |
| | | | 2-year mortality | Open repair (95% CI) | EVAR (95% CI) | |
|---|
| Optimal cutoff point | 76.5 | 77.5 | | | Sensitivity, % | 77.8(52.4-93.6) | 88.9(65.3-98.6) | | | Specificity, % | 52.3(44.0-60.5) | 61.9(53.6-69.6) | | | Positive predictive value, % | 16.0(9.2-25.8) | 21.6(12.9-32.7) | | | Negative predictive value, % | 95.2(88.1-98.7) | 97.9(92.7-99.8) | | | Likelihood ratio positive test | 1.6(1.2-2.2) | 2.3(1.8-3.0) | | | Likelihood ratio negative test⁎ | 2.4(1.0-5.6) | 5.6(1.5-20.7) | | | | |
Discussion We studied the performance of the GAS in predicting 30-day and 2-year mortality in AAA patients who were randomized to receive EVAR or OR. Because of the low number of events, especially for 30-day mortality, the 95% CIs are relatively wide for the sensitivity and the positive-predictive value. To obtain smaller CI ranges, data from a larger cohort are needed. The AUCs of this study reveal several important points. First, they show that the GAS is superior in predicting perioperative death in EVAR-treated patients compared with OR-treated patients. This is quite remarkable, because the GAS was originally developed to predict surgical outcomes after the conventional open procedure. Second, the AUCs indicate that the GAS is suitable to predict 2-year mortality for both EVAR- and OR-treated patients. Third, the different calculated optimal cutoff values indicate that for OR patients, this value is similar for determining perioperative and 2-year mortality (75.5 and 74.5). For the EVAR-treated patients, however, there is a clear difference: the optimal cutoff value is 86.5 for perioperative mortality and 77.5 for 2-year mortality. The high cutoff value for predicting perioperative death after EVAR treatment compared with OR is not very surprising and has been shown previously in a study of 5498 patients of the EUROSTAR Registry.12 Strikingly, we found an identical optimal cutoff value for 30-day EVAR mortality (86.5) as the previous study (86.6), which indicates that our study is representative despite the low number of events. Furthermore, we hereby validate the optimal cutoff value reported by the researchers of EUROSTAR. The high cutoff value confirms that this less-invasive procedure results in better operative survival among patients with more severe comorbidity; in other words, in patients with a higher GAS. It also explains why the GAS is superior in predicting perioperative mortality after EVAR compared with OR; only those patients with severe comorbid conditions are at risk of dying after EVAR. After OR, patients with less obvious comorbidity are at risk of dying postoperatively as well, and therefore, the GAS performance in predicting mortality in these patients compared with the EVAR patients is not as good. The observation that the cutoff value drops for the 2-year mortality prediction in EVAR patients to a value equal for predicting 2-year mortality after OR suggests that patients with more comorbid conditions do not benefit from EVAR in the longer term. This was previously shown in both the DREAM and EVAR-1 trials. The AUC remains good, which indicates that by applying the GAS it is possible to distinguish between groups that are at high or low risk to die of comorbid conditions ≤2 years after AAA surgery. The negative-predictive values are high compared with the positive-predictive values. This means that in patients who are suitable for both OR and EVAR, the GAS is better in identifying low-risk patients than in identifying high-risk patients. This was previously shown for OR, but not yet for EVAR.14 It is not surprising that within a population such as the DREAM trial, with a low prevalence of high-risk patients, it is difficult—if not impossible—to accurately identify high-risk patients. Therefore, testing a scoring system for its ability to identify high-risk AAA patients that are better off with no surgery at all should be validated in a large cohort of high-risk patients, like was recently done for the Customized Probability Index15 in the patients of the EVAR-2 trial.16 The likelihood ratios represent the relative diagnostic gain of the application of the GAS in AAA patients. The 30-day mortality is low (1.2% EVAR, 4.6% OR), yet significant likelihood ratios were achieved. The allocation of “high-risk” using the GAS in OR patients does not lead to a big increase in predicted mortality rate: the positive likelihood ratio is 1.6, which translates into an increase of predicted mortality rate from 4.6% (pretest probability) to 7.3% positive posttest probability. In EVAR patients, however, there is a substantial increase: the positive likelihood ratio of 6.8 shifts the expected mortality rate from 1.2% to 7.4% in EVAR patients with a GAS >86.5. The difference in gain of predicted mortality by applying the GAS between OR and EVAR patients is remarkable. The high positive likelihood ratio of 30-day mortality for EVAR patients possibly results from the relatively high impact of the comorbidities on the GAS. But because the cutoff value is high, this comorbidity only starts to play a role in extreme circumstances. Extreme comorbidity identified by the GAS could therefore be a good starting point from which one can discriminate between AAA patients that are suitable for EVAR and those that are better off with no surgery at all. For the patients treated with OR, additional factors not included in the GAS are apparently important in predicting 30-day mortality; therefore, applying the GAS does not induce a major predictive gain here. The invasive surgery also affects patients with less obvious comorbidity. The high negative likelihood ratio for 2-year mortality of EVAR confirms that absence of severe comorbid conditions greatly enhances the probability of survival. For OR patients, this benefit is less pronounced and could be caused by patient selection. Patients with significant comorbidity who survive invasive conventional surgery may be less likely to die than the patients with severe comorbidity who survive the relatively mild EVAR procedure. The GAS is a good predictor of low-risk AAA patients; however, its usefulness for identifying the individual high-risk patient is not obvious. The GAS therefore may be optimized by including more comorbidities, such as pulmonary disease and diabetes, or by more stringent definitions of the comorbidities now included. For instance the scoring system might be improved by using creatinine clearance instead of creatinine levels, and using specific electrocardiographic deviations instead of all “cardiac disease.” However, the major driver of the GAS remains age; and furthermore, a huge benefit of the GAS is its simplicity, which might be lost after optimization. Conclusion We presented the first evaluation, to our knowledge, of the GAS in a randomized trial comparing AAA patients treated with OR or EVAR. We showed that the GAS can be used to predict 30-day and 2-year mortality after both treatment modalities but that its usefulness is superior in EVAR-treated patients. In this study, the GAS was most valuable in identifying low-risk patients but not very useful for the identification of the small number of high-risk patients in both the OR and EVAR patient groups. Author contributions Conception and design: AB, KJ, JB Analysis and interpretation: AB, KJ, JB Data collection: MP Writing the article: AB, KJ, JB Critical revision of the article: EB Final approval of the article: AB, KJ, EB, MP, JB Statistical analysis: AB Obtained funding: JB Overall responsibility: JB References 1. 1The UK Small Aneurysm Trial Participants. Mortality results for randomised controlled trial of early elective surgery or ultrasonographic surveillance for small abdominal aortic aneurysms. Lancet. 1998;352:1649–1655. Abstract | Full Text |
Full-Text PDF (98 KB)
|
CrossRef
2. 2van Marrewijk C, Buth J, Harris PL, Norgren L, Nevelsteen A, Wyatt MG. Significance of endoleaks after endovascular repair of abdominal aortic aneurysms: The EUROSTAR experience. J Vasc Surg. 2002;35:461–473. Abstract | Full Text |
Full-Text PDF (257 KB)
|
CrossRef
3. 3Prinssen M, Buskens E, de Jong SECA, Buth J, Mackaay AJC, van Sambeek MRHM, et al. Cost-effectiveness of conventional and endovascular repair of abdominal aortic aneurysms; results of a randomized trial. J Vasc Surg. 2007;46:883–890. Abstract | Full Text |
Full-Text PDF (483 KB)
|
CrossRef
4. 4Prinssen M, Buskens E, Blankensteijn JD. The Dutch Randomised Endovascular Aneurysm Management (DREAM) trial (Background, design and methods). J Cardiovasc Surg (Torino). 2002;43:379–384. MEDLINE 5. 5Brown LC, Epstein D, Manca A, Beard JD, Powell JT, Greenhalgh RM. The UK Endovascular Aneurysm Repair (EVAR) trials: design, methodology and progress. Eur J Vasc Endovasc Surg. 2004;27:372–381. Abstract | Full Text |
Full-Text PDF (781 KB)
|
CrossRef
6. 6Prinssen M, Verhoeven EL, Buth J, Cuypers PW, van Sambeek MR, Balm R, et al. A randomized trial comparing conventional and endovascular repair of abdominal aortic aneurysms. N Engl J Med. 2004;351:1607–1618.
CrossRef
7. 7Greenhalgh RM, Brown LC, Kwong GP, Powell JT, Thompson SG. Comparison of endovascular aneurysm repair with open repair in patients with abdominal aortic aneurysm (EVAR trial 1), 30-day operative mortality results: randomised controlled trial. Lancet. 2004;364:843–848. Abstract | Full Text |
Full-Text PDF (183 KB)
|
CrossRef
8. 8Blankensteijn JD, de Jong SE, Prinssen M, van der Ham AC, Buth J, van Sterkenburg SM, et al. Two-year outcomes after conventional or endovascular repair of abdominal aortic aneurysms. N Engl J Med. 2005;352:2398–2405.
CrossRef
9. 9Endovascular aneurysm repair versus open repair in patients with abdominal aortic aneurysm (EVAR trial 1): randomised controlled trial. Lancet. 2005;365:2179–2186. Abstract | Full Text |
Full-Text PDF (109 KB)
|
CrossRef
10. 10Endovascular aneurysm repair and outcome in patients unfit for open repair of abdominal aortic aneurysm (EVAR trial 2): randomised controlled trial. Lancet. 2005;365:2187–2192. Abstract | Full Text |
Full-Text PDF (94 KB)
|
CrossRef
11. 11Samy AK, Murray G, MacBain G. Glasgow aneurysm score. Cardiovasc Surg. 1994;2:41–44. MEDLINE 12. 12Biancari F, Hobo R, Luvonen T. Glasgow Aneurysm Score predicts survival after endovascular stenting of abdominal aortic aneurysm in patients from the EUROSTAR registry. Br J Surg. 2006;93:191–194. MEDLINE |
CrossRef
13. 13Obuchowski NA, Lieber ML. Confidence intervals for the receiver operating characteristic area in studies with small samples. Acad Radiol. 1998;5:561–571. Abstract |
Full-Text PDF (952 KB)
|
CrossRef
14. 14Hirzalla O, Emous M, Ubbink DT, Legemate D. External validation of the Glasgow Aneurysm Score to predict outcome in elective open abdominal aortic aneurysm repair. J Vasc Surg. 2006;44:712–716. Abstract | Full Text |
Full-Text PDF (102 KB)
|
CrossRef
15. 15Kertai MD, Boersma E, Klein J, van Sambeek M, Schouten O, van Urk H, et al. Optimizing the prediction of perioperative mortality in vascular surgery by using a customized probability model. Arch Intern Med. 2005;165:898–904. MEDLINE |
CrossRef
16. 16Brown LC, Greenhalgh RM, Howell S, Powell JT, Thompson SG. Patient fitness and survival after abdominal aortic aneurysm repair in patients from the UK EVAR trials. Br J Surg. 2007;94:709–716. MEDLINE |
CrossRef
a Julius Center Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands b Department of Surgery, University Medical Center Utrecht, Utrecht, the Netherlands c Department of Vascular Surgery, Radboud University Nijmegen Medical Center, Nijmegen, the Netherlands.  Reprint requests: Annette Baas, UMC Utrecht, Julius Center, PO Box 85500, 3584 CX Utrecht, the Netherlands.
Competition of interest: none. PII: S0741-5214(07)01616-3 doi:10.1016/j.jvs.2007.10.018 © 2008 The Society for Vascular Surgery. Published by Elsevier Inc. All rights reserved. | |
|
FULL TEXT01616-3/journalimage?img=PIIS0741521407016163.gr1.lrg.gif&fig=fig1&kwhquery=&issn=0741-5214&ishighres=false&allhighres=false&free=yes','journalimage');)
01616-3/journalimage?img=PIIS0741521407016163.gr2.lrg.gif&fig=fig2&kwhquery=&issn=0741-5214&ishighres=false&allhighres=false&free=yes','journalimage');)