A preliminary clinical scale to predict the risk of in-hospital death after carotid endarterectomy
Article Outline
Objective
Carotid endarterectomy (CEA) remains the gold standard for the treatment of carotid disease, with mortality rates generally at 0.4% to 1.7%. Controversy remains with regards to its role in the treatment of the high-risk surgical population. We developed a new clinical scale incorporating weighted risk factors into a single numerical score that correlates with the risk of in-hospital death after CEA. We propose that this tool may serve to prospectively identify the high-risk patient.
Methods
We performed a retrospective analysis of 10 years (1994 to 2003) of the Maryland hospital discharge database. Included in the analysis were patients with (1) International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) procedure code 38.12 (endarterectomy of the vessels of the head and neck other than intracranial vessels) in the primary coding position but not in any secondary position, or (2) Diagnosis Code 433.00 to 433.91 (occlusion/stenosis, precerebral artery), or (3) the Diagnosis-Related Group 5 (extracranial vascular procedure). ICD codes representing preoperative conditions of the patients were identified and evaluated with stepwise regression modeling techniques for association with in-hospital deaths. Different regression models were evaluated and compared by discriminative power as measured by receiver operating characteristics (ROC) and goodness-of-fit to data as measured by r2 and the Hosmer-Lemeshow statistic. A numeric index correlating with the risk of in-hospital death was constructed by rounding the correlation coefficients for the statistically significant variables from the logistic regression.
Results
We identified 23,237 cases. The mean age of patients was 70.6 years, with 54.7% male patients. There were 125 in-hospital deaths (0.54%). Patient age and four patient medical conditions emerged with significant associations with in-hospital deaths after CEA, and their relationships can be summarized in a single diagnostic scale: 1 point for age ≥75, 2 points for atherosclerosis (ICD code 440), 3 points for cardiomyopathy (ICD code 425), 4 points for iron-deficiency anemia (ICD code 280), and 5 points for cerebral degeneration (ICD code 331). This scale has moderate discriminative power (ROC = 0.67). On average, each point increase on this scale is associated with a 1.58-times increase in mortality risk, with score of 6 on the scale carrying a mortality risk >5%.
Conclusions
This new 5-item scale, based on patient age and past medical history, correlates moderately with the rate of in-hospital death after CEA. This clinical index may serve to identify high-risk patients. Future improvements to this diagnostic scale should focus on the diagnostic values of additional laboratory and demographic data.
Over the last five decades, the carotid endarterectomy (CEA) has emerged as the gold standard treatment for carotid artery disease.1, 2, 3 The safety and efficacy of this procedure has clearly been confirmed by completion of the North American Symptomatic Carotid Endarterectomy Trial (NASCET) and Asymptomatic Carotid Atherosclerosis Study (ACAS) trials.4, 5 More recently, carotid angioplasty and stenting has emerged as alternative endovascular treatment for this patient population and has recently been approved for Medicare reimbursement for symptomatic patients considered to be at unacceptably high risk for conventional carotid surgery.6 The definition of high risk has largely been deduced from industry-sponsored clinical trials7, 8; however, this definition has resulted in considerable controversy, at least among the surgical community.
Further research is needed to validate the definition of patients at unacceptably high risk for CEA. A single scale that combines the different risk factors for in-hospital deaths after carotid endarterectomy (CEA) as well as the relative weights and the differential contributions of each of these factors to the patients’ overall risks can be a useful new clinical tool to identify such high-risk patients.
We hypothesized that we could develop a predictive model correlating weighted preoperative factors with the risk of in-hospital mortality, which the vascular literature cites as from 0.4% to 1.7%. Furthermore, based on our interpretation of the literature and understanding of carotid disease, we hypothesized that the following factors would be of statistical importance in the risk equation: age, gender, diabetes mellitus, hypertension, and cardiac disease.
Methods
We performed a retrospective analysis of 10 years (1994 to 2003) of the Maryland hospital discharge database. Included in the analysis were patients with (1) International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) procedure code 38.12 (endarterectomy of the vessels of the head and neck other than intracranial vessels) in the primary coding position but not in any secondary position, or (2) Diagnosis Code 433.00 to 433.91 (occlusion/stenosis, precerebral artery), or (3) the Diagnosis-Related Group (DRG) 5 (extracranial vascular procedure). ICD codes representing preoperative conditions of the patients were identified and evaluated with stepwise regression modeling techniques for association with in-hospital deaths. We examined all ICD-9 codes present within the database; we did not limit our investigation to Charlson diagnosis codes.
Different versions of our new index were evaluated and compared by their discriminative powers as measured by receiver operating characteristics (ROC) and their goodness-of-fit to data as measured by pseudo r2 and the Hosmer-Lemeshow (HL) statistic.9, 10 ROC reflects the ability of the index to differentiate positive events (in this case, in-hospital carotid endarterectomy death) and negative events (no in-hospital death). A higher ROC represents a better diagnostic index. The other important property of a diagnostic index is its goodness-of-fit to data, which was evaluated by pseudo r2 and the HL statistic. Pseudo r2 is analogous to the regular r2 that is calculated for linear regression models and ranges from 0 (none of the variance is explained) to 1 (all variance is explained).11, 12 The HL statistic is another goodness-of-fit measure; a value <15.5 indicates a good logistic fit.12
We used Intercooled Stata, version 8.2 (Stata Corp, College Station, Tex), for data analysis and index-building. This method of index-building was previously published in the trauma literature.9, 10 We then used the discharge dataset from California from 1999 to 2003 (51,331 patients) to validate the index.
Results
We identified 23,237 cases in the Maryland dataset. The mean age of patients was 70.6 years, with 54.7% male patients. There were 125 in-hospital deaths (0.54%).
A total of 582 different ICD-9 integer diagnosis codes were recorded in the Maryland hospital discharge database for these patients, with 48 codes having ≥5% prevalence. From this list, clinical judgment allowed further elimination of diagnosis codes, leaving 34 codes that were deemed obviously preoperative and were included for analysis by stepwise regression (Fig 1).
Fig 1.
Elimination of initial 582 International Classification of Diseases-9th revision (ICD-9) codes down to four statistically significant codes for the prediction of perioperative death after carotid endarterectomy.
Stepwise logistic regressions were performed with entry criteria set at P = .10; that is, variables were initially removed from the model if P > .10. Both forward and backward selection of variables was performed to confirm the selection of variables. Twelve variables emerged from this process; eight of these were then eliminated by further consideration of truly preoperative conditions (Fig 1).
The remaining variables included ICD-9 code 280 (iron deficiency anemias), code 331 (cerebral degeneration), code 440 (atherosclerosis), and code 425 (cardiomyopathy).
syndrome (280.8), and unspecified etiologies (280.9).
We also examined larger groupings of ICD-9 in hopes of increasing the explanatory power of our model. We grouped ICD-9 codes for heart-related and lung-related comorbidities. Neither set of code groups improved our ROC or r2.
We next turned our attention to age as a predictor of in-hospital mortality, reviewing multiple step function models of age. A single step-function at age ≥75 emerged with the best r2 of 0.0087 (P < .001) and an odds ratio (OR) of death of 1.94 (95% confidence interval [CI], 1.37 to 2.76). Gender was next added to the model, yielding no significant association with outcome. Black vs white race was also found to have no significant association.
An initial index was then created by summing the product of each variable with its regression coefficient:
This initial index produced a pseudo r2 of 0.0380, an HL statistic of 2.82, with a ROC of 0.662. Given the cumbersome nature of these coefficients, we next attempted to simplify the index into an integer scale with the goal of easy clinical use, while ensuring that the r2 and ROC diagnostic properties of the scale were preserved. To this end, we began by dividing all coefficients by their least common denominator, which maintained the ratio between the coefficients while converting at least one of them into an integer without decimal points:
Next, we rounded these coefficients to facilitate clinical use and further eliminate decimal points:
Although the rounding changed the ratios between the different coefficients, we believed that the change was minor and insignificant. This was confirmed statistically: this simplified version still retained the goodness of fit (pseudo r2 = 0.0369, HL statistic = 3.52) as well as the discriminatory power (ROC = 0.6642) of the earlier versions. However, as this set of coefficients was not continuous (1, 1, 2, 4, and 5), we returned to version 2 of our index and tried rounding up the coefficients instead to produce a more continuous set of coefficients:
This rounding up of coefficients also did not significantly disturb the diagnostic properties of the index; it too maintained the goodness of fit (pseudo r2 of 0.0381, HL statistic of 2.96) and discriminatory power (ROC of 0.6660) properties of the original statistical model (Fig 2). Thus, the final index:
Fig 2.
Evolution of predictive index. CEA, Carotid endarterectomy; ROC, receiver operating characteristics.
On average, a one-point increase on this scale is associated with a 1.58-times increase in mortality risk, with score of 6 on the scale carrying a mortality risk >5% (Fig 3, Fig 4). Note that no patient with scores >10 was observed in our dataset, even though the theoretical maximum score on our index is 15. The number of observed deaths for a given score correlated well with that predicted by our index (Fig 5, Fig 6).
Fig 3.
Risk of perioperative death vs index score.
Fig 4.
Risk of perioperative death after carotid endarterectomy vs index score.
Fig 5.
Expected vs observed number of perioperative deaths.
Fig 6.
Scatter plot of observed vs expected deaths for index scores.
Finally, we validated our new index with the 51,331-patient discharge data from California from 1999-2003. With the California data, our index still correlated significantly with the likelihood of death (OR, 1.30; P < .001). Its diagnostic properties differed from the Maryland data; the pseudo r2 was only 0.01, but the HL statistic changed to 0.68. The ROC was lower for the California data at 0.55 (99% CI, 0.52 to 0.58). Overall, California patients who died scored significantly higher on our new index than patients who survived, with mean index scores of 1.2 vs 0.92 (P < .01).
Of interest, calendar year emerged with significant association with patient outcome: every increment in calendar year was associated with an OR of death of 0.935 (P = .04). Thus, the overall outcome of carotid endarterectomies improved in Maryland over the study period.
Discussion
In this study, we hypothesized that age, gender, diabetes mellitus, hypertension, and cardiac disease would predict in-hospital death after CEA. After advanced statistical analysis of the 125 deaths within this 10-year, 23,237-patient cohort, with up to 15 ICD-9 codes per patient, we have identified advanced age, atherosclerosis, cardiomyopathy, and iron deficiency anemia as risk factors for postoperative deaths after CEA. We hope our index will facilitate the preoperative informed consent discussions between surgeons and their patients.
Among the factors in this new scale, only age has been heretofore identified as a predictor of in-hospital CEA death. For instance, Saleh and Hannen14 found a death OR of 1.80 for patients aged 75 to 84 years and an OR of 2.26 for those >84 years compared with those <65 years. The study by Miller et al15 approached statistical significance for an age effect: patients ≥80 years undergoing CEA demonstrated 1.9% mortality compared with 0.8% for those <80 (P = .053). Other authors have found no significant association between age and outcome.16, 17, 18, 19, 20 Our scale grants only 1 point to age ≥75 years, reflective of its relatively weak effect on outcome compared with the other variables.
The other variables in our mode—cardiomyopathy, iron deficiency anemia, and cerebral degeneration—are newcomers to the risk assessment for CEA mortality. These comorbidities have not emerged as historical risk factors for CEA in-hospital death previously, most likely because they have not been studied. Although cardiac disease generally is a well-known risk factor for poor outcome in a spectrum of surgical procedures, cardiomyopathy itself has not been commonly named in the literature as a risk factor. However, a study of 1700 forensic autopsies found an association of cardiomyopathy with unexpected in-hospital death. In this study from the anesthesia literature, cardiomyopathy was implicated in 8 of 50 cases of sudden cardiac death in American Society of Anesthesiology (ASA) class I, noncardiac surgery patients.21
We were surprised by the association of iron deficiency anemia with in-hospital CEA death. Of note, code 280 does not include anemia secondary to surgical blood loss but instead encompasses such conditions as anemia secondary to inadequate dietary iron, Plummer-Vinson syndrome, idiopathic hypochromic, asiderotic, and iron deficiency nitric oxide synthase. While anemia secondary to chronic blood loss is included, acute posthemorrhagic anemia is excluded.13 There is no prior documentation in the literature regarding such an association between in-hospital CEA death and iron deficiency anemia. We hypothesize that we may be observing the well-documented mortality effect of in-hospital anemia on coronary disease,22 given the high association of carotid and coronary disease. Of course, this prompts the question as to why coronary artery disease wouldn’t have fallen out of our analysis of risk factors. Alternately, we may have identified anemia as a new risk factor for carotid surgery. Further study is warranted to investigate this association.
Finally, cerebral degeneration weighed most heavily as a risk for in-hospital death. Although it seems logical that demented patients fare worse than their lucid counterparts, this is not documented in the carotid endarterectomy literature. ICD-9 code 331 includes “cerebral degeneration” due to Alzheimer’s, beriberi, senile degeneration, and hydrocephalus, but it also includes that due to vascular disease. So perhaps this variable captures the mortality effect that previous authors have identified as presentation for CEA after transient ischemic attack or stroke, or a history of such cerebrovascular events.2, 20
Of note, we found no association between gender and the probability of in-hospital death. Previous studies have reported conflicting data on the influence of gender on outcome.14, 16, 18, 19, 20, 23, 24
Our study has several strengths that differentiate our index from other reports in the literature. We relied on 10 years of a statewide hospital discharge dataset, and the large, 23,237-sample size available to us dramatically increases the power of our analysis. Our comparative approach mitigates concerns for coding errors, as any errors would have affected both sides of our analysis. Additionally, instead of simply identifying and enumerating the risk factors as other studies have done, we went a step further and combined them into a single index to allow scoring of patients, which can then be used to compare between patients with different sets of risk factors. We used regression weights to assign point values to each variable on the index, a more evidence-based approach than the traditional expert-panel subjective or arbitrary assignment of point values. We have also manipulated the point values so that they can be easily used in clinical practice, aiming to produce a set of numbers that are integers without decimal points and that are easy to memorize, all the while maintaining the diagnostic properties of the scale. This is in contrast to many papers in the literature that simply present their statistical regression models with a cumbersome set of coefficients or scales that rely on expert-panel assignment of point values with unknown diagnostic properties. Finally, we examined and report the diagnostic values of our index, providing a benchmark for future improvements.
We acknowledge several limitations to our study. This scale has only moderate discriminative power (ROC = 0.66).25 As a reference ROC value from the radiology literature, Monnier-Cholley et al26 found an ROC of 0.770 in a series of 60 challenging test cases of missed lung cancers presented to 12 radiologists. Previous estimates of ROC for noncancer chest radiographs have ranged from 0.75 to 0.95 for radiologists.
Additionally, although each of the factors in the index was found to carry statistical significance for association with in-hospital death, the r2 for the overall model remains relatively meager, indicating that the model is limited in its explanatory power. This limitation of the model may be surmountable by the inclusion of more variables, such as demographic data and surgeon and hospital volumes. However, given the absence of reports in the literature (apart from trauma surgery) that mention goodness-of-fit data in relation to regression models and diagnostic properties, it is difficult to gauge whether the r2 of our scale is acceptable or not. Finally, the possibility remains that it is simply not possible to predict CEA mortality with a high degree of certainty in the preoperative setting.
Although our scale has rather a low ROC, r2, and HL statistic at this point, dismissing a test straightaway based on its low diagnostic properties is clinically ill advised. In fact, many existing clinical diagnostic scales possess rather poor diagnostic properties when actually measured. No clinical test is ever used in isolation, however. A string of tests with low diagnostic properties used together in a series can actually produce more discriminatory power than a single highly-diagnostic test used alone.27 Provided that a diagnostic test has at least some discriminatory power (ie, ROC > 0.50), it will contribute to the overall diagnostic evaluation of any patient. Clinicians must consider any diagnostic test, such as this new index, in conjunction with other available information before arriving at clinical decisions.
We also acknowledge limitations in the data that we used. This index was designed only to predict and quantify the risk of in-hospital death. It does not predict 30-day mortality or other long-term outcomes. Additionally, this index does not predict complications, most importantly stroke. Another project is underway to develop such an index, or even a combined index for stroke or death, a common outcome measure in the literature.
The only variable we included in the was patient history, not laboratory work nor the results of radiologic or diagnostic studies, nor socioeconomic information, all of which may be important but are not available in typical hospital discharge datasets. We have also not included systems-level factors that may impact outcome, such as surgeon and hospital volumes. We are currently investigating these factors in a concurrent study.
We are further limited by possible hidden selection bias in our datasets. For example, very few patients with clearly known risk factors for poor outcomes, such as recent myocardial infarction or renal failure, would undergo a CEA. Therefore, the sample size of patients with these clear risk factors in the dataset may be inadequate. As such, these factors may not emerge with significant association with poor patient outcome, even though they are clearly known as risk factors.
Theoretically, our source data may have been affected by coding bias. With the changes in healthcare payment systems over the past 10 years, progressive overcoding may have emerged as hospitals strive to establish medical necessity and increase reimbursement. In fact, the mean number of ICD-9 codes did increase in 2001-2003 compared with the time periods 1994-1997 and 1998-2000. Nonetheless, in our comparative study, historical shifts in coding should affect both outcomes equally, and thus not introduce bias.
Similarly, any trends in coding amongst individual hospitals would not introduce data bias, as one can assume equal coding anomalies for both survivors and deaths. In addition, Maryland adopted a new diagnosis-related reimbursement syndrome (the DRG system) mid-way through the study period. As above, these changes would affect the entire database without introducing bias.
Despite the limitations of this first version of our index, in view of the evidence-based strengths of our analytical approach, this new preliminary scale may be useful in quantifying the risk of in-hospital deaths against the known risk of death from a given degree of carotid stenosis. It may aid in patient selection and in developing strategies for managing patient risks; but certainly cannot serve as a substitute for clinical judgment. Each patient and physician must weigh the acceptable level of risk for a CEA with the risks of alternate therapies or of not having the procedure. Therefore, we hope that this scale will engender more accurate pre-operative risk discussion in the informed consent process.
Conclusion
Carotid endarterectomy is an effective and commonly performed surgical procedure with documented low in-hospital mortality. Identification and management of the factors relating to the risk of mortality associated with this procedure may lead to effective methods for improving its safety. Using a large multipractice database, we have developed and validated a simple and generalizable five-item scale, based on patient age and past medical history, which correlates with the rate of in-hospital death after CEA.
Discussion
Dr William R. Flinn (Baltimore, Md). It has been about 5 years since hospitals in Maryland began a diagnosis-related reimbursement system similar to the DRG system used in the rest of the country. This necessitated that Maryland hospitals append secondary diagnoses to patients’ charts at the time of discharge to achieve appropriate reimbursement. As a result, the data from the first 5 years of your study might be considerably different than for the second half.
Another concerning aspect of your scoring scale would be the impact of anemia, which was the second highest predictor of mortality in your model. This observation is surprising, since CEA is an operation that is usually free of significant blood loss. Could your selection codes have included major brachiocephalic revascularizations and/or combined coronary/CEA where both the blood loss and the risk might have been greater, acting to skew your data?
The impact of your scoring scale on mortality went up exponentially at or above a value of 6; however, few of those studied had a score over 6. Considering the very low mortality overall in your series, would patients with scores under 6 really be influenced by this scoring system? And alternatively, are there actually going to be any people that have scores over 6 that we would realistically consider for surgical treatment?
Nevertheless, on the basis of your study, I think even those of us most resistant to the implantation of a metallic foreign body into the carotid lumen (rather than the elegant removal of the entire atherosclerotic plaque) might agree that in an anemic 80-year-old with endocardial fibroelastosis and cerebral degeneration a carotid stent might be a reasonable consideration.
Dr Susanna L. Matsen. Regarding your first question about the DRG coding, this is a comparative study such that any coding bias would affect all patients equally, both those with perioperative mortality and surviving patients.
Regarding anemia, ICD-9 code 280 is “iron deficiency anemia,” and under that follows a number of different types of anemia. Besides true iron deficiency anemia, also included are anemias secondary to chronic blood loss, anemia secondary to inadequate intake, “unspecified,” and a few other random etiologies such as Plummer-Vinson syndrome. Code 280 does not include anemia secondary to blood loss. In other words, this anemia is likely to be a preoperative condition of the patient. Perhaps this is a surrogate for the effect of anemia on patients with chronic coronary artery disease, which is a well-documented risk factor for mortality.
We also plan to look at the California discharge database, which identifies preoperative risk factors separately from conditions that developed during the hospital stay. This will further confirm that we have identified preoperative anemia as a risk factor, or whether this finding is partially clouded by postoperative anemia.
Regarding the rarity of death: we agree that, fortunately, carotid endarterectomy is a very low-risk procedure in terms of perioperative death. We plan to expand our index to include strokes and perhaps other complications such as myocardial infarction. This will expand the capture rate for our index to make it more useful clinically.
Dr Alan Dardik (New Haven, Conn). I enjoyed your paper and applaud your efforts to try to come up with an index that’s useful. Can you tell us why you started with death? I understand death is very easy to code in a database, but we do carotid endarterectomy to prevent stroke, and I would think that if the scale would predict stroke it would be far more useful to us.
Second, when you were making your algorithm, did you confine your codes to the Charlson and Romano codes that are specific for the preoperative risk factors, or did you take all codes?
And lastly, how should we really use this index? I echo Dr Flinn’s concern. Most of my patients are over 75. They generally all have carotid arthrosclerosis, and I really don’t offer them carotid endarterectomy if they have cerebral degeneration or severe cardiomyopathy.
Dr Matsen. Regarding why we started with death, it was because this is the most objective and measurable outcome.
We are in the process of investigating the preoperative risk factors associated with stroke and other outcomes, as I mentioned before, to make an index that is more clinically useful. This is a preliminary study, and in essence, this is as much a methodology paper as an outcomes paper.
Secondly, you asked about the source of codes. The codes all come from the Maryland state discharge database. These are entered into the database by the coders upon discharge of the patient from the hospital, and so they include a wide range of ICD-9 codes. We analyzed all codes that had a prevalence of greater than 5% in this patient population. We did not limit the analysis to codes on a known comorbidity index such as the Charlson index because we wanted to be open to the possibility that other comorbidities may have an impact on carotid endarterectomy outcomes.
Thirdly, regarding the use of the index: I do believe that we need to do some work in terms of making this a more clinically meaningful index. By adding in the radiographic and laboratory data, we will hopefully increase the predictive value of the index. Quite honestly, however, the ultimate point of this project may be that it is not possible to satisfactorily predict death after this operation, since it is such an infrequent occurrence. So either we’re going to be able to fine-tune this into a tool with a better ROC and better r2, or it may be that there are factors influencing perioperative mortality in this clinical setting which we simply cannot measure accurately, even by analyzing a large administrative database.
Dr Sean D. O’Donnell (Washington, DC). Dr Matsen, I’d like to ask you what your thoughts are on how justifiable it is to use this type of database for outcome studies like you did. There are a number of papers that come up using similar discharge coding databases and state databases, but are they ultimately designed to answer the questions you’re trying to answer? And is this type of database the right place to be going to answer those questions?
Dr Matsen. Clearly, any outcome analysis will only be as good as the data that was entered. The strength of our study is that it includes such a large number of patients, 23,000 cases, and the long duration of the study period. However, we are limited by the accuracy of the data as in analyzing any administrative database.
One of the limitations I can see from this data was alluded to by Dr Flinn; namely, the potential for overcoding bias. When reimbursement is enhanced by an increased comorbidity and increased complications, this is always a potentially confounding influence. I think that this is something that we need to acknowledge and try to control for in such analyses in the future.
References
- . Carotid endarterectomy (the “gold standard” in the endovascular era) . J Am Coll Surg . 2002;194:S2–S8
- . Carotid endarterectomy remains the standard of care, even in high-risk surgical patients . Ann Surg . 2005;241:356–363
- . Carotid endarterectomy remains the gold standard . Am J Surg . 2002;183:580–583
- Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators . N Engl J Med . 1991;325:445–453
- Endarterectomy for asymptomatic carotid artery stenosis. Executive Committee for the Asymptomatic Carotid Atherosclerosis Study . JAMA . 1995;273:1421–1428
- Centers for Medicare & Medicaid Services. March 17, 2005. www.cms.hhs.gov/media/press/release.asp?Counter=1391.
- Protected carotid-artery stenting versus endarterectomy in high-risk patients . New Engl J Med . 2004;351:1493–1501
- Wholey M. ARCHeR: Acculink for Revascularization of Carotids in High-Risk Patients. Preliminary 30-day results. Presented March 2003, American College of Cardiology National Meeting.
- . The tip of the iceberg for child abuse (the critical roles of the pediatric trauma service and its registry) . J Trauma . 2004;57(6):1189–1198
- . The multi-institutional validation of the new Screening Index for Physical Child Abuse (SIPCA) . J Pediatr Surg . 2005;40:114–119 discussion 119
- Improving the Glasgow Coma Scale score (motor score alone is a better predictor) . J Trauma . 2003;54:671–678
- A comparison of the abilities of nine scoring algorithms in predicting mortality . J Trauma . 2002;53:621–628
- Hripcsak G. www.dmi.columbia.edu/hripcsak/icd9. Online ICD-9 Index.
- . Carotid endarterectomy utilization and mortality in 10 states . Am J Surg . 2004;187:14–19
- . Carotid endarterectomy in octogenarians (does increased age indicate “high risk?”) . J Vasc Surg . 2005;41:231–237
- Predicting complications of carotid endarterectomy . Stroke . 1993;24:1285–1291
- . Influence of age and hospital volume on the results of carotid endarterectomy (A statewide analysis of 9918 cases) . J Vasc Surg . 1998;27:25–33
- Preoperative risk factors for carotid endarterectomy (defining the patient at high risk) . J Vasc Surg . 2003;37:1191–1199
- . Risk factors for death or stroke after carotid endarterectomy. Observations from the Ontario Carotid Endarterectomy Registry . Stroke . 2003;34:2568–2575
- . Clinical and radiographic risk factors for operative stroke and death in the European Carotid Surgery Trial . Eur J Vasc Endovasc Surg . 2002;23:108–116
- Unsuspected cardiac lesions associated with sudden unexpected perioperative death . Eur J Anaesthesiol . 2000;17:230–235
- Prevalence of risk factors, and not gender per se, determines short- and long-term survival after coronary artery bypass surgery . J Cardiothorac Vasc Anesth . 2003;17:585–593
- . Influence of patient-related variables on the outcome of carotid endarterectomy . J Vasc Surg . 1996;24:120–126
- . Epidemiology of carotid endarterectomies among Medicare beneficiaries . J Vasc Surg . 1992;16:201–208
- . The meaning and use of the area under a receiver operating characteristic (ROC) curve . Radiology . 1982;143:29–36
- . Detection of lung cancer on radiographs (receiver operating characteristic analyses of radiologists’, pulmonologists’, and anesthesiologists’ performance) . Radiology . 2004;233:799–805
- . Why clinicians are natural bayesians . BMJ . 2005;330:1080–1083
Competition of interest: none.
PII: S0741-5214(05)01254-1
doi:10.1016/j.jvs.2005.08.005
© 2005 The Society for Vascular Surgery. Published by Elsevier Inc. All rights reserved.
