Under-representation of women and ethnic minorities in vascular surgery randomized controlled trials
Article Outline
- Abstract
- Methods
- Results
- Discussion
- Conclusions
- Author contributions
- Supplementary data
- References
- Copyright
Objectives
Gender and ethnicity are factors affecting the incidence and severity of vascular disease as well as subsequent treatment outcomes. Although well studied in other fields, balanced enrollment of patients with relevant demographic characteristics in vascular surgery randomized controlled trials (RCTs) is not well known. This study describes the reporting of gender and ethnicity data in vascular surgery RCTs and analyzes whether these studies adequately represent our diverse patient population.
Methods
We conducted a retrospective review of United States-based RCTs from 1983 through 2007 for three broadly defined vascular procedures: aortic aneurysm repair (AAR), carotid revascularization (CR), and lower extremity revascularization (LER). Included studies were examined for gender and ethnicity data, study parameters, funding source, and geographic region. The Nationwide Inpatient Sample (NIS) database was analyzed to obtain group-specific procedure frequency as an estimate of procedure frequency in the general population.
Results
We reviewed 77 studies, and 52 met our inclusion criteria. Only 85% reported gender, and 21% reported ethnicity. Reporting of ethnicity was strongly associated with larger (>280 participants), multicenter, government-funded trials (P < .001 for all). Women are disproportionately under-represented in RCTs for all procedure categories (AAR, 9.0% vs 21.5%; CR, 30.0% vs 42.9%; LER, 22.4% vs 41.3%). Minorities are under-represented in AAR studies (6.0% vs 10.7%) and CR studies (6.9% vs 9.5%) but are over-represented in LER studies (26.0% vs 21.8%, P < .001 for all).
Conclusions
Minority ethnicity and female gender are under-reported and under-represented in vascular surgery RCTs, particularly in small, non-government-funded and single-center trials. The generalizability of some trial results may not be applicable to these populations. Greater effort to enroll a balanced study population in RCTs may yield more broadly applicable results.
Evidence-based medicine has revolutionized health care with the increased use of rigorous scientific analysis to address clinical questions. As a result, greater appreciation of the relative strengths of different types of clinical evidence has also affected the pursuit of new clinical analyses. Properly designed and conducted randomized controlled trials (RCTs) are considered the highest level of evidence (Level I) available to guide clinical practice. The strength of the RCT lies in the randomization of patients to treatment options, because randomization controls the effect of known and unknown confounders. As with all clinical studies, one limitation of the RCT is that results may not be applicable to patients who are under-represented in the study sample.
Disparities exist between gender and between race/ethnicity with regard to the prevalence, presentation, and outcome of vascular disease. There is evidence that African Americans (AAs) and Hispanic Americans (HAs), for example, have a higher prevalence of cardiovascular risk factors, including hypertension, diabetes,1, 2, 3, 4 and peripheral arterial disease (PAD) as measured by the ankle-brachial index compared with whites.1, 5, 6 These disparities also extend to disease severity and treatment outcomes. AAs and HAs have more severe PAD as well as subsequent worse outcomes of bypass grafting and greater need for amputation compared with whites.3, 4, 7 Furthermore, AAs have higher risk-adjusted mortality after cardiac and vascular procedures, including coronary artery bypass grafting, aortic aneurysm repair (AAR), and carotid endarterectomy compared with HAs and whites.8 Gender differences in presentation and outcome of vascular disease have also been demonstrated, with women having a higher ratio of emergency AAR than men,9 and higher mortality rates, longer length of stay, and increased discharge to a nursing facility after both elective and emergency AAR.10 Both women and AAs have been shown to experience lower graft patency after lower extremity revascularization (LER).11
Clinical studies should account for these gender and ethnicity disparities in vascular disease. Enrollment of a representative patient population in RCTs is important to broaden the generalizability of the results. Other specialties, particularly medical oncology, have documented under-representation of women and minorities in RCTs.12, 13 However, little information is available to assess whether the study populations in vascular surgery in RCTs are representative of the patients at risk for vascular disease.
The disparity in enrollment into studies and trials by gender and ethnicity was addressed in the National Institutes of Health (NIH) Revitalization Act of 1993, titled, “Women and Minorities as Subjects in Clinical Research.”14, 15 This act mandated representative inclusion of women and minorities as subjects in clinical research. It further specified that, particularly in diseases where pathophysiology or treatment may differ depending on gender or ethnicity, women and ethnic minorities must be adequately represented to detect differences in end points. Practically speaking, this act significantly affects clinical research through its implications for trial design, subject recruitment, and study documentation. Implicit in these mandates are complete reporting of gender and ethnicity in all clinical research, even if adequate representation cannot be achieved.
We hypothesized that there is disproportionately low enrollment of women and ethnic minorities in RCTs for the treatment of vascular disease in the United States (U.S.). To evaluate our hypothesis, we initiated a retrospective analysis of published RCTs for three common vascular surgery procedures: AAR, carotid revascularization (CR), and LER. We analyzed the reporting of gender and ethnicity in these RCTs and compared the enrolled population mix with a contemporary estimate of procedure frequency based on a nationwide inpatient care database.
Methods
Study design
In August 2007 we conducted an electronic search of RCTs published during the last 25 years using the Medline and Cochrane Clinical Trials databases. Our three categories of vascular procedure (AAR, CR and LER) were intended to be broadly inclusive to maximize the number of clinically relevant studies.
The Medline database was searched for articles published in English between 1983 and August 2007 using a combined keyword and medical subject heading (MeSH) strategy. The term randomized controlled trial was inclusively paired (and) with an array of search terms related to the three vascular procedures. For AAR this included aortic aneurysm, thoracic aneurysm, and abdominal aneurysm. For CR this included carotid, carotid stenosis, and carotid endarterectomy. For LER this included claudication, peripheral vascular disease, critical limb ischemia, and lower extremity bypass.
Similarly, the Cochrane Randomized Controlled Trial Database was searched for studies conducted between 1983 and 2007 that included the keywords carotid, aortic, or lower extremity. Finally, six Cochrane Review articles generated by a generalized Cochrane Database search of these terms were reviewed for relevant RCTs (Appendix 1, online only).
The resulting articles were individually screened to include only U.S. RCTs that included a vascular surgical or endovascular procedure as a component of the trial design. Studies that contained multiple procedures were included in the pooled analysis of gender and ethnicity reporting; however, these multiple procedure trials only included procedures typically performed by a vascular surgeon. For example, a trial involving LER and AAR would be included, but a trial involving AAR and aortic root repair would be excluded.
All included studies were evaluated for the reporting of the gender and ethnicity of their participants. Studies that reported gender and ethnicity were further analyzed for the proportion of women and minority race/ethnicity participants enrolled. The number enrolled, the funding source of the research, the number of centers enrolling participants, and the publication year of each study was also recorded. For the purposes of our evaluation, “ethnic minority” was defined as an enrolled patient other than a “white” or “non-Hispanic white” as reported in each study.
Nationwide Inpatient Sample database
The Nationwide Inpatient Sample (NIS) database is the largest all-payer inpatient care database publicly available in the United States, It contains data from 5 to 8 million hospital stays from approximately 1000 hospitals representing a 20% stratified sample of U.S. hospitals. The 2004 and 2005 NIS databases (the two most current years available) were queried for combined diagnoses and procedures for each of the three categories of AAR, CR, and LER. The patient sample with a pertinent code for each diagnosis (aortic aneurysm, carotid stenosis, PAD) was paired with an appropriate procedure code for AAR, CR, or LER. As a result, only patients with a diagnosis of aortic aneurysm, carotid stenosis, or PAD who underwent treatment for that diagnosis were included in the data set (Appendix 2, online only).
The resulting raw 20% patient sample for each procedure category was subsequently stratified by gender and race/ethnicity, yielding a relative frequency estimate for both categories. These frequency estimates served as the expected demographic ratios and were then compared with the demographic data from RCTs that reported demographic data for gender or race/ethnicity, or both. The NIS data collection and stratification was completed using SAS 9.1 software (SAS Institute Inc, Cary, NC).
Statistical analysis
The reporting of gender or race/ethnicity, or both, was evaluated as a binomial variable using the Fisher exact test to evaluate the association between demographic reporting and study characteristics. These characteristics include publication year, number of centers enrolling participants (single-center or multicenter), funding source, and study size. Funding sources were dichotomized between government-funded studies such as National Institutes of Health and Veterans Affairs (VA), and non-government-funded studies by industry, private institutions, and nonspecified sources. Study size was dichotomized into “large” and “small” subgroups according to enrollment above or below the mean enrollment number (280 patients) among all trials. Studies that reported enrollment data by gender or race/ethnicity, or both, were further evaluated by comparison with the procedure frequency by gender and race/ethnicity, as estimated from the NIS database using the Fisher exact test. An α ≤ 0.05, corresponding to a P ≤ .05, was considered statistically significant. All statistical analyses were performed using SAS 9.1 software.
Results
The initial literature search yielded 77 studies that were individually reviewed for project inclusion. Of these, 25 were excluded because of geographic (outside the United States) and surgical procedural characteristics (nonvascular surgical procedures), leaving 52 studies for our final analysis. These studies were divided by category of diagnosis (11 AAR, 18 CR, 13 LER, and 10 multiprocedure) and evaluated for the reporting of gender and ethnicity in the study samples (Appendix 3, online only). The 52 studies randomized 14,604 participants. Each RCT enrolled a mean of 280 individuals (median, 100; range, 9-2226). Four studies enrolled >1000 individuals. Included studies were published between 1983 and 2007, with a median publication year of 1998 and mode for publication year of 1997 (9 studies).
Gender and ethnicity reporting
The randomized sample found 44 of 52 studies (85%) had reported the gender of participants. The 12 government-funded studies and the 12 large studies (≥280 subjects) reported gender in their demographics. Both of these study characteristics showed a trend toward significance (P = .17) when compared with non-government-funded studies and small studies (<280 subjects), respectively. No significant reporting differences were found between single center (N = 17) or multicenter trials. Because the NIH mandated inclusion of race and gender data starting in 1994, the publication year was also evaluated for differences of gender reporting. This demonstrated no difference between the frequency of gender reporting before or after 1994 (P > .99, N = 41 for post-1994 studies). To account for a delay between trial enrollment and publication, gender reporting was also compared in five separate analyses using years 1995 through 1999. Likewise, no significant difference in reporting was found for any of the publication years evaluated.
Race or ethnicity was reported in 11 of 52 studies. Reporting of ethnicity was associated with large, multi-center trials and government-funded trials (P < .001 for all). Ethnicity reporting was not associated with publication year of the study for any single year between 1994 and 1999. Aggregate reporting, enrollment, and estimated procedure frequency data are summarized in Table I, Table II.
Table I. Data summary of randomized controlled trials that reported and enrollment by gender and ethnicity
| Procedure category | Studies | Reporting gender, No. (%) | Reporting ethnicity, No. (%) | Subjects, No. | Designated gender No. (%) | Women, No. (% of total designated) | Designated ethnicity, No. (%) | Minorities, No. (% of total designated) |
|---|---|---|---|---|---|---|---|---|
| AAR | 11 | 9 (81.8) | 1(9.1) | 1744 | 1669(95.7) | 151(9.0) | 1136(65.1) | 68(6.0) |
| CR | 18 | 16(88.9) | 4(22.2) | 6576 | 6353(96.6) | 1907(30.0) | 4400(66.9) | 305(6.9) |
| LER | 13 | 10(76.9) | 5(38.5) | 5044 | 4654(92.3) | 1044(22.4) | 2979(59.1) | 776(26.0) |
| Multi | 10 | 9(90.0) | 1(10.0) | 1239 | 1057(85.3) | 156(14.8) | 99(8.0) | 14(14.1) |
| Total | 52 | 44(84.6) | 11(21.1) | 14603 | 13,733(94.0) | 3258(23.7) | 8614(59.0) | 1163(13.5) |
Table II. Data summary of Nationwide Inpatient Sample (2004-2005) procedure estimates by gender and ethnicity
| Procedure category | Procedures, No. | Designated gender, No. (%) | Women, No. (% of total designated) | Designated ethnicity, No. (%) | Minorities, No. (% of total designated) |
|---|---|---|---|---|---|
| AAR | 14,288 | 14,267(99.9) | 3069(21.5) | 10,676(74.7) | 1138(10.7) |
| CR | 48,397 | 48,344(99.9) | 20,745(42.9) | 35,357(73.0) | 3354(9.5) |
| LER | 18,556 | 18,534(99.9) | 7666(41.3) | 13,502(72.8) | 2941(21.8) |
| Multiple | NA | NA | NA | NA | NA |
| Total | 81,241 | 81,145(99.9) | 31,480(38.8) | 59,535(73.3) | 7433(12.5) |
Enrollment comparisons
RCTs of only one procedure (N = 42) and also reporting gender (N = 35) or ethnicity (N = 10), or both, were further analyzed for representative enrollment by comparison with nationwide procedural rates from the NIS database. According to the NIS, women comprised approximately 21.5% of AAR, 42.9% of CR, and 41.3% of LER procedures in 2004 to 2005. In contrast, the RCTs showed women represented 9.0% of AAR, 30% of CR, and 22.4% of LER participants. In each of these procedure categories, women were significantly under-represented in RCTs compared with the estimated procedural incidence in the population (P < .001 for all, Fig, A).
Fig.
Proportion of participants enrolled in vascular surgery randomized controlled trials (RCT) compared with estimated proportion of patients receiving procedures from the Nationwide Inpatient Sample (NIS) for (A) female gender and (B) minority ethnicity. **P < .001 by χ2.
In the NIS, minorities comprised approximately 10.7% of AAR, 9.5% of CR, and 21.8% of LER procedures in 2004 to 2005. In contrast, minorities represented 6.0% of AAR, 6.9% of CR, and 26.0% of LER subjects in RCTs. In AAR and CR studies, minorities were significantly under-represented (P < .001). Studies of LER enrolled a higher proportion of minorities compared with the estimated procedure frequency (P < .001; Fig, B).
Six of the single-procedure RCTs in our study were from the VA health care system (1 for AAR, 2 for CR, and 3 for LER). These studies tended to be larger in enrollment (median, 598; range, 189-1136), and all reported gender demographic data. Because of the inherent gender imbalance in the VA population, we performed a secondary gender enrollment analyses with these VA RCTs excluded. This demonstrated female enrollment of 26.4% in AAR studies vs 21.5% of AAR procedures estimated from NIS, 33.3% in CR studies vs 42.9% of CR procedures, and 37.0% in LER studies vs 41.3% of LER procedures. These comparisons demonstrate that non-VA RCTs of AAR enrolled significantly more women than the estimated procedure frequency from the NIS (P = .008), but women remained under-represented in non-VA studies of CR and LER (P = .0001) even though the gap between enrollment and estimated procedure frequency narrowed markedly. The exclusion of VA-based trials from the evaluation of minority race/ethnicity enrollment was also performed and demonstrated similar percentages to procedure-specific enrollment in all of the available studies related to CR and LER. Exclusion of VA-based trials from the AAR studies left us with no studies that reported race/ethnicity demographic data.
Discussion
Vascular surgeons treat a diverse patient population, and Level I evidence derived from RCTs guides the standard of surgical care based on the representative sampling of that population. This study shows that complete demographic reporting of gender and race/ethnicity is absent from a large portion of the vascular surgery RCTs conducted in the United States. Complete demographic reporting, particularly for ethnicity, was associated with larger, multicenter or government-funded trials, or both. There does not appear to be significant increases in reporting over time, particularly in the years after the NIH Revitalization Act of 1993. Among studies reporting gender and ethnicity data, women were significantly under-represented in RCTs for AAR, LER, and CR procedures, whereas ethnic minorities were under-represented in RCTs involving AAR and CR.
The under-reporting and under-representation of women and minorities occurs in a broad array of clinical research across multiple specialties. Even in specialties with known health disparities between gender and ethnicity, such as in diabetes, cardiac disease, HIV, and cancer, there is persistent under-reporting of complete demographic data in clinical research.16 There is also a demonstrated under-representation of women and minorities in clinical trials and in statistical models aimed at better understanding the disparities in disease prevalence and outcomes.17 Oddone et al18 evaluated trials containing an invasive arm and demonstrated that most enrolled fewer minority patients than expected. Interestingly, this same study showed that most trials for diseases with disproportionate minority prevalence (diabetes, hypertension, end-stage renal disease) enrolled more minorities than expected.18 As we have noted, PAD is more prevalent in AAs and HAs. As such, our study corroborates the Oddone findings in RCTs of LER, where minority enrollment exceeded estimated procedure frequency. A potential explanation for this over-representation may be that many RCTs occur at large academic institutions that already care for a larger number of minority patients. Thus, the pool of trial participants is already over-represented in these institutions.
Significant consideration has been given to the causes of under-representation of minorities and women in RCTs. One long-held belief is that ethnic minorities are less willing to participate in human and clinical research, attributed to a mistrust of the medical and scientific community stemming from the Tuskegee syphilis and other similar experiments that have been widely publicized.19 This notion has become increasingly controversial, however, because multiple recent studies have demonstrated equivalent willingness of individuals to participate in clinical trials regardless of ethnicity.20 Despite this, reports indicate that minorities are more likely to believe that they had been previously treated in a trial without their consent.21 This suggests a persistent element of differing expectations and mistrust that needs to be overcome.
In contrast, women, particularly those of child-bearing age, have a long history of being excluded from RCTs. Historically, this partly stems from a desire to limit the exposure of women of child-bearing age to experimental treatments.22 However, with the recognition that gender is an important variable in the manifestation of disease and in the response to treatment, there has been significant effort toward balanced enrollment by gender into clinical trials. That there remains a persistent disparity in representation of women in a broad spectrum of clinical trials suggests the interplay of multiple subtle factors, both in society and in clinical trial design and execution.23
This study has several limitations. First, race/ethnicity is self-reported in RCTs and in the NIS database. Furthermore, race/ethnicity is a complex topic, with significant contributions from the fields of genetics, sociology, anthropology, and history. Not all minorities share similar backgrounds, making broad categorization difficult, especially considering immigration patterns in the United States. Despite these inherent difficulties in defining race/ethnicity, studies should at least try to report this important demographic in some manner. Presumably, gender is a more straightforward demographic variable to evaluate; yet, many studies do not even report these data. The studies not reporting gender or ethnicity may possibly have over-representation of these groups, and thus, the overall RCT enrollment distribution may be appropriate. However, when important demographic information is not available, no meaningful conclusions can be made.
Second, appropriate trial design may necessitate exclusion of portions of the population that have significant minority or gender representation. For example, end-stage renal disease is frequently a component of the exclusion criteria of vascular surgery RCTs and it disproportionately affects AAs.24, 25 As such, the goal of developing accurate end point measurement may have the unintended, and potentially unavoidable, consequence of under-representing specific patient populations.
Third, enrollment results are not entirely under the control of researchers. Very few RCTs can draw enrollment from a nationally representative patient population. Even for multicenter studies, the local institutions conducting the trial may have distinct referral patterns. In particular, the VA patient population is not representative of the U.S. population in that it under-represents women. However, many high-quality vascular surgery RCTs that guide clinical decision making come from VA collaborations. Our subanalysis that excluded VA studies demonstrated attenuated disparities in enrollment. A more balanced incorporation of the VA population into the main analysis would require access to VA population and procedural data that were not available to us. Furthermore, with the increasing military enlistment of women and minorities in recent decades, the current veteran population may be more demographically similar to the entire U.S. population than in previous times.
Finally, the limited numbers of studies available in each category do not lend themselves to more sophisticated statistical measures. Because many of the parameters examined were strongly co-associated (eg, large trials and multicenter trials consisted of overlapping groups), multivariate logistic regression did not yield interpretable results and were therefore left out of this analysis.
Conclusions
Despite these limitations, this study clearly demonstrates under-reporting of gender and race/ethnicity as well as disparities in the enrollment of women and minorities in vascular surgery RCTs. This has significant implications for the interpretation and generalizability of these studies to the overall population. Recognition of these limitations may help surgeons assimilate other studies with better representation for their patient population. We do not imply that the disparities in reporting and enrollment are intentional; rather, patient concerns, unrecognized bias, and local population demographics likely contribute to these disparities. Nevertheless, this issue is highlighted to promote compliance to NIH regulations in future studies.
Further, in the interest of better understanding our diverse patient population, we believe these regulations should extend to all vascular surgery clinical studies, not just those funded by the NIH. With this recognition, researchers can focus on identifying and improving barriers to balanced recruitment and retention appropriate for their local environment. Such strategies can include focused recruitment and education about clinical research ethics and safeguards, recruitment of institutions with intrinsic minority representation into multi-center trials, and recruitment goals/caps determined by estimates of disease incidence or procedural frequency.
In summary, gender and ethnicity demographics are under-reported in RCTs of AAR, CR, and LER. Among studies with adequate reporting, disparities exist between the observed distribution of enrolled patients and expected enrollment as determined by national procedural frequencies. These findings affect the generalizability of RCT results and should serve as a stimulus to better comply with NIH clinical research guidelines.
Author contributions
Supplementary data
Appendix online only.
References
- . Ethnic differences in peripheral arterial disease in the NHLBI Genetic Epidemiology Network of Arteriopathy (GENOA) study. Vasc Med. 2003;8:237–242
- . Drug development and the importance of ethnicity: lessons from heart failure management and implications for hypertension. Curr Pharm Des. 2004;10:3569–3577
- Comparative analysis of autogenous infrainguinal bypass grafts in African Americans and Caucasians: the association of race with graft function and limb salvage. J Vasc Surg. 2005;42:695–701
- Disparities in the treatment and outcomes of vascular disease in Hispanic patients. J Vasc Surg. 2007;46:971–978
- Ethnicity and peripheral arterial disease: the San Diego Population Study. Circulation. 2005;112:2703–2707
- Ethnic-specific prevalence of peripheral arterial disease in the United States. Am J Prev Med. 2007;32:328–333
- . Racial differences in primary and repeat lower extremity amputation: results from a multihospital study. J Vasc Surg. 2005;41:823–829
- . Impact of hospital volume on racial disparities in cardiovascular procedure mortality. J Am Coll Cardiol. 2006;47:417–424
- . Increasing incidence of ruptured abdominal aortic aneurysm: a population-based study. J Vasc Surg. 2006;44:237–243
- . A decade of change in abdominal aortic aneurysm repair in the United States: Have we improved outcomes equally between men and women?. J Vasc Surg. 2006;43:230–238
- Disparity in outcomes of surgical revascularization for limb salvage. race and gender are synergistic determinants of vein graft failure and limb loss. Circulation. 2009;119:123–130
- . Representation of the elderly, women, and minorities in heart failure clinical trials. Arch Intern Med. 2002;162:1682–1688
- . Participation in cancer clinical trials: race-, sex-, and age-based disparities. JAMA. 2004;291:2720–2726
- . NIH guidelines on the inclusion of women and minorities as subjects in clinical research. Federal Register. 1994;59:14508–14513
- Inclusion of women and minorities in clinical trials and the NIH Revitalization Act of 1993--the perspective of NIH clinical trialists. Control Clin Trials. 1995;16:277–285discussion 286-9, 293-309
- . Adequacy of reporting race/ethnicity in clinical trials in areas of health disparities. J Clin Epidemiol. 2003;56:416–420
- . Adherence to federal guidelines for reporting of sex and race/ethnicity in clinical trials. J Womens Health (Larchmt). 2006;15:1123–1131
- Enrollment in clinical trials according to patients race: experience from the VA Cooperative Studies Program (1975-2000). Control Clin Trials. 2004;25:378–387
- . African Americans' participation in clinical research: importance, barriers, and solutions. Am J Surg. 2007;193:32–39
- Are racial and ethnic minorities less willing to participate in health research?. PLoS Med. 2006;3:e19
- The Tuskegee Legacy Project: Willingness of minorities to participate in biomedical research. J Health Care Poor Underserved. 2006;17:698–715
- . Trials of women. Lancet. 1994;343:745–746
- . Females in clinical studies: where are we going?. Clin Pharmacol Ther. 2007;81:600–602
- . The excess incidence of diabetic end-stage renal disease among blacks (A population-based study of potential explanatory factors). JAMA. 1992;268:3079–3084
- . Does racial variation in risk factors explain black-white differences in the incidence of hypertensive end-stage renal disease?. Arch Intern Med. 1991;151:1359–1364
This work was supported by a National Institutes of Health (NIH) K23 Research Career Development Award (HL084386) to Dr Nguyen.
Competition of interest: none.
Additional material for this article may be found online at www.jvascsurg.org.
PII: S0741-5214(09)00018-4
doi:10.1016/j.jvs.2009.01.012
© 2009 Society for Vascular Surgery. Published by Elsevier Inc. All rights reserved.
