How can professional competence be measured—and possibly even enhanced—after formal training has been completed and board certification has been attained in a specialty field of medicine or surgery? In response to a growing interest in this question on the part of the public and payers alike, a task force appointed in 1998 by the American Board of Medical Specialties (ABMS) has reached several important conclusions that promise to influence the future recertification process for every medical and surgical discipline in the United States.1 The most fundamental of these conclusions is that the competence implied by board certification should be maintained by a lifelong commitment to learning and self-assessment, not merely be “recertified” on the basis of a passing grade on a written examination every decade or so. Although such examinations undoubtedly will continue to play a role in the maintenance of certification by the 10 surgical boards that are represented on the ABMS, the current directive to these boards is to find a meaningful way for diplomates to demonstrate their competence by blending a sustained program of specialty-specific education with an ongoing appraisal of their own surgical results.
The Directors of the American Board of Surgery (ABS) held a retreat in January 2000 to discuss this mandate with representatives from the ABMS, the Accreditation Council for Graduate Medical Education, the American College of Surgeons, the employee health plan of the Ford Motor Company, and two groups with previous experience in the collection of surgical outcome data (the Veterans Administration National Quality Improvement Program and the Northern New England Cardiovascular Study Group). During this retreat, the ABS endorsed the ABMS concept for the maintenance of certification and subsequently adopted some basic principles to guide the implementation of its new responsibilities in the area of surgical outcomes. Perhaps most important, the ABS has emphasized that its plans for diplomates to provide outcome data in conjunction with the maintenance of their ABS certification are meant merely to comply with the ABMS directive to all of its member boards and have no hidden or punitive implications. Furthermore, the ABS has indicated that the data it will request from its diplomates will concentrate on short-term outcomes that should be relatively easy to collect and are intended to be used for self-assessment to achieve the best possible clinical results. Individual surgeon data will be held in absolute confidentiality and will be reported back to each diplomate at regular intervals, probably with a quartile comparison with the results of all other participants in the same specialty area. Finally, the ABS is currently exploring the possibility of a partnership with the American College of Surgeons in an effort to link some of the educational activities of the College as closely as possible to the outcome measures that will be requested from diplomates.
This change in the traditional format for recertification will be of immediate interest to surgeons who hold Certificates of Special or Added Qualifications in Vascular Surgery (formerly General Vascular Surgery). Concerned that General Surgery represents such a wide variety of surgical procedures that data for this field would be difficult to collate until some preliminary experience with subspecialties has been obtained, the ABS has elected to begin its initiative to collect outcome data through its Board (formerly Sub-Board) of Vascular Surgery, its Pediatric Surgery Sub-Board, and its Surgical Oncology Advisory Council. An operational meeting was held at the Philadelphia office of the ABS in January 2001 to determine the procedures and outcome indicators that will be used for the data collection in vascular surgery. This meeting was attended by an invited ad hoc committee (G. Patrick Clagett, Blair A. Keagy, Joseph L. Mills, James M. Seeger, Jonathan B. Towne, and the author) and Wallace P. Ritchie, Jr, and Robert S. Rhodes of the ABS executive staff. As the result of this meeting, these early end points have been selected as a means of measuring outcomes for several index procedures in vascular surgery: 30-day mortality rates for open and endovascular repair of nonruptured infrarenal aortic aneurysms; 30-day mortality and stroke rates for carotid endarterectomy in patients with and without symptoms; and 30-day mortality and amputation rates for infrainguinal bypass grafting (vein and synthetic) in patients with claudication, advanced ischemia, or both. Severity indexing beyond the symptom status and the procedure description was discussed, but was thought to be impractical at present.
So far, so good—at least in the sense that the most influential certifying groups in the United States now have taken the position, with which few can argue, that the acquisition of new knowledge and a dispassionate awareness of personal clinical results are perhaps the most reliable indicators of continuing physician competence in any specialty field. Vascular surgery does lend itself to the type of data collection planned by the ABS because of the clarity of its complication end points (death, stroke, and amputation) and because, with the present exception of endovascular aortic aneurysm repair, most certified vascular surgeons generally perform all the signature procedures that have been selected for early outcome assessment. Good intentions alone are not enough to ensure the success of the ABMS initiative, even in vascular surgery, however, and the ABS is aware that a number of questions undoubtedly will be asked by the diplomates who are expected to participate in it. A few examples follow.
How will “self-reported” outcome data be collected and validated? The data that are to be submitted to the ABS have to be reliable, because they will determine the credibility of the entire outcomes project. The overall ABS database (ie, the “denominator” that is necessary for self-assessment) will never be entirely dependable unless all the submitted data are verified in some unbiased and relatively uniform manner to avoid the inaccuracies that could be caused by incomplete compliance or simple oversights on the part of surgeons who have not previously had to report their 30-day complication rates. There are some potential remedies to these problems that need to be considered by the ABS for its pilot project in vascular surgery. First, in-hospital mortality and stroke rates are rarely different from 30-day outcomes, and they are much easier to document. (Amputations after failed attempts at infrainguinal revascularization will be the most difficult end points to track because they can be performed days, weeks, or months later, or they may never become necessary at all.) Second, outcome data should be gathered prospectively to immunize them from retrospective guesswork. Collectively, these two considerations support the notion that hospitals should assume the responsibility for the ongoing collection and validation of surgical results, as was recommended by Moore et al2 in their guidelines for hospital privileging in vascular surgery more than 10 years ago. Hospital administrators and chiefs of services had better get used to this idea if the ABMS seriously intends for all medical and surgical boards to measure outcomes in the future. As an opening gambit, the ABS has initiated contact with the Joint Commission for the Accreditation of Healthcare Organizations in an attempt to enlist its cooperation in this matter.
Why has general surgery not yet been included in the ABS outcomes project? As already mentioned, the ABS maintains that it is reluctant to address the ABMS mandate in general surgery without the benefit of at least some experience in specialty fields like vascular surgery, pediatric surgery, and surgical oncology. The heterogeneity of the average general surgical practice was one of the principal findings of a study of 2434 applicants for ABS recertification in general surgery from 1995 to 1997, especially in rural areas where general surgeons perform a larger number of endoscopic procedures and a wider variety of gynecologic, genitourinary, and orthopedic operations than those in midsized communities or urban centers.3 Because of this heterogeneity, the ABS is concerned that it might be difficult to select a group of index procedures for outcome measurement that would be appropriate to the practice patterns of even a plurality—much less the majority—of its diplomates in general surgery. Nevertheless, the ABS appears to recognize that its dilemma with general surgery must soon be resolved, not only to satisfy the ABMS, but also to reassure vascular surgeons, pediatric surgeons, and surgical oncologists that its commitment to the importance of self-assessment truly is universal.
What happens next? The ABS currently intends to conduct a pilot study in which sample data will be requested from representative members of the three surgical specialties that have been chosen to measure outcomes. This field test should help to determine the availability of 30-day postoperative results, and it will allow the ABS to construct the “firewalls of confidentiality” that it wants to have in place before outcome assessment actually is implemented as one of the criteria for the maintenance of certification (Wallace P. Ritchie, Jr, written communication, Mar 15, 2001). So many details have yet to be decided that it may take a while before vascular surgeons see anything resembling a final policy in this matter.
If the project being planned by the ABS for the Board of Vascular Surgery can be further legitimized by an accurate database, it will represent one more step in the direction of hospital audits and surgeon self-assessment that the Society for Vascular Surgery and the American Association for Vascular Surgery (formerly the International Society for Cardiovascular Surgery, North American Chapter) officially have taken for quite some time.2, 4, 5 As an example that this approach actually can be correlated with an improvement in surgical outcome, Kresowik et al6 found that the combined stroke and mortality rate that was associated with carotid endarterectomy in Iowa declined by nearly half (from 7.8% to 4.0%) after a statewide audit of Medicare patients had brought earlier, unfavorable results to the attention of both hospitals and surgeons. It must be noted, however, that the Iowa database was established on the basis of an impartial abstraction of the full hospital records for all the patients in the study population. This is something that the ABS simply has to keep in mind as it develops its strategy for outcome assessment. Unless the data it collects are accurate and ultimately reflect the performance of all its diplomates, those who are required to participate in this project may view it as just a chore, no matter how well intentioned it might be.
FULL TEXT