Extended outcome assessment in the care of vascular diseases: Revising the paradigm for the 21st century☆☆☆★

Article Outline

• Dimensions of extended outcome assessment
  • Clinical status
  • Cost
  • Functional status
  • Satisfaction
  • Patient preferences
• Practical and operational issues in assessing extended outcomes
  • Why should vascular physicians and surgeons assess extended outcomes?
  • Is the measurement of extended outcomes a research tool or an aid in an individual practice?
  • How does one get started?
  • Is there a prescribed set of outcome measures?
• A vision for the 21st century
• Acknowledgements
• References
• Copyright

We enter the 21st century with an impressive therapeutic armamentarium for addressing vascular diseases. With this capacity comes a societal imperative that we use the tools wisely. In many cases, we are fortunate to have a variety of means available to treat a specific vascular problem. Our challenge is to make a wise choice among alternative therapies by examining the results that each is likely to produce.

Many of the treatments available to us are costly. Therefore, it is critical to understand the relative value of treatments through examination of the costs, risks, and benefits associated with each. Potential delivery systems are in fierce price competition with each other. In settings in which prices and benefits are comparable, the patients’ perspective on quality of care becomes the determining factor in choices among delivery systems. The increasing sophistication of our patients also means that we will predictably be asked what a patient can expect to happen under a variety of treatment scenarios. All of these forces, pressing powerfully on us, dictate that we systematically broaden our capacity to evaluate the results of the care that we deliver.

This article examines the dimensions into which we might broaden our assessment of clinical outcomes. The topic is important to clinical vascular specialists, who want to improve patient care and qualify for reimbursement; to practitioner groups and practice plans, who need to demonstrate the value of their care to contracting payers and for accreditation purposes; to researchers, as they conduct and evaluate the results of clinical trials; and to designers of public policy, who wish to maximize the good that comes from medical care and minimize its cost.

There are numerous ways of conceptualizing the many dimensions of health care. Most conceptualizations agree, however, that there are more dimensions than simply the clinical or physiologic response of the target disease to the treatment offered. Other dimensions include the costs of treatment, the function and well-being of patients, and patients’ satisfaction with the care that they receive,¹, ², ³, ⁴ as well as their priorities and preferences for particular outcomes. Assessing these additional dimensions of a health care intervention is what we mean by extended outcome assessment in this article.

Back to Article Outline

Dimensions of extended outcome assessment 

Clinical status 

Within the conceptual framework just described, clinical status measures most closely correspond to outcomes that we are presently used to measuring. These might include morbidity, mortality, graft patency, complications, and laboratory test results. Many of these measures have been clearly defined and recommended for general use by the Ad Hoc Committee on Reporting Standards of the joint council of the Society for Vascular Surgery and the International Society for Cardiovascular Surgery, North American Chapter (SVS/ISCVS-NA).⁵, ⁶, ⁷, ⁸, ⁹, ¹⁰

Cost 

The value of health care can be conceptualized as a function of quality, volume, and cost: value = quality × volume/cost.¹

The public’s desire that physicians be accountable for the value of medical care offered means that we must be able to measure the cost of that care in an appropriate way. Currently, the vascular surgery literature rarely reports costs.¹¹

Exactly which costs should be measured depends entirely on the frame of reference. If, for example, the central question is personal cost from a patient’s perspective, then out-of-pocket costs and time lost from the patient’s usual activities might be appropriate and sufficient measures of cost. If the perspective is that of a provider, then actual costs of providing care become appropriate to measure. Societal costs at large are perhaps the most difficult to assess, inasmuch as arriving at estimates of these requires measurement of costs from differing perspectives—those of the patient, the provider, the payer, and the community (eg, net productive work time gained or lost because of the care). A complete description of methods for determining costs in medical care is beyond the scope of this review, but the topic is important.¹²

Functional status 

Clinical measures such as morbidity, mortality, graft patency, complications, and laboratory test results are excellent ways by which to judge technical procedural success. However, it has become increasingly evident in other medical specialties, as well as in our own, that clinical measures may correlate poorly with patients’ day-to-day function or with their perceptions of their health status. In benign prostatic hyperplasia, for example, objective measures (uroflometry, postvoid residual, prostate size, and degree of bladder trabeculation) do not correlate with patient-reported symptom severity.¹³ After total knee replacement, radiologic characteristics do not correlate with functional outcome.¹⁴, ¹⁵ Many vascular surgeons are surprised to learn how poor the correlation is between ankle-brachial index and walking ability.¹⁶, ¹⁷ This lack of correlation between hemodynamic and functional measures of success means that we must measure functional health itself (and not use clinical measures as proxies) if we are to achieve a full understanding of the results of health care.

Functional health status measures help quantify how symptoms affect aspects of everyday life. This extension of measurement addresses the World Health Organization’s charter definition of health: “a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity.”¹⁸ In addition, inclusion of such measures brings us closer to understanding the net value of a health care intervention and to being able to fully understand patients’ experiences of these interventions. In this sense, functional status measures can include global descriptors of “quality of life” but can also be more specific. Specific dimensions, such as physical and mental well-being, pain, ability to carry out desired activities, sleep, energy or vitality, and global self-estimation of health, are often assessed in this area. These are dimensions that pertain to all patients, and instruments that aim at broadly evaluating all of these dimensions are usually referred to as generic functional health assessment instruments. The most widely used generic functional health assessment instruments are listed in Table I.

Table I. Commonly used generic functional health assessment instruments

PAD, Peripheral arterial disease; US, United States.

Frequently, disease-specific functional health assessment instruments are also used, so that functions impacted by the disease in question can be examined in finer detail. Table II details the reasons for which one might wish to use generic and disease-specific functional health assessment instruments.

Table II. Pros and cons of using generic and disease-specific instruments to assess functional health

Usually, when a group of patients is being surveyed, a generic instrument and a disease-specific instrument are used together, because each yields important information. Currently, the Walking Impairment Questionnaire (WIQ) is most commonly used for detailed assessment of patients with lower extremity arterial occlusive disease.¹⁹

Information regarding a patient’s functional status is typically obtained by survey, which either is conducted by an interviewer or is self-administered. Some vascular surgeons have questioned the validity of using subjective information as an outcome measure. In many ways, as noted at the outset of this article, patients’ opinions are a driving force in medical decision making; after all, patients are the consumers of our interventions, and only they can know how the interventions affect their daily lives. Outcome is subjective from a patient’s perspective. If we follow the tenets of survey research and keep in mind that global outcome assessment requires blending clinical information with function, cost, and satisfaction outcome information, some of the concerns about using subjective information as an outcome measure may be allayed. Table III summarizes the attributes of a useful survey instrument.

Table III. Attributes of a useful survey assessing functional health

Potential sources of bias are legion in subjective outcome reporting. A relative, a caregiver, or someone else who completes a survey on behalf of a patient may overestimate or underestimate the patient’s performance in a given area.²⁰ Subtle changes in intonation, phrasing, or prompting may influence a patient’s response if the questions are posed aloud rather than in writing, and proper training of those who administer verbal surveys aims at minimizing such effects. Involved parties, such as attending surgeons, who seek feedback that they can link to individual patients can expect to receive unduly positive responses as the patients try to please (or spare the feelings of) their physicians. The careful evaluator of survey findings will be watchful for such sources of biased information.

Despite concerns about bias, patients’ reports of their functional health appear to have good face validity in populations with diseases treated by vascular physicians and surgeons. Patients with hemodynamically severe lower extremity arterial occlusive disease have more bodily pain and worse physical function than patients with mild disease,²⁰ patients with severe strokes have worse physical and role function and worse general health than patients with mild strokes,²¹ and patients with venous ulcers have impaired social interaction, domestic activities, and emotional status.²² Patients with vascular diseases also report that clinically successful therapy improves their function in these dimensions. Hemodynamically successful revascularization for lower extremity arterial occlusive disease lessens bodily pain and improves physical function and mobility.¹⁷, ²³, ²⁴, ²⁵ Healing of venous ulcers is also associated with functional health improvement.²² Observations such as these, which are appearing with increasing frequency in the vascular literature, speak strongly to the role of subjective reporting in the evaluation of the results of care for vascular diseases.

In the United States, the most commonly used validated generic functional health assessment instrument is the Medical Outcomes Study (MOS ) short-form 36 (SF-36). Answers given by patients to these 36 questions can describe patient function in physical and social spheres, role limitations due to physical health problems or emotional problems, and patient perceptions of general health as well as bodily pain, mental health, and vitality. In addition, the SF-36 evaluates patient perception of change in health over the past year. Thus, the SF-36 combines elements of the functional status questionnaire and the quality of life questionnaire. Answers are typically scored on a scale from 0 (poorest) to 100 (best). Because so many people with defined characteristics have responded to this questionnaire, a patient’s scores can be normalized and reported in comparison with the scores for a general population or a population with defined characteristics. In this case, the distribution of scores in the comparison population is reported with a mean of 50 and an SD of 10; the patient’s score is reported on the normalized scale.²⁶ Table IV reproduces the questions of the SF-36.

Table IV. Concepts and questions of Medical Outcomes SF-36²⁹, ⁵¹

The questionnaire is not presented here in the format in which it should be administered.

Surgeons might wonder what value the measurement of generic functional health has for their patients. The accumulation and analysis of a vast amount of functional data from patients responding to the SF-36 have revealed that there is much practical and prognostic information to be gleaned from patients’ scores. For example, a patient with a raw score (not normalized) of less than 80 on the bodily pain scale has a greater than 50% chance of having pain interfere with normal work; virtually all patients scoring less than 50 on this scale report that pain interferes to a moderate or extreme extent with normal work. As a corollary, improvement from a score of 50 to a score of 60 on this scale is associated with marked decrease (from 37% to 8%) in the likelihood that pain will interfere significantly with normal work.²⁷

The prognostic information associated with SF-36 scores is also of interest. For example, the annual medical expenditures of Medicare patients who in 1995 rated their own health as “poor” averaged $12,000, whereas the comparable figure for patients who rated their health as “fair” (the next best possible rating) was one half of that amount.²⁸ The physical health summary score (a composite of responses to questions concerning physical function, pain, general health, vitality, and social function) is linearly related to job loss at 1 year and to mortality at 5 years.²⁹ The latter relationship is of particular interest: if the summary score is less than 35, 5-year mortality is 17.3%; if it is greater than 55, 5-year mortality is only 1.8%.

In what way might functional health assessment be relevant to patients with vascular disease? Relevance is most easily seen in assessing the value of interventions for symptomatic conditions such as claudication or ischemic rest pain. In this case, the principal aim of an intervention is to relieve the target symptoms. Technical/clinical success of the intervention (in this case, patency of the segment undergoing intervention) is usually the sine qua non of functional success. However, functional measures go beyond the technical/clinical measures, both to ask whether the underlying symptoms—in these examples, walking disability and lower extremity pain—have been relieved and to help assess what the overall effect of the intervention has been on a patient’s ability to function in his or her daily life away from the medical care setting. Has the intervention had unintended outcomes with regard to overall pain, ability to carry out household chores, interaction with others, and feelings of vitality and optimism? Incisional pain, lower extremity edema, depression, and discouragement might conceivably affect all of these important functions and negate the clinical value of a procedure (eg, revascularization for claudication, surgery for venous insufficiency) in the eyes of an individual patient. Our current technically oriented outcome measurement standards give us few tools to use in assessing these important outcomes.

The relevance of functional health assessment to asymptomatic patients undergoing interventions aimed at averting possible catastrophe (eg, aneurysm surgery, carotid endarterectomy for asymptomatic disease) is also readily seen. What are the functional trade-offs of intervention in the asymptomatic state? Are they worse than the looming catastrophe? Does intervention relieve or cause anxiety about an identified risk? Currently, we are not meeting the patients’ need for us to assess these unintended consequences and be able to provide answers when they ask what daily life might be like after proposed interventions. Having patient-derived functional health information will probably not change the operations that we perform to avert life-threatening catastrophes, but it will improve our understanding of the consequences for patients and allow us to improve our systems of care to lessen the adverse impact of these procedures.

Satisfaction 

Assessment of patient satisfaction with medical care is an ongoing challenge. Individual practices and providers or organizations have circulated many different patient satisfaction surveys thought to be suited to their individual needs. Managed-care plans are increasingly required to assess themselves using more widely adopted satisfaction measures, such as those from the Health Plan Employer Data and Information Set.³⁰ In general, these measures of satisfaction are generic and focus on access to providers, courtesy and respect shown to patients, satisfaction with the physical setting where care is delivered, waiting times, and provider/staff communication skills. These are undoubtedly important aspects of the patient’s encounter with the medical system. However, in surgical care, in which there is an identifiable intervention occurring as a discrete episode, other dimensions may be equally or more important. These dimensions are largely unexplored, but they might include some or all of the following: perceived health benefit of the operation or procedure, patient perception of surgeon competence, patient understanding of what could happen, and ability of the surgeon and/or system to address the patient’s fears and provide information on what the patient could do to hasten recovery.

Patient preferences 

The value a patient places on a specific health state may have an important influence on the choice that he or she makes among therapeutic alternatives. Achieving a better understanding of patients’ preferences for various specific health states is an important goal of extended outcome assessment. With an understanding of patients’ values, two types of analysis can inform our choices among alternatives.³¹ In quantitative decision analysis (also known as expected value decision making ), the likelihood of a specific clinical result is multiplied by the patient-derived value of the result to determine the relative expected values (on either an individual-patient basis or a population basis) of therapeutic alternatives. Cost-effectiveness analysis , more often done on a population basis for public policy purposes, determines the net benefit per net unit cost of an intervention. Population-derived preferences are used to define the net benefit, often expressed in quality-adjusted life years (QALYs), such a unit being equal to the fraction of a year in perfect health that the patient believes to be equivalent in value to a year in the health state in question.³¹

Table V outlines the domains of extended outcome assessment, the methods by which these domains are typically measured, and the uses that are typically made of information in each domain.

Table V. Extended outcome assessment at a glance

HEDIS, Health Plan Employer Data and Information Set.

Back to Article Outline

Practical and operational issues in assessing extended outcomes 

Why should vascular physicians and surgeons assess extended outcomes? 

From a theoretical viewpoint, extended outcome assessment should enable us to reach several goals, including the following: (1) achieving a more complete understanding of the effectiveness of different interventions; (2) giving physicians, payers, and patients the information that they need to make better decisions; and (3) developing standards to aid physicians, public policy makers, and third-party payers in optimizing the use of resources. However, acquiring more data pertaining to the outcomes of specific interventions may by viewed by practicing vascular clinicians as a costly and time-consuming venture with limited return. Many of us believe that we already follow best medical practices and that our patients have good medical outcomes. However, there are significant benefits of outcomes assessment that will reward the expenditure of effort.

One benefit of pursuing such a course might simply be that it fulfills the expectations that society has for us. Medicine is one of the three learned professions, and part of the definition of a profession is “a vocation in which a professed knowledge of learning or science is used in its application to the affairs of others.”³² As detailed in this article, it is now accepted that there are many dimensions of the outcomes of our interventions. It is imperative that we, as members of our profession, assess these extended outcomes and apply our findings to the “affairs” (lives) of our patients.

Another benefit of the course outlined above is of a practical nature. Medicine is a service industry, and we, as providers, should meet the needs of our customers. Currently, patients as well as payers are demanding more detailed assessments of the outcomes of procedures so that they can make informed decisions. Payers are more likely to pay for procedures from which patients clearly perceive that they have derived benefit.

We are in an era of medical cost containment, and this will continue for the foreseeable future. This implies that using medical resources for procedures with limited to no overall benefit decreases our ability to offer services that are clearly beneficial. Extended outcomes assessment provides the data that will allow us to be more confident in the wisdom of our use of resources for medical treatments and ensure that we will deliver the most appropriate, cost-effective, and medically effective care.

By taking the lead in assessing extended outcomes, we will take more effective control of our professional future. The Task Force on Competence of the American Board of Medical Specialists is reworking its concepts of appropriate criteria for maintenance of certification for medical specialists. One of the criteria in the preliminary draft versions dealing with these concepts is “evidence of satisfactory practice performance.” This clearly means that we must know, in detail, the outcomes of our interventions. Several government and private agencies have already launched programs directed at determining the effectiveness of medical interventions. It is highly probable that they will use this information for the development of national guidelines for medical practice. The dimensions in which they determine effectiveness may or may not be clinically appropriate, but if we do not have data with which to support alternative interpretations, these agencies will define effectiveness without us and determine for us what care is appropriate as well as where resources will be allocated.

What’s in extended outcomes assessment for vascular surgeons and physicians? Simply put, wise assessment of an extended set of outcomes will allow optimal decisions for all parties involved.

Is the measurement of extended outcomes a research tool or an aid in an individual practice? 

Prediction rules, allocation decisions, and population health can all be made or described on the basis of outcome assessment in a research setting. Many may wonder, therefore, why individual physicians should trouble to measure functional health or other dimensions of extended outcome in their own practices.

We strongly believe that there are many reasons why data obtained in research settings or in other practices may not be sufficient for optimization of the care of individual patients. Chief among these is the individuality of patients and their concerns. The extreme contrast between one patient who experiences terrible emotional distress when confronted with the knowledge of an asymptomatic but potentially life-altering condition and another patient who is much more phlegmatic is known to all vascular surgeons, and each of us has used such information to optimize the timing for repair of an abdominal aortic aneurysm. Not all such feelings are as well sought out by or communicated to physicians, however. In the geriatric general practice experience, assessment of functional health status in conjunction with every visit improves care both in patients’ eyes and as judged by external standards.³³ Having patientspecific information readily available at the time of a clinical encounter makes for the best opportunity to optimize the care that is offered to the patient.

How does one get started? 

The primary purpose of taking the initiative to assess extended outcomes is to improve them over time. The first, and perhaps most difficult, task in starting on the venture is to be realistic and relentlessly practical in understanding which characteristics of your practice might benefit most from change. This dictates the highest priorities for data collection and helps to streamline the number and types of questions that you pose to patients and the data that you subsequently aggregate and analyze. No program will be perfect at its outset, and it is important to be reassured that all outcome assessment programs evolve over time, both because of increasing expertise and because of changing goals. The most important step to take is to begin.

Following the characterization of the dimensions of extended outcome assessment discussed above, one might evaluate one or two elements of each area (clinical status, cost, functional status, satisfaction, and preferences). Useful measures of clinical status have been well characterized in the publications of the Ad Hoc Committee on Reporting Standards of the joint council of the SVS/ISCVS-NA.⁵, ⁶, ⁷, ⁸, ⁹, ¹⁰

As noted, the choice of appropriate measures of cost is dictated by the specific question that the assessor wishes to answer. Because of their complexity, cost considerations are frequently omitted from an initial attempt to measure extended outcomes.

Assessment of functional status is more problematic at present. As our specialties’ collective expertise in assessing these areas increases, we may reach consensus on the measures that yield the most information about patients with vascular diseases. In the interim, it is easiest to use published measures that are aimed at assessing the problems that vascular patients are likely to have, are sensitive to clinically important change and reproducible in the absence of clinical change, and involve the asking of questions that are understood in the same way by a broad variety of patients (ie, questions that have been “validated”).³⁴ If these measures can be scored easily and the data can be entered directly by patients into the analyzing computer, so much the better. For these reasons, and for the other reasons cited above, a logical first step might be to assess patients’ responses to the SF-36 over time. The SF-36 is easy to obtain and is free to use for personal or organizational noncommercial purposes as long as permission has been obtained from the developer.³⁵ As previously noted, preferred scoring algorithms have changed over time as a larger database of responses from patients of different ages and defined comorbid conditions and diseases has been accumulated. Electronic scoring algorithms, available commercially, greatly simplify the task of comparing a patient or population to other defined populations (such as healthy Americans of the same age and sex). Printed tables of norms for specific patient populations are also available.²⁹, ³⁶

Readers new to the field of functional health assessment frequently wonder how much change in an SF-36 score might be generally thought to be clinically significant. The answer to this question depends on the variability in the underlying population. One general way to assess whether a treated population is different from an untreated or comparison population is to calculate the effect size (ratio between the means or medians of the two groups and the SD or interquartile range of the same two populations).³⁷ Combining the use of electronic data entry by patients and the concepts of item response theory may allow researchers to estimate an individual’s functional health in a given area with far greater precision than in the past, thereby making it quicker and easier to assess the effectiveness of clinical treatments over time.³⁸ The power of this concept is exemplified by the rapidity with which the general mental health of a middle-aged, chronically ill person can be assessed with enough precision to flag positivity of a depression screen. Formerly, this would have required that the patient respond to 31 questions; now, use of the dynamic technique reduces the number of questions to two to four, with an associated reduction of 80% to 90% of the time required.³⁹

Functional health assessment tools specific for vascular diseases are increasingly available¹⁹, ²² and help to allow more detailed and clinically relevant functional health assessment. Increasing experience with these instruments may someday allow the establishment of expected norms for a given population.

Finally, measures of patient satisfaction and preference may be incorporated into an extended outcome assessment. These, however, are the least well standardized of measures of the different dimensions in extended outcome assessment. Satisfaction with surgical care has not been well explored, so one might be tempted to incorporate generic measures of satisfaction with health care.³⁰ Although several methods have been developed and used for assessing patient preferences,³¹ these are cumbersome and not easily compacted, though adaptation to patient entry by computer has been achieved.⁴⁰

The best method for collecting and analyzing the data depends on local resources and personnel. Paper questionnaires require data aggregation, usually by secondary entry into a computer. Bar coding and scanning technology require somewhat less work but involve a greater investment in hardware. Having patients respond to the questions electronically, either through use of freestanding office computers or through the Internet, increases the ease of data aggregation but also the requirement for technical sophistication. Partially customizable but commercially available outcome tracking programs may represent a good investment for some practices.

Is extended outcome measurement necessarily more time-consuming and costly than current practice? Presently, members of our profession tend to accumulate dissimilar data elements in various formats, both paper and electronic, for purposes of clinical notation, billing, and outcome assessment. Inefficiencies of parallel data collection and reporting systems are the rule rather than the exception.

Is it possible to imagine that more complete and more useful data collection could be accomplished? We believe that with the aid of advances in patient/provider and electronic sophistication, such improvement is possible. Patients are increasingly able to respond electronically, either in the office or through the Internet, to surveys such as those used to quantitate functional health and assess patient satisfaction. Such electronic response largely obviates data reentry by other people and provides data for immediate computer generation of clinical chart notes. By engaging in extended outcome assessment, we might be able to have more complete patient-derived data available to us in real time to help with clinical care while spending less time in documentation.

Is there a prescribed set of outcome measures? 

To advocate a mandatory set of “outcome measures” would be both presumptuous and unproductive, inasmuch as each improvement to be made or question to be answered has a different spectrum of specific priorities. However, many of us are performing similar procedures (such as revascularization for limb threat) and are asking questions from similar perspectives (such as those of our individual practices). By analogy to the achievements of the Ad Hoc Committee on Reporting Standards of the joint council of the SVS/ISCVS-NA, achieving consensus on measures that are to be used for evaluating similar concepts in an extended outcome measurement scheme would represent a significant advance in our field. Doing this would facilitate discussions in a common language about an extended set of outcomes of procedures for vascular conditions, facilitate rapid comparisons among patient groups or specific interventions, and thereby make improvement in clinical care as rapid as possible. This is the kind of accountability that we owe our patients.

Back to Article Outline

A vision for the 21st century 

In summary, our next task is to have the data that we need available in real time so that we can answer the questions of patients, practitioners, and payers and planners—respectively, “What is likely to happen to me if we pursue alternate strategy a , b , or c ?” “How am I doing in treating condition x ?” and “What is the value of specific therapies for defined populations?” We can take a first step in this direction by working together in the effort to reach consensus about which extended measures of procedural outcome we will use.

Back to Article Outline

Acknowledgements 

The Committee wishes to thank John D. Birkmeyer, MD, and the members of the Dartmouth Surgical Outcomes Assessment Group for their critical review of and input to this article.

Back to Article Outline

References 

1. Nelson EC, Mohr JJ, Batalden PB, Plume SK. Improving health care, part 1: the clinical value compass. Jt Comm J Qual Improv. 1996;22:243–256
   • View In Article
   • MEDLINE
2. Nelson EC, Batalden PB, Plume SK, Mohr JJ. Improving health care, part 2: a clinical improvement worksheet and users’ manual. Jt Comm J Qual Improv. 1996;22:531–548
   • View In Article
   • MEDLINE
3. Mohr JJ, Mahoney CC, Nelson EC, Batalden PB, Plume SK. Improving health care, part 3: clinical benchmarking for best patient care. Jt Comm J Qual Improv. 1996;22:599–616
   • View In Article
   • MEDLINE
4. Batalden PB, Mohr JJ, Nelson EC, Plume SK. Improving health care, part 4: concepts for improving any clinical process. Jt Comm J Qual Improv. 1996;22:651–659
   • View In Article
   • MEDLINE
5. Baker JD, Rutherford RB, Bernstein EF, Kempczinski RF, Zarins CK. Suggested standards for reports dealing with cerebrovascular disease. J Vasc Surg. 1988;8:721–729
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (841 KB)
   • CrossRef
6. Johnston KW, Rutherford RB, Tilson MD, Shah DM, Hollier L, Stanley JC. Suggested standards for reporting on arterial aneurysms. J Vasc Surg. 1992;13:452–458
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (495 KB)
   • CrossRef
7. Ahn SS, Rutherford RB, Becker GJ, Comerota AJ, Johnston KW, McClean GK, et al.  Reporting standards for lower extremity arterial endovascular procedures. J Vasc Surg. 1993;17:1103–1107
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (480 KB)
   • CrossRef
8. Porter JM, Moneta GL. Reporting standards in venous disease: an update. J Vasc Surg. 1995;21:635–645
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (1039 KB)
   • CrossRef
9. Ahn SS, Rutherford RB, Johnston KW, May J, Veith FJ, Baker JD, et al.  Reporting standards for infrarenal endovascular abdominal aortic aneurysm repair. J Vasc Surg. 1997;25:405–410
   • View In Article
   • Full Text
   • Full-Text PDF (363 KB)
   • CrossRef
10. Rutherford RB, Baker JD, Ernst C, Johnston KW, Porter JM, Ahn S, et al.  Recommended standards for reports dealing with lower extremity ischemia: revised version. J Vasc Surg. 1997;26:517–538
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (1996 KB)
    • CrossRef
11. Shackley P, Slack R, Michaels J. Costing vascular surgery: a review of current reporting practice. J Vasc Surg. 1999;30:668–678
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (51 KB)
    • CrossRef
12. Gold M, Siegel J, Russell L, Weinstein M. Cost-effectiveness in health and medicine. New York: : Oxford University Press; 1996;
    • View In Article
13. Barry MJ, Cockett AT, Holtgrewe HL, McConnell JD, Sihelnick SA, Winfield HN. Relationship of symptoms of prostatism to commonly used physiological and anatomical measures of the severity of benign prostatic hyperplasia. J Urol. 1993;150:351–358
    • View In Article
    • MEDLINE
14. Cameron H. Clinical and radiologic effects of diaphyseal stem extension in noncemented total knee replacement. Can J Surg. 1995;38:45–50
    • View In Article
    • MEDLINE
15. Li PLS, Zamora J, Bently G. The results at ten years of the Insall-Burstein II total knee replacement: clinical, radiological, and survivorship studies. J Bone Joint Surg Br. 1999;81:647–653
    • View In Article
    • MEDLINE
    • CrossRef
16. Arfvidsson B, Wennmalm A, Gelin J, Dahllof A-G, Hallgren B, Lundholm K. Co-variation between walking ability and circulatory alterations in patients with intermittent claudication. Eur J Vasc Surg. 1992;6:642–646
    • View In Article
    • MEDLINE
    • CrossRef
17. Feinglass J, McCarthy WJ, Slavensky R, Manheim LM, Martin GJ, Chicago Claudication Outcomes Research Group . Effect of lower extremity blood pressure on physical functioning in patients who have intermittent claudication. J Vasc Surg. 1996;24:503–512
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (913 KB)
    • CrossRef
18. World Health Organization . The constitution of the WHO. WHO Chron. 1947;1:29
    • View In Article
19. Regensteiner JG, Steiner JF, Panzer RJ, Hiatt WR. Evaluation of walking impairment by questionnaire in patients with peripheral arterial disease. J Vasc Med Biol. 1990;2:142–152
    • View In Article
20. Pell JP. Impact of intermittent claudication on quality of life. on behalf of Scottish Vascular Audit Group Eur J Vasc Endovasc Surg. 1995;9:469–472
    • View In Article
    • Abstract
    • Full-Text PDF (344 KB)
    • CrossRef
21. Anderson C, Laubscher S, Burns R. Validation of the short-form 36 (SF-36) health survey questionnaire among stroke patients. Stroke. 1996;27:1812–1816
    • View In Article
    • MEDLINE
    • CrossRef
22. Smith JJ, Guest MG, Greenhalgh RM, Davies AH. Measuring the quality of life in patients with venous ulcers. J Vasc Surg. 2000;31:642–649
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (42 KB)
    • CrossRef
23. Klevsgard R, Hallberg IR, Risberg B, Thomsen MB. The effects of successful intervention on quality of life in patients with varying degrees of lower-limb ischemia. Eur J Vasc Endovasc Surg. 2000;19:238–245
    • View In Article
    • Abstract
    • Full-Text PDF (189 KB)
    • CrossRef
24. Currie IC, Wilson YG, Baird RN, Lamont PM. Treatment of intermittent claudication: the impact on quality of life. Eur J Vasc Endovasc Surg. 1995;10:356–361
    • View In Article
    • Abstract
    • Full-Text PDF (491 KB)
    • CrossRef
25. Chetter IC, Spark JI, Kent PJ, Berridge DC, Scott DJA, Kester RC. Percutaneous transluminal angioplasty for intermittent claudication: evidence on which to base the medicine. Eur J Vasc Endovasc Surg. 1998;16:477–484
    • View In Article
    • Abstract
    • Full-Text PDF (1154 KB)
    • CrossRef
26. Ware JE. Tech notes: norm-based interpretation. Med Outcomes Trust Bull. 1994;2:3
    • View In Article
27. Ware JE. Content-based interpretation of health status scores. Med Outcomes Trust Bull. 1994;2:3
    • View In Article
28. Eppig FJ, Poisal JA. Measuring the health status of Medicare beneficiaries: 1995. Health Care Fin Rev. 1997;18:207–210
    • View In Article
29. Ware JE, Kosinski M, Keller SD. SF-36 Physical and Mental Health Summary Scales: a user’s manual. Boston: : The Health Institute, New England Medical Center; 1994;
    • View In Article
30. National Committee for Quality Assurance . HEDIS 2000 list of measures. Available at http://www.ncqa.org/pages/ policy/hedis/h00meas.htmAug 30, 2000.; Accessed
    • View In Article
31. Sox H, Blatt MA, Higgins MC, Marton K. Medical decision making. Boston: : Butterworth Publishers; 1988;
    • View In Article
32. Simpson JA, Weiner ESC. 2nd ed. The Oxford English dictionary. Vol 12. Oxford: : Oxford University Press; 1995;
    • View In Article
33. Magari ES, Hamel MB, Wasson JH. An easy way to measure quality of physician-patient interactions. J Ambulatory Care Manage. 1998;21:27–33
    • View In Article
34. McDowell I, Newell C. Measuring health: a guide to rating scales and questionnaires. 2nd ed. Oxford: : Oxford University Press; 1996;
    • View In Article
35. QualityMetric Inc. Available at: http://www.qmetric.com/ products/assessments/license.php3.
    • View In Article
36. Ware JE, Snow KK, Kosinski M, Gandek B. SF-36 health survey manual and interpretation guide. Boston: : The Health Institute, New England Medical Center; 1993;
    • View In Article
37. Light RJ, Pillemer DB. Summing up: the science of reviewing research. Cambridge, Mass: : Harvard University Press; 1974;
    • View In Article
38. Bjorner JB, Ware JE. Using modern psychometric methods to measure health outcomes. Med Outcomes Trust Monitor. 1998;3:12–16
    • View In Article
39. AmIhealthy.com. Available at: http://prosite.amihealthy.com. Accessed Aug 30, 2000.
    • View In Article
40. Sumner W, Nease R, Littenberg B. U-titer: a utility assessment tool. In: Proceedings of the 5th Annual Symposium on computer applications in medical care. Washingotn (DC). 1991;p. 701–705
    • View In Article
41. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36), I: conceptual framework and item selection. Med Care. 1992;30:473–483
    • View In Article
    • MEDLINE
    • CrossRef
42. McHorney CA, Ware JE, Raczek AE. The MOS 36-item short-form health survey (SF-36), II: psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care. 1993;31:247–263
    • View In Article
    • MEDLINE
    • CrossRef
43. McHorney CA, Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36), III: tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 1994;32:40–66
    • View In Article
    • MEDLINE
    • CrossRef
44. EuroQol Group . EuroQol: The current state of play. Health Policy. 1996;37:53
    • View In Article
    • Abstract
    • Full-Text PDF (1303 KB)
    • CrossRef
45. Nelson EC, Landgraf JM, Hays RD, Wasson JH, Kirk JW. The functional status of patients: how can it be measured in physicians’ offices?. Med Care. 1990;28:1111–1126
    • View In Article
    • MEDLINE
    • CrossRef
46. Van Weel C. Functional status in primary care: COOP/ WONCA charts. Disabil Rehabil. 1993;15:96–101
    • View In Article
    • MEDLINE
    • CrossRef
47. Hopton JL, Porter AMD, Howie JGR. Measure of perceived health in evaluating general practice: the Nottingham Health Profile. Fam Pract. 1991;8:253–260
    • View In Article
    • MEDLINE
48. Jette AM, Davies AR, Cleary PD, Calkins DR, Rubenstein LV, Fink A, et al.  The Functional Status Questionnaire: reliability and validity when used in primary care. J Gen Intern Med. 1986;1:143–149
    • View In Article
    • MEDLINE
    • CrossRef
49. Kaplan RM, Bush JW. Health-related quality of life measurement for evaluation research and policy analysis. Health Psychol. 1982;1:61–80
    • View In Article
    • CrossRef
50. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The Sickness Impact Profile: development and final revision of a health status measure. Med Care. 1981;19:787–805
    • View In Article
    • MEDLINE
    • CrossRef
51. Trust Inc MOS. How to score the MOS 36-item short-form health survey (SF-36). In: Boston: : International Resource Center for Health Care Assessment, The Health Institute, New England Medical Center Hospital; 1992;p. 19
    • View In Article