Hold it—my patients are sicker!

The importance of case-mix adjustment to practitioner profiles in British Columbia

Practitioner profiles are used to influence practitioner practice patterns, to provide physicians an opportunity to compare their practice patterns with their peers, and to focus audit resources. To measure the efficiency of practice, an estimate of the level of expected cost is needed. Traditional measures of expected cost based on age and gender of patients have been demonstrated to be a relatively poor predictor of individual need for health care services. A case mix adjustment method known as adjusted clinical groups has been evaluated extensively in the United States and has been the subject of recent evaluations in British Columbia and Manitoba. The adjusted clinical groups system places each patient into a mutually exclusive morbidity category, based on the combinations and types of ICD-9 codes that pertain to the individual over an extended period of time. Adjusted clinical groups are far more predictive of resource use than are age/gender categories. Measures of practitioner efficiency are more meaningful using adjusted clinical groups expected costs rather than age/gender expected costs.

Introduction

"It was the best of times, it was the worst of times.”—Charles Dickens, A Tale of Two Cities

It is an exciting time to practise medicine because of advances in the understanding of the genetic and molecular basis for many diseases, new diagnostic techniques, and therapeutic advances. It is also a very stressful time: physicians are under pressure to balance services with patient needs while trying to keep overall costs down. In British Columbia, physicians are now being asked to account for their overall billing patterns. One of the many worries they face is the prospect of a billing audit.

The BC Ministry of Health, in cooperation with the British Columbia Medical Association’s Patterns of Practice Committee, produces an annual practice profile for each of BC’s approximately 7000 physicians. Health systems increasingly are using profiling as a means to shape practice activity and to identify high cost outliers.[1-3] The BC profile has several purposes. The first is to provide physicians with information about how their practice patterns compare to those of their peers. With this information physicians can examine why their practice patterns are more or less costly. Second, practice profiles are used to guide the audit program in the selection of physicians with very high per-patient costs for investigation. The stated objectives of the MSP audit program are to detect unjustified billing patterns, to deter inappropriate billing patterns, both specifically and generally, and to recover payments made as a result of inappropriate billing patterns.

As examples, Table 1 provides a simplified version of a BC practice profile for two general practitioners identified as high-cost outliers. It shows the numbers of different patients served by the practitioners and the payments for GP services made by the BC Medical Services Plan. The table is divided into two parts: one for all the patients who visit a practitioner during the course of a year, and the other for only those patients for whom the practitioner provides most of the care. We focus on these latter patients, called majority source of care (MSOC) patients, because these are the patients for whom the practitioner is most responsible. In our examples, Practitioner 1 provided general practice services to 1360 patients while Practitioner 2 served only 755 patients. (Among all GPs practising in BC in 1999, the median number of patients was 1632. The average total billings for GP services was $158,306). However, these practitioners provided the MSOC for only 58.1% and 59.7% of all the patients they saw. In other words, almost half of the patients seen by these practitioners were seen principally by other GPs. Overall, the practitioners had high per-patient costs for their practice populations, especially for the MSOC patients. In fact, they were both above the 95th percentile when ranked in their per-patient costs with their peers. These practitioners would both be considered high-cost outliers because per-patient costs are greater than 2 standard deviations above the mean.

Is it fair that Practitioner 1 should bill almost $94,000 more than the median for a practice that has 272 patients fewer than most? Is it fair that Practitioner 2 should bill very near the median amount on a practice that is 46% of the median practice size? Are these physicians practising efficiently? Should one or the other, or both, have an on-site audit of their medical records?

“But my patients are sicker!”

Several factors may determine why costs are higher in some practices compared with others. These include differences in the morbidity levels and patient need, the local availability of medical facilities and other health care professionals, and patients’ propensity to seek care. Moreover, some physicians may have a more costly practice style than others because they recall their patients for follow-up more frequently, order more laboratory and radiology tests, or are more likely to hospitalize. A comparison of per patient costs without adjustment for the health status of patients makes the implicit assumption that all of the non-physician factors are more or less equal from one practice to another. It is well known, however, that some patient populations are much sicker than others.[4] Differences in the burden of morbidity among patient populations may occur for many reasons. Some practices may be located in impoverished neighborhoods, which are known to have greater health needs than more affluent communities. Alternatively, some physicians may have special skills or interests (such as in diabetes or asthma care) that result in a following of sicker (or healthier) cohort of patients. Other physicians may have spent a long time in the same practice where they developed a loyal following of relatively ill or complex patients.

Even among patients who suffer from similar diseases, their need for clinical care may not be the same. This may be due to increased severity of particular problems (such as persistent versus intermittent asthma) but, even more likely, it may be due to the presence of important co-morbid conditions. For instance, some diabetic patients also suffer from coronary artery disease, peripheral vascular disease, or impaired renal function, and these additional conditions can have multiplicative effects on their health service needs. In addition, patients with similar illness levels may not have the same social support. For instance, some patients with schizophrenia have good social supports at home while others are homeless with few resources. Even if it were possible to measure and account for these factors, individuals suffering from similar disorders may have a different propensity to seek medical care. If these factors are not randomly distributed among practice populations, the costs of one practice’s patients could be potentially much higher than another. Without methods to account for these influences, one could inappropriately flag a practitioner as inefficient simply because he or she has a particularly sick or disadvantaged practice.

Leveling the playing field

The BCMA and the BC Medical Services Plan regard the audit program as an important tool in managing physician services and helping to ensure equity and fairness in physician payment. As a general principle, it is felt that all physicians should be playing on a level field. Therefore, the profile system has been designed to account for patient- and morbidity-related factors that could justifiably result in higher per-patient costs.

In 1997, MSP adopted age and gender standardization for its practitioner profiles. Costs that were incurred by a practitioner’s panel of patients were examined after accounting for the age and gender differences. This profile answers the question: “What would the payments to the physician have been if each patient in the practice had used an average amount of care for their age and gender group?”

Age and gender are convenient proxies for morbidity levels because they are easy to measure and are almost always error-free. However, demographics are relatively crude proxies since a large amount of variation in illness levels exist between individuals even after accounting for age and gender. For instance, the presence of severe or chronic morbidity is more likely as people get older, but the elderly do not universally have the same number or types of illnesses. In BC, research has shown that age and gender characteristics account for only a small minority of variants in the amount of resources consumed by patients.[5]

Case-mix adjustment

Because of the inadequacy of demographics as a proxy for illness in physician practices, the BCMA and the Medical Services Plan sought a more sophisticated approach for their profiling initiative. A variety of case-mix measurement tools (the term case mix refers to the mix and severity of illnesses for individual patients or groups of patients) have been developed that are intended to measure the overall morbidity of patients, and when aggregated, populations, by combining diagnosis codes contained on physicians’ billing records and hospital discharge abstracts.

The adjusted clinical group (ACG) system developed at Johns Hopkins University is one such tool that categorizes individuals into mutually exclusive illness categories based on their age, gender, and the mix of diagnoses they receive. While many case-mix methods are designed for particular episodes of illness (such as an admission to hospital or a series of related outpatient visits), the ACG system assigns individuals to illness categories based on all the diagnoses they receive over an extended period of time, regardless of site. Since ACGs are population based and quantify morbidity across episodes, they are particularly useful in profiling applications. The ACG system uses routinely collected administrative data, making it more practical for large-scale use than using self-reported health status from patient surveys. Moreover, because it uses only diagnoses—not procedures or hospitalizations—to define illness levels, it does not reward practices that elect to hospitalize patients more readily or perform more procedures. The main focus of the ACGs is to account for both severity of particular illnesses as well as the number and complexity of co-morbid conditions.

The main problem with using diagnostic data to quantify a person’s overall burden of morbidity is the vast number of different diagnoses and the number of combinations that can occur. To manage this multiplicity of codes, the ACG system groups all ICD-9 and ICD-9 CM diagnosis codes into 32 clinically similar aggregated diagnostic groups (ADGs) based on the following criteria:[6]

• Expected duration of illness (e.g., acute, chronic, or recurrent)

• Disease severity (i.e., expected prognosis with respect to disability and longevity)

• Diagnostic certainty (e.g., signs and symptoms versus diagnosed conditions)

• Etiology (e.g., infections, neoplasms, psychosocial conditions)

• Expected need for specialist care or hospitalization

Aggregated diagnostic groups are the building blocks of the ACG system. It is possible (and indeed likely) that an individual’s diagnoses fall into multiple ADGs. Through a simplifying algorithm (with a statistical technique known as recursive partitioning), various combinations of ADGs and age and gender categories are merged into approximately 100 mutually exclusive ACG categories. In the end, an individual is assigned to a single ACG that represents their overall burden of illness over the period in question (further information about ACG design and theory can be found at the following web site: http://acg.jhsph.edu). While individuals in the same ACG may have many different conditions, this method is very useful for comparing patients’ resource use because people in the same ACG are expected to consume a similar amount of health care services.

Evaluating the ACGs in BC

ACGs have been extensively evaluated in the US[7] and are now considered the gold standard case-mix adjustment method for physician profiling. BC and Manitoba are the first Canadian provinces, however, to evaluate and use ACGs with Canadian claims data. Because ACGs rely on diagnosis codes, their validity is threatened because of the lack of secondary diagnosis codes in BC, the lack of specificity in the codes used (i.e., MSP uses 3-digit ICD codes versus more precise 4- and 5-digit ICD-9 CM codes), and the limited attention that has historically been placed on the accuracy of diagnosis recording. Moreover, there was also a concern that if diagnoses were used for management purposes, this would create the incentive for physicians to use more severe codes or code more extensively.

Table 2 summarizes the results of the linear regression statistical models used to evaluate ACG performance with BC data. The models examine the degree to which ACGs can explain differences in health care costs among individual patients. Similar models using simple age and sex models are provided for comparison. Compared to age and gender categories, the ACGs have approximately 5 times the explanatory power in explaining resource use.[5] These results suggest that the ACGs are robust enough to reasonably account for the lack of specificity in diagnosis coding and physicians’ idiosyncratic coding behavior.

Adjusting for illness in practice profiles

The above and other evaluations suggest that the ACG system is a reliable and feasible case-mix adjustment method for patient populations. The next question is, how well does this case mix adjustment work in physicians’ practices? Let us return to the examination of our sample practitioners (from Table 1) after accounting for case mix. The remaining analyses are directed only at those patients for whom the practitioners are principally responsible (i.e., where the practitioner is the majority source of care). Table 3 presents both the actual payments made for these patients as well as what would be expected if they used an average amount for their ACG category. The ACG expected costs can be used to estimate the relative “sickness” level of each practice by dividing the per capita expected costs by the average costs for all GPs. For instance, Practitioner 1 has a relatively sick practice because her patients would be expected to incur 34% higher costs than average based on their mix of diagnoses. Conversely, Practitioner 2 has a relatively healthy practice with expected costs that are 8% lower than average.

The total costs incurred by a physician’s patients can be compared to what would have been incurred if each of them had been average for their case-mix group. This comparison is called the observed to expected cost ratio (O:E ratio) and is often expressed as a percentage. By comparing observed to the expected costs, we are provided with an indication of how efficient a particular provider is at delivering care. Providers with O:E ratios that are considerably greater than 100% are considered less efficient than are those with ratios at or below 100%.

The Figure entitled “Comparison of ACG- and age/gender-based O:E ratios—Practices of all BC physicians” shows the distribution of the ACG-based O:E ratios (and similarly constructed ratios based on age and gender categories) for all GPs in BC. Overall, the distribution of age/gender ratios is much flatter, with thicker tails than the distribution of the ACG-based ratios. This finding means that a greater proportion of GPs would be seen as providing too many (or too few) services using age and sex adjustment versus ACG adjustment. With ACGs, 417 fewer GPs (11% of the total) would be categorized as having patients that are 25% more costly than the group norm.

Practitioner 1 has a O:E ratio that is close to 100%, which suggests that her patients are using about what would be expected based on their mix of diagnoses. While she would have been flagged as an outlier using the actual costs of her patients, when case mix is taken into account, her patients are actually using close to an average amount of services. This example illustrates the usefulness of the ACGs in protecting physicians with particularly sick patients against potentially inappropriate billing investigations.

Practitioner 2 has a relatively low volume and revenue practice. Despite that, he did not appear to have justification for the unusually high cost per patient based on the case mix of his patients. For the 451 patients who received the majority of their care from him, the observed total payments were 54% more than would be expected if each patient had been average for his or her respective clinical group.

Conclusions

In May 2001, the annual profile report issued to general practitioners uses ACG-based case-mix adjustment for the first time. The tables provided in this article are a simplified version of what was presented in the actual profiles.

Practice populations have significant differences in the needs for medical services and the amount of services needed to care for them. Without accounting for differential illness levels, there is the potential of flagging high-billing physicians for investigation simply because their patients are sicker than average. In applying ACGs to the profiling system, the MSP and BCMA committees are seeking to protect physicians from unnecessary and disruptive practice billing audits. ACGs not only account for the severity of individual diagnoses, but also the number and complexity of additional diagnoses those patients face. ACGs represent a significant advance compared to using age and sex as a proxy for illness. ACG-adjusted profiles also provide the ability for physicians to more fairly compare their resource-use profile to their peers.

Table 1. Comparison of patient populations and payments for two general practitioners identified as high-cost outliers, 1999.

* Includes all services grouped as "general practice" services in fee-for-service payment schedule.
† The practitioners' peer group is defined as practitioners deriving the majority of their revenue from services in the general practice "owned" items in the MSC Payment Schedule.
‡ "Majority source of care" patients are defined as those for whom the practitioner provides the majority of the GP services. In other words, these are the patients for whom the practitioner is most responsible.

Table 2. Performance of ACG versus age/gender statistical models to explain health care expenditures.*

* Figures represent adjusted R-square values from linear regression models with costs truncated at the 99th percentile.

Table 3. Comparison of case-mix adjusted practice profiles for two general practitioners identified as high-cost outliers in unadjusted analyses.

* "Majority source of care patients" are defined as those for whom the practitioner provides the majority of the GP services. In other words, these are the patients for whom the practitioner is most responsible.

† Includes all services grouped as "general practice" services in fee-for-service payment schedule.

‡ The practitioners' peer group is defined as practitioners who derive the majority of their revenue on general practice "owned" fee items in the MSC Payment Schedule

§ Expected costs are defined as the total cost of services if all patients consumed an average amount for their ACG.

** The ACG morbidity ratio is defined as the average expected per-person cost for the practitioner's panel of patients based on ACGs divided by the average expected per-person cost based on age/gender for all practitioners. It can be interpreted as how "sick" the practitioner's panel of patients is compared to average.

¶ The observed to expected cost ratio (or efficiency ratio) is defined as the total observed costs for a practitioner's panel of patients divided by the expected costs if all patients were average for their ACG. It can be interpreted as the costliness of a provider's practice after removing the effects of morbidity.

References

1.    Tucker AM, Weiner JP, Honigfeld S, et al. Profiling primary care physician resource use: Examining the application of case mix adjustment [see comments]. J Ambulat Care Manage 1996; 19:60-80. PubMed Abstract 
2.    Ruffin M. Physician profiling: Trends and implications. Physician Exec 1995;21:34-37. PubMed Abstract 
3.    Lasker RD, Shapiro DW, Tucker AM. Realizing the potential of practice pattern profiling. Inquiry 1992;29:287-297. PubMed Abstract 
4.    Menec V, Black C, Roos NP, et al. Defining practice populations: Methods and issues. Manitoba Centre for Health Policy and Evaluation Reports, February 2000. http://www.umanitoba.ca/centres/mchpe/reports/pdfs/roster.pdf (no date; retrieved 11 June 2001).   
5.    Reid RJ, MacWilliam L, Verhulst L, et al. Performance of the ACG case-mix system in two Canadian provinces. Med Care 2001;36:86-99. PubMed Abstract 
6.    Johns Hopkins University School of Hygiene and Public Health. The ACG Assignment Process. ACG Software Documentation and Users Manual (version 4.5). Baltimore, MD: Johns Hopkins University, 2000:27-46. 
7.    Weiner J, Starfield B, Steinwachs D, et al. Development and application of a population-oriented measure of ambulatory care case-mix. Med Care 1991;29:452-472. PubMed Abstract   

hidden

Lorne Verhulst, MD, MPA, Robert J. Reid, MD, PhD, and Christopher B. Forrest, MD, PhD

Dr Verhulst is a medical consultant at the Ministry of Health and Ministry Responsible for Seniors, Province of British Columbia. Dr Reid is an assistant professor at the UBC Centre for Health Services & Policy Research, and Department of Health Care and Epidemiology. Dr Forrest is an associate professor of Health Policy and Management and Pediatrics at Johns Hopkins University in Baltimore.