Background: Between 2004 and 2007 the number of CT and MRI examinations per 1000 people in BC increased by approximately 50%, a finding that raised concerns about the appropriateness of the examinations. To address these concerns, the BC Ministry of Health commissioned a study of requests for CT and MRI in all health regions.
Methods: An independent company considered expert in analyzing medical imaging data was contracted to assess 2000 randomly selected CT and MRI requisitions issued across BC between 2010 and 2011. The requisitions were analyzed using a five-point rating scale for appropriateness that was based on the Canadian Association of Radiologists guidelines and a meta-analysis of other guidelines. A computer program rated each requisition according to the appropriateness scale. Subsequently, a subset of the requisitions was reanalyzed by two independent reviewers.
Results: In the computer analysis, 2% of the requisitions were rated “inappropriate,” while 46% were rated “indeterminate” because the computer program was not able to interpret the written portion of a significant number of requisitions. However, the two independent reviewers encountered no difficulties in assessing the appropriateness of each requisition in the data subset and found that only a small number of examinations were “indeterminate” and none were “inappropriate.”
Conclusions: The results suggest that physicians in all regions of BC are requesting CT and MRI in compliance with established guidelines. In the computer analysis, CT and MRI orders rated “inappropriate” amounted to only 2% of all examinations, a figure considerably lower than an often quoted but poorly substantiated 30%. While the level of appropriateness found in BC orders for CT and MRI is acceptable, continued monitoring is needed and would be facilitated by the use of computerized physician order entry.
Physicians in all regions of BC are requesting CT and MRI in compliance with established guidelines, according to a review of orders issued.
According to the Canadian Institute for Health Information (CIHI), between 2004 and 2007 the number of computed tomography (CT) and magnetic resonance imaging (MRI) examinations per 1000 people in British Columbia increased by approximately 50%. While these numbers can only be truly understood in the context of utilization in other provinces and other developed countries, the increase raised concerns about the appropriateness of these examinations.
Objective data concerning the appropriateness of medical imaging tests have rarely been published. A widely quoted estimate that as many as 30% of imaging examinations are inappropriate is not supported by published data.[2-7] One Canadian prospective study of ultrasound, CT, and MRI done by Butler and Stolberg found that only 2.5% were inappropriate.
The purpose of this study was to accurately and objectively determine the appropriateness of CT and MRI examinations ordered in all health regions in BC.
The Ministry of Health in BC contracted with an independent company considered expert in analyzing medical imaging data to determine the appropriateness of CT and MRI examinations in BC. Appropriateness ratings were determined using both Canadian Association of Radiologists (CAR) guidelines and a meta-analysis of other available imaging guidelines. Thirty-seven hospitals representing all health authorities in BC were included in the study.
Requisitions for imaging were randomly selected to obtain a representative sample of approximately 1200 CT and 800 MRI exams completed between 2010 and 2011. The samples were drawn from the most commonly requested CT and MRI examinations, specifically those for the brain and lumbar spine. In addition, samples of requisitions for MRI examinations of the knee were also acquired.
Information from the paper requisitions was encoded to capture patient and physician data, geographical data, and clinical indications. Attempts were made to convert the free text (narrative portion of order) that characterized the clinical indications on the majority of forms by using a coding system to derive ratings of appropriateness. After unreadable or duplicate requisitions were discarded, 1927 valid requisitions remained from the original 2000. A further 26 were removed because they were not for the most commonly requested examinations being studied, leaving a sample of 1901 unique diagnostic orders.
The remaining 1901 documents were assessed using a computer program that categorized each requisition according to the five-point rating scale for appropriateness shown in Table 1.
A subset of 325 requisitions representing all cases from one health authority (17% of the total) was then reviewed by hand by two independent analysts. Rating was done using the appropriateness scale shown in Table 1 as well as by a second scale derived from the Butler study. The results of each reviewer’s assessments were compared and any discrepancies discussed until agreement on a rating was reached.
A review of detailed data on the hospital and health region as well as the patient age and gender for each requisition confirmed that a representative sample was obtained for analysis. The results of the computer analysis summarized in Table 2 show that although only 2% of the imaging orders were deemed “inappropriate,” a large number were “indeterminate” (46%) or “not validated” (25%).
The subset analysis performed by the two reviewers using the Table 1 rating scale found that 5 of the 325 requisitions (1.5%) were “indeterminate,” and none were “inappropriate.” The results using the Butler scale were similar.
The study results indicate that physicians in BC appear to be adhering to clinical guidelines when ordering CT and MRI examinations, with only 2% of orders being deemed “inappropriate.” This finding is similar to the results of Butler’s prospective study of ultrasound, CT, and MRI in Ontario hospitals. Using slightly different methodology, Butler found a very similar 2.5% of “inappropriate” imaging orders.
Data analysis challenge
One challenge when analyzing data gathered for the study was converting the free text from the requisition forms so that the clinical reasons for ordering a test could be assessed by computer. The very high number of “indeterminate” ratings found in the analysis was a result of this challenge, and motivated both a human review of data as well as a more detailed analysis of the computer program by the software vendor. This analysis revealed problems with limited software vocabulary. For example, 94% of knee MRI exams were called “indeterminate,” even when a specific diagnosis such as “medial meniscal tear” was the provided indication. The “indeterminate” rating was thus frequently a reflection of the software’s inadequate vocabulary and consequent failure to recognize valid clinical reasons for ordering a test. The “not validated” cases were the result of situations where the information on the requisitions did not match coding terminology.
One limitation of this study was the restricted number of body regions for imaging considered: CT of the brain and lumbar spine and MRI of those two regions plus the knee. These examinations were chosen because they are among the most common, making up at least 60% of CT and MRI studies on a typical day in BC. Furthermore, these regions were selected specifically with the expectation that they would involve more “inappropriate” orders when compared to orders for imaging other body regions, where much more specific questions are often asked. We would expect, therefore, that including imaging requisitions for other body regions would yield the same very low number of “inappropriate” orders.
We could find no scientific support for the 30% “inappropriate” rate for CT and MRI quoted by some sources, and suspect that no evidence exists. However, given the discrepancy between this figure and the very low 2% “inappropriate” rate we obtained, we carried out a detailed literature search to determine the origin of this frequently quoted 30% figure.
The 2010 Health Council of Canada (HCC) publication mentions this high rate to encourage family physicians to be more selective when ordering diagnostic imaging. The two sources that HCC cites for this assertion include a brochure, which uses the 30% number but provides no references, and a report prepared for the government of Saskatchewan. The report mentions the 30% number, for which the authors provide two references. The first is a Canadian study of the use of plain film in the 1990s for investigating cervical spine injury and not for CT or MRI. Nowhere in this study is mention made of a 30% inappropriate rate, or any inappropriate rate. The second reference in the Saskatchewan report is an abstract from a Pennsylvania private insurance company that says they did a study but provides no details or references.
The 30% inappropriate rate is also quoted by Picano in a 2004 British Medical Journal article focused on radiation exposure. The references the Italian researcher uses are two articles published in Pediatric Radiology in 2002.[9,10] These articles both discuss radiation exposure in childhood and note that rates of CT scanning are high in the United States, but neither article discusses an appropriate or inappropriate rate of any sort, let alone mentions a specific number.[9,10] The often-quoted figure of 30% seems like the admonition to drink eight glasses of water a day; both ideas are accepted wisdom, but neither has any credible source and instead reflect mere repetition without scientific backing.
Putting BC data in context
In BC the low number of “inappropriate” ratings for CT and MRI requisitions may reflect restricted access to these technologies. With respect to CT in BC, the Ministry of Health determines the number and distribution of scanners for the entire province. As far as MRI is concerned, the ministry provides fixed annual funding to control the number of examinations performed.
According to CIHI data from 2008 to 2009, the utilization of MRI in BC (26 per 1000 population), as shown in Table 3, is the third lowest of all 10 provinces, just ahead of Newfoundland and PEI. CT utilization in BC (106 per 1000 population) is closer to average, but is still less than the Canadian average (121 per 1000 population) and is the second lowest of all 10 provinces, just ahead of PEI.
Considering Canada as a whole, our present use of CT and MRI per 1000 is approximately 50% of the US rate. On a more global scale, Canada’s utilization rates for CT and MRI are below the OECD averages, and BC’s use of MRI is 53% of the OECD average (49 per 1000 population).
The 2011 performance measure document for one of the principal regions of BC, the Vancouver Island Health Authority, shows that only 30% of patients received routine MRI exams within the target time of 91 days, and that some patients waited a full year for an MRI exam. Such delays can mean that inappropriate tests are ordered or a more invasive procedure is done.
An example of the former is the examination of a young patient with inflammatory bowel disease using CT rather than MRI—the test of choice with its lack of ionizing radiation. An unfortunate example of the latter is the use of knee arthroscopy without an MRI examination first to determine the cause of the knee pain and establish the need for arthroscopy.
While the level of appropriate imaging found in BC is acceptable, continued monitoring is needed. This would be helped by further development of computerized physician order entry to improve access to clinical information and make data more readily available for computer analysis. Computerized physician order entry could also benefit communication between clinicians and radiologists and result in improved interpretations relevant to patient care.
1. Canadian Institute for Health Information. Medical imaging in Canada, 2007. Ottawa, ON:CIHI, 2008.
2. Health Council of Canada. Decisions, decisions: Family doctors as gatekeepers to prescription drugs and diagnostic imaging in Canada. Toronto: Health Council of Canada. Accessed 2 November 2012. www.healthcouncilcanada.ca/tree/2.33-DecisionsHSU_Sept2010.pdf.
3. Canadian Association of Radiologists. Do you need that scan? Ottawa, ON: CAR; 2009. Accessed 2 November 2012. http://www.car.ca/uploads/news%20publications/car_cat_scan_eng.pdf.
4. ProMed Associated Ltd. Report to the Saskatchewan Department of Health for a province-wide diagnostic imaging review and framework for strategic planning. Regina, SK: Saskatchewan Department of Health; 2004. Accessed 16 November 2012. www.health.gov.sk.ca/diagnostic-imaging-review.
5 Stiell IG, Wells GA, Vandemheen K, et al. Variation in emergency department use of cervical spine radiography for alert, stable trauma patients. CMAJ 1997;156:1537-1544.
6. Dehn TG, O’Connell B, Hall RN, et al. Appropriateness of imaging examinations: Current state and future approaches. Imaging Economics 2000;(Mar/Apr):1821.
7. Picano E. Sustainability of medical imaging. BMJ 2004;328:578-580.
8. Butler GJ, Stolberg HO. The use of cross-sectional diagnostic imaging in a major Canadian health care centre. Can Assoc Radiol J 2004;55:174-177.
9. Hall EJ. Lessons we have learned from our children: Cancer risks from diagnostic radiology. Pediatr Radiol 2002;32:700-706.
10. Ron E. Ionizing radiation and cancer risk: Evidence from epidemiology. Pediatr Radiol 2002;32:232-237.
11. Tsindos S. What drove us to drink 2 litres of water a day? Aust N Z Public Health 2012;36:205-207.
12. Canadian Institute for Health Information. Use of MRI and CT exams varies greatly among provinces. Accessed 16 November 2012. www.cihi.ca/CIHI-ext-portal/internet/en/document/types+of+care/specialized+services/medical+imaging/release_22jul2010.
13. Vancouver Island Health Authority. VIHA performance measures: MRI wait times, December 2011. Accessed 16 November 2012. www.viha.ca/NR/rdonlyres/6F46C5CC-6EF4-4BFD-AA8F-FBFA031916E1/0/mri_wait.pdf.
Ms Eddy and Mr Ednie are third-year medical students at the University of British Columbia, Trail Integrated Community Clerkship Program. Ms Connell is a student at Queen’s University in Kingston, Ontario. Dr Eddy is a radiologist in the Vancouver Island Health Authority in Victoria. Mr Eaton is director of Diagnostic Imaging in the BC Ministry of Health. Dr Mathieson is medical director of Medical Imaging at Vancouver Island Health Authority.
Above is the information needed to cite this article in your paper or presentation. The International Committee
of Medical Journal Editors (ICMJE) recommends the following citation style, which is the now nearly universally
accepted citation style for scientific papers:
Halpern SD, Ubel PA, Caplan AL, Marion DW, Palmer AM, Schiding JK, et al. Solid-organ transplantation in HIV-infected patients. N Engl J Med. 2002;347:284-7.
About the ICMJE and citation styles
The ICMJE is small group of editors of general medical journals who first met informally in Vancouver, British Columbia, in 1978 to establish guidelines for the format of manuscripts submitted to their journals. The group became known as the Vancouver Group. Its requirements for manuscripts, including formats for bibliographic references developed by the U.S. National Library of Medicine (NLM), were first published in 1979. The Vancouver Group expanded and evolved into the International Committee of Medical Journal Editors (ICMJE), which meets annually. The ICMJE created the Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals to help authors and editors create and distribute accurate, clear, easily accessible reports of biomedical studies.
An alternate version of ICMJE style is to additionally list the month an issue number, but since most journals use continuous pagination, the shorter form provides sufficient information to locate the reference. The NLM now lists all authors.
BCMJ standard citation style is a slight modification of the ICMJE/NLM style, as follows:
- Only the first three authors are listed, followed by "et al."
- There is no period after the journal name.
- Page numbers are not abbreviated.
For more information on the ICMJE Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals, visit www.icmje.org