Monday, August 16, 2004
Expert witnesses and the problem of observer bias
Spoons points to this troubling article in the Chicago Tribune summarizing the results of a study published in the August issue of Academic Radiology. The study
The only thing surprising about this result is that the editors of Academic Radiology find the result "as disquieting as it is startling." The only people who would find this result "startling" are radiologists and judges. Everybody else understands that radiology -- which is the human interpretation of medical images -- is subject to tremendous observer bias.
Indeed, the problem of observer bias in radiology is so pervasive that the United States Food and Drug Administration requires that clinical trials dependant on radiological outcomes structure observer bias out of the trial.
So, for example, when a medical device company designs a clinical trial that depends upon the interpretation of medical images to measure outcomes, it uses blinded panels of radiologists -- generally two or three physicians per image -- to review each image of each patient. The trial's protocol will provide for some sort of adjudication in the event that the reviewing panel splits its opinion about the significance of an image. All of this is necessary because the FDA recognizes that there is an enormous amount of observer bias inherent in radiological interpretation.
If the government in its capacity as a regulator of the medical device industry recognizes that a single unblinded radiological interpretation -- even if not self-interested -- is inadequate to determine the effectiveness of a new medical device, why do the federal courts permit the introduction of result-oriented interpretations in personal injury cases?
Of course, you will never find the editors of an esteemed journal like Academic Radiology admitting that radiologists are trapped by observer bias, because to do so would shake our faith in their everyday practice of medicine. So the mandarins of the specialty attribute the bias revealed in this study to self-interest on the part of treasonous expert witnesses, which is easier than admitting that even the most honest radiologist will tend to see ghosts in the images if he is expecting to see them there.
So judges let in evidence that the FDA would not consider the least bit probative in other contexts, and the medical profession does not campaign for the obvious solution -- blinded panels of experts, rather than paid individual expert witnesses -- because admitting the problem of bias in the absence of venality would damage the self-image of radiologists.
casts considerable doubt on expert witness testimony in asbestos litigation. The study compared the findings of physicians who interpreted X-rays for plaintiffs in asbestos lawsuits with those of independent radiologists interpreting the same X-rays.
The study by Dr. Joseph Gitlin and Dr. Elizabeth Garrett-Mayer of the Johns Hopkins School of Medicine in Baltimore compared interpretations of 492 chest X-rays. Physicians working for the plaintiffs detected evidence of possible asbestos-related lung damage in 95.9 percent of them. Yet evidence of disease was detected in only 4.5 percent of the cases when they were reviewed by the independent radiologists.
The only thing surprising about this result is that the editors of Academic Radiology find the result "as disquieting as it is startling." The only people who would find this result "startling" are radiologists and judges. Everybody else understands that radiology -- which is the human interpretation of medical images -- is subject to tremendous observer bias.
Indeed, the problem of observer bias in radiology is so pervasive that the United States Food and Drug Administration requires that clinical trials dependant on radiological outcomes structure observer bias out of the trial.
So, for example, when a medical device company designs a clinical trial that depends upon the interpretation of medical images to measure outcomes, it uses blinded panels of radiologists -- generally two or three physicians per image -- to review each image of each patient. The trial's protocol will provide for some sort of adjudication in the event that the reviewing panel splits its opinion about the significance of an image. All of this is necessary because the FDA recognizes that there is an enormous amount of observer bias inherent in radiological interpretation.
If the government in its capacity as a regulator of the medical device industry recognizes that a single unblinded radiological interpretation -- even if not self-interested -- is inadequate to determine the effectiveness of a new medical device, why do the federal courts permit the introduction of result-oriented interpretations in personal injury cases?
Of course, you will never find the editors of an esteemed journal like Academic Radiology admitting that radiologists are trapped by observer bias, because to do so would shake our faith in their everyday practice of medicine. So the mandarins of the specialty attribute the bias revealed in this study to self-interest on the part of treasonous expert witnesses, which is easier than admitting that even the most honest radiologist will tend to see ghosts in the images if he is expecting to see them there.
So judges let in evidence that the FDA would not consider the least bit probative in other contexts, and the medical profession does not campaign for the obvious solution -- blinded panels of experts, rather than paid individual expert witnesses -- because admitting the problem of bias in the absence of venality would damage the self-image of radiologists.
0 Comments:
