Erroneous Ratings Data: Part II
In the prior issue we discussed our national
study of 2100 impairment rating reviews; 80% of all ratings
reviewed were erroneous, with 89% of the erroneous ratings
being elevated. In this issue we will present more of the
results; this information is critical to the work you
perform.
Each physician’s report was rated for quality
on a scale of 4 – excellent, 3 – good, 2 – fair, 1 – poor and
0 – unacceptable. A rating of “good” reflects adherence to
standards defined in the Guides and fair reflects adequate
information to assess the case, however deficits were present.
The mean scores for all physicians were: history 2.2 (fair),
examination 2.1 (fair), clinical analysis 2.1 (fair),
application of criteria 1.5 (between poor and fair).
Comparison to criteria was consistently the weaker aspect of
most reports. Review of the data suggests that rating
physicians typically demonstrate consistent performance with
rating, either consistently producing accurate or inaccurate
ratings.
Of all the erroneous reports, 90% of the Fifth
Edition ratings were higher than appeared appropriate. The
corrected value for a permanent impairment rating averaged
less than half of the original rating. For all Fifth Edition
whole person ratings the mean original rating was 14.2% whole
person permanent impairment and the mean expert rating was
6.2% whole person permanent impairment. Examining the subset
of the eighty percent of Fifth Edition whole person ratings
that were erroneous, the rating by the original physician
averaged 15.5% (standard deviation of 9.4%) whole person
permanent impairment and upon re-rating the corrected rating
averaged 5.6% (standard deviation of 14.3%) whole person
permanent impairment, with a correlation coefficient of
0.54.
For Fourth Edition whole person ratings the
mean original rating was slightly greater at 18.4% whole
person permanent impairment and the expert rating was also
higher at 9.0% whole person permanent impairment. Of all Fifth
Edition erroneous reports where ratable permanent impairment
was originally identified, upon expert re-rating 37% were
found to have no impairment; in Florida 76% previously rated
as having impairment were found to have no impairment.
Examining the 839 Fifth Edition whole person
erroneous ratings, 7% of the cases (61) were under- rated,
i.e. based on the clinical information provided the rating
should have been higher. With these cases the mean original
rating was 9.9% whole person, however it should have been
16.2% whole person. Thirteen of the erroneous ratings had the
same numeric value when rated, however the rating process used
by the physician was incorrect and the similar results were
coincidental.
Of 850 Fifth Edition cases that were found
either through critique or review of records to have
impairment, 90% of the cases had impairment less than 22%
whole person. Less than 5% of all ratable cases have
impairment greater than 30% whole person permanent
impairment.
For all regions the original ratings were
rated higher than what appears appropriate based on the
medical information provided. The error rate was consistent
among regions with approximately 80% error rate, with an
exception for multiple regions of the spine with the error
rate being 90%. The actual mean expert rating of all regions,
with the exception of thoracic spine, was between 3% and 8%
whole person permanent impairment, with an overall mean of 6%
whole person permanent impairment. Many of the thoracic cases
involved corticospinal tract involvement and therefore were
associated with greater impairment. The data suggests that are
higher ratings are seen for older, male patients and for older
injuries.
In summary, the analysis of the 2100 reviews
reveals that the vast majority of AMA Guides ratings are
erroneous and substantially higher than appropriate. The goal
of the Guides is to provide consistent ratings, therefore
reducing conflict. The Guides state “Two physicians, following
the methods of the Guides to evaluate the same patient, should
report similar results and reach similar conclusions.” (AMA
Guides, p. 17) Yet, review of this data reveals that this is
often not achieved.
In the next issue of the ezine we explore the
common reasons for erroneous ratings.
