There’s so much excitement about big data and the potential for it in work comp that it would be all to easy to forget some basics.
The law of small numbers chief among them.
Work comp accounts for a tad more than one percent of US medical spend – $30 billion in comp medical vs $2.8 trillion in total medical spend in 2012.
Many docs treat just a couple of work comp claims a year, and those who do handle a lot of WC claims see a wide range of injuries: knees, ankles, backs, shoulders, hands damaged by cuts, sprains strains and severe trauma. When looking to compare providers – or procedures for that matter – researchers need enough data points to develop a statistically-valid sample set. In most cases, no single provider has enough claims to enable clear-cut evaluation. And, if they do, there aren’t any other providers in their service area with the necessary volume, making comparisons nigh-on-impossible.
The issue is statistical validity and statistical accuracy. Simply put, is the measurement procedure capable of measuring what it is supposed to measure. Without enough data, there just isn’t enough information to accurately assess performance.
That’s not to say researchers can’t do very meaningful and helpful analyses; the one just published on opioid prescribing by physicians dispensing docs to work comp claimants is a perfect example; the ongoing research by CWCI, WCRI, and NCCI provide plenty of additional examples.
The problem occurs when consultants, payers or managed care firms try to make definitive statements about individual providers based on inadequate data. In my experience, provider rankings are often – but not always – based on little more than reimbursement or “savings” figures, and in no way account for “quality” measured by return to work, disability duration, cost-per-claim. There isn’t enough data to case-mix adjust, not enough data to make comparisons, or really “rate” docs.
I would note that some payers, most often state funds, and some managed care firms, notably MedRisk (HSA consulting client) have a wealth of data and can (and do) make valid comparisons.
What does this mean for you?
Beware of rankings, ratings, and comparisons of individual providers. Unless the underlying data is robust.