OK, let’s agree that the best metrics to measure workers’ comp claims are:
- speedy and
- sustained recovery,
- return to work, and
- medical cost.
- reductions below billed charges, or
- discounted medical costs, or
- network penetration, or
- litigation rate, or
- case management “impact”, or
- successful denials of treatments.
The latter are process measures – NOT outcomes. Yes, process can drive outcomes – but often does not. (See discussion of mythical “savings” from WC bill review and PPO)
How to evaluate a program, vendor, network, provider group, or delivery system: First, what types of information are relevant and for what? (we’ll get to benchmarks in a bit)
or…anecdote vs data, and treatment records vs, aggregate reports.
from Statistically Funny
- Using the outcomes metrics listed above, review their reports – aggregate reports that is – across their patient population. This will tell you two things; the entity’s overall “results,” how they think about outcomes, and what metrics they use.
- Ask a lot of questions, such as:
- Who selected these metrics, and what is their relevant experience, education and training?
- Why did you choose those metrics?
- Explain each metric, how it is helpful, its limitations, and the research you conducted to elect that metric.
- What other metrics did you consider? Why didn’t/don’t you use them?
- What is the statistical validity of your data?
- How do our results compare to the rest of your customers?
- Why are they similar/different?
- What would you like to include but didn’t, and why not?
- How are other customers using this information?
- How are you using this information?
- How do you see your outcomes measures evolving in the future?
Be prepared to be underwhelmed…and do NOT allow your vendor to provide case-specific reports or findings instead of aggregate data.
Vendors love to cite specific cases, which a)distracts you from their overall impact and b) is way more interesting as it is “real” and not just a bunch of numbers and statistics.
That’s NOT to say those specific case notes/treatment records aren’t helpful..in fact the next step – once you’ve decided the vendor produces good “outcomes” – is to find out how they do that.
And that is where and when case notes/treatment records can be quite useful…they show HOW the vendor delivers the outcomes.
So how do we know if outcomes are “good”, that is, if injured workers are being treated appropriately and employers/taxpayers are well served?
Enter an industry bugaboo – comparisons to meaningless/wrong/misleading “benchmarks.”
That, dear reader, is the subject of the next post.
What does this mean for you?
The mistake most work comp payers make is skipping the first – and by far the most important step – evaluating vendor performance based on REAL outcomes.