September 21, 2015

Everything I Do is Wrong: The Danger of 100-Percent Error Rate Audits


“Well, I can't get along no matter what I do
Well, we can't get along no matter what I do
Yes, everything I do is wrong, no matter what I do”
-B.B. King – “Everything I Do Is Wrong”

“Women may have many faults, but men have only two: everything they say and everything they do.”-Anonymous

In any given year, I am engaged as a statistical expert in some 75-plus government and private payer audits. My job, in most cases, is to validate the accuracy of the statistical models used by the government to create their samples, gauge point estimates and error, and when taking part in an extrapolation audit, verify that the calculation used to extrapolate the damages is done correctly.  While in almost every audit I find that the auditor committed statistical mistakes, it has been rare that they find that 100 percent of claims were billed in error. But this has not been the case recently.

Since the beginning of this year, I have been the statistical expert on nearly a dozen audits for which the error rate was deemed by the auditor to be 100 percent. That means that, of the sample selected by the auditor, it determined that 100 percent of the claims submitted were submitted in error, for one reason or another. Maybe they didn’t feel that the documentation was sufficient, or that the services didn’t meet criteria for medical necessity. For many claims, the error was determined to be the result of an administrative issue; for example there wasn’t any signature in the file, or it was provided in the wrong place, or the procedure was billed out by the wrong provider. Irrespective, a finding of 100-percent incorrect billing result has some serious consequences.

For one thing, if the audit results in an extrapolated overpayment estimate, it is quite likely that the calculation will be based on a proportion of error, which is quite common here. In the case of a 100-percent error rate, this presents some complicated issues. Most often, the government relies upon the Wald method, which works poorly when the sample size is less than 100 or the proportion is either very high or very low. In the most common scenario, these audits consist of 30 audit units if non-stratified and rarely more than 90 even when stratified. The confidence interval calculation is important because the government most often bases its overpayment demand amount on the lower boundary of a one-sided, 90-percent confidence interval. For example, in an audit situation in which the finding is 30 denied claims out of 30 examined claims, the lower boundary is still 100 percent using the normal approximation, but it goes down to 94.78 percent using an exact calculation. This may not seem like much, but in one example, for which the extrapolation was based on $11.4 million in total payments, the difference would have been well over $500,000. To work with a 100-percent overpayment determination and a sample size of under 100, the auditor should rely upon an exact test, such as the Blythe-Still-Cassella, the Clopper-Pearson, or even the LePlace method. As you can imagine, this is a much harder point to argue to either a qualified independent contractor (QIC) or an administrative law judge (ALJ) since they are already a bit gun-shy when it comes to statistics.

Another problem with a 100-percent error finding is that the overpayment amount correlates 100 percent to the payment amount, making it harder to argue the variable of interest and the precision of the findings. Section of the Program Integrity Manual reads like so:

“Other methods of obtaining the point estimate are discussed in the standard textbooks on sampling theory. Alternatives to the simple expansion method that make use of auxiliary variables include ratio and regression estimation. Under the appropriate conditions, ratio or regression methods can result in smaller margins of error than the simple expansion method. For example, if, as discussed earlier, it is believed that the overpayment for a sample unit is strongly correlated with the original paid amount, the ratio estimator may be efficient. The ratio estimator is the ratio of the sample net overpayment to the sample total original payment multiplied by the total of original paid dollars in the frame. If the actual correlation between the overpayment and the original paid amount is high enough, greater precision in estimation will be attained, i.e., the lower limit of the one-sided 90 percent confidence interval will be closer to the point estimate.”

In most cases, the use of a ratio estimation method will result in a higher estimate than the traditional expansion method, so reducing the denial rate as much as possible, which is accomplished by appealing the findings on a claim-by-claim basis, is often far more effective than challenging the statistics. True, the sample still can be challenged if it is found not to be representative of the universe, which is what I have often found in these cases, but it really limits your ability to use all the tools that otherwise would be available to defend your position.

In my opinion, the bigger problem is when the audit is not part of an extrapolation because the practice chooses just to pay the estimated overpayment amount and move on. Again, in my opinion, no matter how big or small the amount, this is a huge mistake, because what you have done in essence is admit that 100 percent of everything you have done in the past was billed improperly. What likely follows is a Zone Program Integrity Contractor (ZPIC) audit, which can include extrapolation and the possibility of civil penalties at the claim level. This consequence can turn an audit costing several thousand dollars into a several-million-dollar overpayment demand.

Here is the wording from an audit on which I recently worked:

“The ‘ratio’ is calculated by dividing the sample net overpayment amount by the sample total original payment amount for each stratum. The ‘ratio’ for Stratum 1 is .9451 (94.51 percent). The ‘ratio’ for Stratum 2 is 0.9714 (97.14 percent). The overpayments are highly correlated with the original paid amounts; therefore, the ratio estimator will result in better precision.” 

Simply put, a 100-percent overpayment rate is a statistician’s nightmare. The bottom line is this: it is highly improbable that any organization does everything wrong all the time. It just defies common sense, yet it seems to be happening more and more. Since my experience is that the auditors do something wrong nearly 100 percent of the time (notice that I didn’t say 100 percent of the time), it is wholly incumbent on any organization that is found to have a 100-percent error rate to hire their own auditors to review and contest the findings during the appeals process. Failing to do so is one way to almost guarantee follow-up audits and more severe consequences. Think about it: if you really are doing everything wrong, just how devastating an outcome would that produce?

And that’s the world according to Frank.

About the Author

Frank Cohen is the director of analytics and business intelligence for DoctorsManagement. He is a healthcare consultant who specializes in data mining, applied statistics, practice analytics, decision support, and process improvement. Mr. Cohen is also a member of the National Society of Certified Healthcare Business Consultants.

Contact the Author

Comment on this Article

This email address is being protected from spambots. You need JavaScript enabled to view it.