Audits and Auditors: A Step in the Right Direction

I have often written (and whined) about extrapolation, and how it can be both a blessing and a curse. For example, if the process is meticulous – meaning that the sample frame, sample, and extrapolation methods and calculations are appropriate – then it is a blessing in that it can save a practice hundreds of hours and thousands (if not tens of thousands) of dollars that otherwise would have been devoted to pulling, duplicating, and providing charts and other written documentation to the auditor. 

But if not done correctly, meaning that the sample frame, sample, and extrapolation are not properly prepared, it can be a curse. And that is because extrapolation will exaggerate any error to often huge proportions.

Here is an example from a recent case in which I was engaged as the statistician. The auditor pulled six years’ worth of data, which amounted to over 120,000 claims. The sample size was 45, which appeared to be appropriate, based on accepted methods. The audit was conducted, and it was determined that the practice was overpaid $3,822, based on that sample of 45 claims. That comes out to an average of $84.93 per claim. If you multiply this by the 120,000 claims in the universe (which is how extrapolation works), this results in an extrapolated overpayment estimate of $10.2 million. That’s a lot of money! 

In this case, the auditor relied upon the lower boundary of a two-sided, 90-percent confidence interval; or at least, that is what was in their documentation. This resulted in an extrapolated overpayment estimate of $8.87 million: still a lot of money. The problem is that the auditor used a one-sided, 90-percent confidence interval, not a two-sided one, and this resulted in an overestimate of $6.22 per claim. Apply that to the universe and now you have an overstated overpayment estimate of three-quarters of a million dollars. 

Unfortunately, at least in my expert opinion, auditors commit fatal flaws all the time – and I guess that is why I am so busy with post-audit extrapolation mitigation engagements. But sometimes, the problem goes beyond a statistical or computational error. Sometimes the problem has to do with policies, guidelines, and standards, which are always up for grabs. For example, Chapter 8 of the Medicare Program Integrity Manual (PIM) (100-08) is titled “Administrative Actions and Sanctions and Statistical Sampling for Overpayment Estimation.” Excluding the table of contents and that transmittal table at the end, the chapter is 46 pages long (or short, depending on how you look at it). But of that, only some nine pages are focused on the actual statistical sampling and extrapolation guidelines. Nine pages! Not a very thorough treatise, to be sure.

In fact, perhaps the most relied-upon and most-cited of all texts in this area is called “Sampling Techniques; Third Edition” by William Cochran, former professor emeritus of statistics for Harvard University and a world-famous statistician. Not counting the table of contents in this book, there are nearly 400 pages that deal solely with the issue of sampling (so one can just imagine how inadequate those nine pages in Chapter 8 of the PIM are). Another issue to consider is that Chapter 8 is not codified, meaning that it is not organized into some adherent structure, like a statute or a law. In essence, when you go before an administrative law judge (ALJ) to appeal one of these audits, the judge often relies upon those guidelines just as if they were, in fact, law. And they are not. And in my experience, the ALJ will sometimes do this to the exclusion of standards of statistical practice – whatever that means. 

One of the key related issues, and actually the purpose of this article, has to do with how underpayments are handled. Throughout Chapter 8 of the PIM, the word “underpayment(s)” is mentioned four times. The first time is in section 8.2.4, under the heading “Coordination with Audit and Reimbursement Staff,” and the language simply states that “overpayments and underpayments are accurately determined and reflected in the provider’s cost report.” The next mention is in 8.4.4.4.4, with the heading “Overpayment/Underpayment Worksheets,” where it states that underpayments should be appropriately documented. In 8.4.1.3, under the heading “Steps for Conducting Statistical Sampling,” it says, under No.  7: “examining each of the sampling units and determining if there was an overpayment or an underpayment.” But perhaps the most important mention is in section 8.4.5.2, under the heading “Calculation of the Estimated Overpayment Amount.” Here, the guidelines state that “sampling units that are found to be underpayments, in whole or in part, are recorded as negative overpayments and shall be used in calculating the estimated overpayment.”

Sounds great, but there are three problems involving this process that I see on a regular basis. First, it is very rare that the auditor reports an underpayment. In fact, I have only seen it maybe a handful of times out of the last 300 audits in which I was the statistical expert. And while one might expect there to be fewer underpayments than overpayments, it shouldn’t be that rare. Even the Comprehensive Error Rate Testing (CERT) study reports underpayments. But for some codes, it is quite high. For example, CERT reports that the 99212 is estimated to have been underpaid 9.8 percent of the time. 99231 is estimated to have been underpaid 9.5 percent of the time, and 99283, a mid-level ED visit, is reported to have been underpaid 5.9 percent of the time. So why don’t we see this come up more often? It is because the auditors rarely, if ever, audit claims that contain codes that are reported or known to be underpaid. And the reason for that should be obvious; how the contract gets paid is often tied to how much they are able to recover, and if you have to net out the underpayments with the overpayments, that reduces their take.

The second problem has to do with the restriction in the Chapter 8 guidelines to audit only those claims that have been paid. Under section 8.4.3.2.1 (Composition of the Universe), under section B., it states that “the universe shall consist of all fully and partially paid claims submitted by the provider/supplier for the period selected for review and for the sampling units to be reviewed.” In essence, the auditor is avoiding (or not including) zero-paid claims. So, at first, this feels like it makes sense, right? If a claim has not been paid, then obviously, there isn’t any reason to audit it, since there isn’t any amount of money to recoup. But think about this for a moment: which claims do you think are most likely to be underpaid – claims that have been paid some amount, or claims that have been paid nothing? The answer is the latter. Zero-paid claims have the greatest likelihood of being underpaid, since they are paid, well, nothing. I used to always object to this during an appeal hearing, but it almost always fell on deaf ears. 

The third problem, and perhaps the most frustrating, is when there are underpayments and you can’t get the judge in an ALJ hearing to recognize that these should be netted out against the overpayments. I mean, it says so in the guidelines, but the auditor will often either exclude these from the calculations or argue against it at the hearing. So in Volume 30, Number 25 – July 12, 2021, the “Report on Medicare Compliance” weekly newsletter started out with an article that dealt directly with this latter situation. The opening paragraph states that “in recent developments that could potentially affect overpayment findings, an administrative law judge (ALJ) invalidated a Medicare auditor’s statistical sampling method because it removed underpayments, and the chief statistician for a Medicare Administrative Contractor (MAC) came to a similar conclusion in an unrelated appeal. If their point of view catches on, extrapolated overpayment amounts may be smaller in some cases, experts say.”

I was so excited when I read this, because while one ALJ’s ruling is not necessarily binding on another ruling, it does in the very least set some degree of precedent. And perhaps most notable about this is that it wasn’t just the plaintiff’s statistical expert that objected to this. It turns out that the MAC’s chief statistician agreed. In fact, the consensus was that those zero-paid claims should be included in the sample size calculation and sample selection. The potential downside, however, is that it is possible that including zero-paid claims could cost a practice more money in the long run. How? Well, let’s say that, using our example above, there were 10,000 zero-paid claims, with four of those in the sample. If those four did not produce any negative impact, meaning that they were not underpaid, then those 10,000 claims that were excluded in the initial calculation would now be included in the extrapolated estimate. If we kept the average of $84.93 per claim, this would have resulted in an additional $849,300, based on the extrapolated estimate. 

In the end, one should understand that extrapolation always presents some risk, but the use of extrapolation is well-established within the courts, and trying to argue against it is just kicking against goads, making for sore feet and no progress. And depending upon the contractor, one has to be prepared for a battle. For example, according to the article, “audits conducted in connection with OIG corporate integrity agreements (CIAs) no longer take underpayments into account.” And that makes sense, from their perspective. After all, they are only interested in overpayments. And perhaps one of the biggest barriers to equity is that you cannot extrapolate underpayments (by themselves), only overpayments (or net of overpayment/underpayment).  he argument is that if you extrapolate underpayments, you likely will be paying for claims that should not be paid, which is in violation of the Social Security Act, section 1862(a)(1) (I think). 

So, what is one to do? In any type of a government or private payor audit, the provider should have their own auditing experts review the claims audited by the contractor or auditor. And if that independent person determines that there were underpayments, then the provider should be prepared to argue their inclusion during the appeals process. In any case, at least to me, this is a huge step forward in progress for providers’ rights – and being a huge provider advocate, I couldn’t be more pleased.

I conclude with this quote from Doug Baldwin, perhaps the one of the greatest wide receivers in Seattle Seahawks history: “change is inevitable, change will always happen, but you have to apply direction to change, and that’s when it’s progress.”

And that’s the world according to Frank.