Publication

OIG RAT-STATS: Response Strategies to Government Audits

Cornelia Dorfschmid | April 2010

The Centers for Medicare and Medicaid (CMS) are now combating provider fraud, waste, and abuse through nationally coordinated strategies and new or newly aligned contractors. These are armed with new data analytics, pattern recognition methods, and analysis tools. They also are mandated to apply statistical methods and practices. Providers will have no choice but to respond in a similarly sophisticated manner and proactively analyze and data mine claims, and when hit as an audit target, respond with like sophistication and expertise.

On the Medicare side, the Medicare Administrative Contractors (MAC) will process both Part A and B claims, and the seven new Zone Program Integrity Contractors (ZPICs) take over the role of the Program Safeguard Contractors (PSCs) and focus on fraud. Four permanent Recovery Audit Contractors (RACs) are now fully operational in all states and equipped to detect on a large scale payment errors in Medicare claims. On the Medicaid side, the new and active Medicaid Integrity Contractors (MICs) will launch the next wave of enforcement through national contractors.

Providers need to step up efforts

In light of these developments, providers need to step up to the challenge and formulate a response strategy with proactive steps that aim to prevent becoming a government audit target. As soon as providers are hit as a healthcare audit target, they need to be prepared with a plan to act at once. This means assembling a team at the ready and coordinating the various requests for documents and information materials, but also being able to handle a recovery demand letter, if received.

A well-organized response to any demand letter that asks for repayment of a certain amount (calculated on the basis of sampling) will benefit providers the most, if it includes and develops a technical response that applies:

• statistical expertise in sampling, OIG RAT-STATS (a statistical package recommended by and available from the OIG [1]) and statistical formula;
• a clinician’s or Health Information Management (HIM) professional’s expert judgment on clinical issues and medical necessity; and
• a regulatory analyst’s and/or legal expert’s assistance in interpreting Medicare/Medicaid regulations and coverage rules, if necessary.

Government auditors, in many cases, rely on sampling and the use of RAT-STATS. CMS contractors are also held to the rules of a “fair” game using sampling. Not just any sampling will do; it must be statistically valid random sampling (SVRS), which is inherently fair and replicable. Hence, no recovery amounts should be requested, unless they are based on a fair and professionally documented process that allows for verification by an independent and knowledgeable party; otherwise they can be challenged.

Both statistical and clinical documentation requirements matter

A demand letter requesting repayment of claims payments usually states or points to a document that includes the objective and reasons for audit, the estimated amount, confidence levels, methods, criteria, and findings. In the early stages of the response, the provider and their inside and/or outside counsel typically develop a response that challenges the findings, and they search for additional medical documentation to support the validity of the individual claims that are referenced as part of the audit.

Documentation, or medical expertise, is sought to justify reasons why the particular claims should have been paid. However, one also needs to keep in mind that whenever sampling is the basis for the estimate, the audit target (i.e., provider) should be afforded the opportunity to verify the validity of the sampling and sampling methods. And the provider should take it.

It is most important that the sample is valid in order for it to support any estimation or projection. All too often, this aspect of verification is set aside in the early phases of the defense strategy, and the immediate focus becomes re-analyzing the claims for medical necessity or compliance with other documentation and coverage requirements. This may simply be due to the fact that looking into the statistics and estimation methods is outside the provider’s or attorney’s comfort zone and expertise, and therefore, initially given lower priority.

However, this may be a strategic mistake. A simple rule to remember is that if a sample is not valid, any projection is invalid, and the repayment is not owed, with good reason, and would need further evidence. If validity cannot be confirmed, there is no justification for the recovery amount projected to a claims universe. Hence, all that remains are the individual, sampled claims and the payments for those. Questionable sampling and insufficient documentation that disallows for verification of the statistical part of the government audit are immediate grounds for rebuttal and redetermination. With this mind, a solid response to a demand letter requesting recovery of overpayments can be separated into a three-pronged approach:

• Assess the statistical validity of the audit. Assess the validity of the random sampling method and estimation technique
• Assess the validity of the sample actually drawn. Assess the criteria and characteristics applied against clinical and documentation requirements for compliance with coverage rules of the audit.
• Assess the overpayment estimate, if the sample is valid. Re-estimate, if the sample is valid, but there is disagreement on the clinical and regulatory criteria applied.

Size versus validity – Validity is critical

In the many years that I had the opportunity to assist providers and attorneys in statistical and claims audit matters, the question I was asked most often is the one least relevant: How big does the sample have to be? The confusion between “validity” and “size” of the sample is common and understandable. However, what matters is validity, and validity must be the focus when developing a counterargument. A large sample can be invalid and no reliable statistical conclusions can be drawn from an invalid sample for the universe of underlying claims.

On the other hand, even a very small sample, if valid, can allow for inference with respect to the overpayments in the universe as a whole. A small sample may not render an estimate of overpayments with a high confidence level and precision, but still can be an unbiased and valid estimate. And, a valid small sample can be amended to a larger sample and the estimate thus improved (i.e., “cured”) to the confidence level needed.

An invalid sample, no matter what size, cannot. With respect to sample size, a general rule of thumb to remember is that a sample of 30, often referred to as a “probe” sample, is usually considered the minimum size in assessments, for reasons beyond the scope of this article. And like any other sample, a probe sample can be a valid sample, if properly assembled. What matters in a response to a government audit and needs to be answered affirmatively is whether the sample was valid.

RAT-STATS and probability sampling

CMS mandates that the sampling methodology to project overpayments must be reviewed by a statistician or person with equivalent expertise in probability sampling and estimation. A “probability sample” is required by CMS, and statistical expertise is needed to ensure that a statistically valid sample is drawn.[2] One feature of probability sampling is that, when properly executed and hence valid, the level of uncertainty (i.e., confidence interval) can be incorporated into the estimate of overpayment. OIG requires a 90% confidence level at 25% precision for the “point estimate,” and 90% is also the confidence level required for the “lower bound” limit estimate often used by CMS contractors.

To generate a probability sample, CMS recommends that RAT-STATS is used, or a similar package of statistical software tools, designed to assist in selecting random samples and evaluating the audit results. SPSS and SAS [3] are similarly reputable packages that may be used for generating a sample. RAT-STATS is recommended by OIG and by CMS for PSCs, ZPICs, and contractor Medical Review (MR) units. It is, therefore, most likely referenced in the supporting documentation that comes with the demand letter; and hence, an expert in using this package would be a good addition to the provider’s response team.

The statistical expert can, early on in the process, assess whether the documentation provided by the government auditor supports the overpayment estimate (i.e., whether he/she can replicate the estimation steps). A professionally documented report by the government auditor should make this straightforward. However, if documentation is incomplete or inconsistent, which happens, it provides immediate grounds for rebuttal and challenge (i.e., appeal).

Beyond the statistical tools used and the random numbers generated with such tools, it is also important to understand the various estimates presented by a government auditor. This is especially true for the difference between the “point estimate,” which is the unbiased amount of the overpayment dollars in the sample when expanded to yield an overpayment figure for the universe, and the estimate for the “lower limit” the 90% confidence interval of the overpayment estimate.

Although the government contractor can ask for the “point estimate” to be repaid, in most situations, the “lower limit” of the one-sided 90% confidence interval shall be used as the recovery amount, which is in the provider’s favor. The provider’s strategy should incorporate these distinctions in any settlement discussions. Note: When considering self disclosure to a MAC or the OIG, the lower limit should also be part of the disclosure strategy and settlement negotiations, as it is to the advantage of the provider.

Steps in a statistical response

More specifically, the following steps should be taken to develop a statistical response and verify the approach behind the recovery amount demanded, when based on sampling:

1. Don’t assume the statistical portion of the government audit is necessarily correct and justifies the recovery amount.
2. Request the sampling plan to ensure the government contractor had a sampling plan when designing the sample.
3. Request the random numbers and information on the random number generator used.
4. Ensure the random sample numbers can be regenerated and matched with the selected claims (i.e., the sample used was a valid probability sample).
5. Ensure the claims universe is properly assembled and consistent with the sampling frame.
6. Verify that the confidence interval, point estimate, and upper and lower bounds of the overpayment estimate can be confirmed using RAT-STATS estimation.
7. Verify that the lower limit estimate of 90% confidence level was calculated and can be confirmed using an appropriate statistical formula for variable sampling.
8. Assess whether the lower limit is the basis of the recovery amount demanded, and if not, aim for this instead of the point estimate in any settlement negotiations.
9. If the sample is found invalid for reasons of lacking documentation or evidence of mismatches and inconsistencies, consider drawing your own statistical sample to self assess and get a sense of what the underlying claims overpayment risk really is.
10. If the government’s sample is found valid and confirmed, examine claim by claim.
11. Reanalyze claim by claim using an independent auditor who has appropriate credentials and credibility. If the overpayment analysis on a sample claim by-claim analysis refutes the government auditor’s sample results, re-estimate using RAT-STATS for the point estimate and appropriate statistical formula for the lower limit.
12. Ensure that any response to a demand letter that is based on sampling incorporates statistical aspects and expertise.

Conclusion

These steps can serve as a starting point. However, as mentioned above, a solid and comprehensive response to a government audit demand letter requires the targeted organization to have a well-defined plan of action. The plan should be a team effort, with a team that is ready to act, avoids panic, and is in a position to assess the method and assumptions underlying the overpayment determination. Having your statistical facts and strategy in order will go a long way toward a favorable resolution in your healthcare audit.

[1] Office of the Inspector General (OIG). In the Health and Human Services OIG Publication of the OIG’s Self-Disclosures Protocol (1998), the OIG strongly recommends RAT-STATS statistical sampling software as part of providers’ self-assessment of overpayments. It is available for free on the Internet at (http://www.oig.hhs.gov/organization/oas/ratstats.asp; http://oig.hhs.gov/authorities/docs/selfdisclosure.pdf)

[2] Centers of Medicare & Medicaid, Medicare Program Integrity Manual, Chapter 3 “Verifying Potential Errors and Taking Corrective Actions,” 3.10 Use of Statistical Sampling for Overpayment Estimation.

[3] SPSS and SAS are providers of statistical software. More information is available on their websites.

Editor’s note: Cornelia M. Dorfschmid is Executive Vice President with Strategic Management, located in Alexandria, VA. She may be contacted by e-mail at [email protected] or by telephone at 703/683-9600 x419.