Systems Reviews versus Transaction Reviews — A Closer Look at a New Era of Mandatory Compliance
A New Era of Mandatory Healthcare Compliance Programs
Compliance programs initially were voluntary in the healthcare industry. Eventually managed care organizations had mandatory healthcare compliance programs and states began requiring them (e.g., NY OMIG for the NY Medicaid program). Entities under corporate integrity agreements (CIAs) with the Department of Health and Human Services (HHS) Office of Inspector General (OIG) were also required to have compliance programs. Although a patchwork of mandatory healthcare compliance programs took shape over the years, with the Patient Protection and Affordable Care Act (PPACA) effective compliance programs have become the norm. PPACA makes compliance programs mandatory for suppliers and providers as they become a Condition of Participation (CoP) in the federal healthcare programs. PPACA requires that the compliance programs contain the core elements established by the HHS Secretary in consultation with the Inspector General [i.e., compliance program guidance (CPG)] with respect to that provider or supplier and industry or category [1].
If providers and suppliers can be denied participation due to the lack of an effective compliance program, one may ponder a) what standards, benchmarks, and best practices are currently available to make that requirement auditable, transparent, and sufficiently well defined and b) what type of examinations and reviews exist to confirm or deny effectiveness. Many of these issues will be discussed by the healthcare industry stakeholders in the coming years. In spite of the CPG, few metrics have been set forth to define an effective program and no accredited agency or body currently exists to specify universally accepted metrics or deem an entity’s program effective. However, some auditing and monitoring approaches, namely those reviews conducted by Independent Review Organizations (IRO) approved by the OIG, provide some insight into measurability and how to tackle effectiveness.
IROs conduct external reviews of entities under CIAs, must be independent, and follow OIG approved methods and standards.[2] They routinely report their findings and results to the OIG. One may argue that if passing an IRO-type review is accepted by the OIG as evidence of an effective compliance program, it behooves all entities, not only those under a CIA, to examine these IRO-type reviews. Adopting some aspects and metrics of these types of reviews or even conducting “mock” IRO reviews as part of a proactive auditing and monitoring program warrants a closer look.
IRO Reviews and Evidencing Effectiveness
So what are these IRO reviews and how can they help with evidencing effectiveness in this new world of “mandatory” compliance? IROs examine healthcare entities under a Corporate Integrity Agreement (CIA) with the HHS OIG and typically look at two aspects of an effective healthcare compliance program: outcome and process. While each CIA is tailored to the specifics of the settlement and problem context and the scope of the particular IRO’s review duties is particular to the CIA, most IROs conduct two types of reviews: Transaction Reviews and Systems Reviews.[3]
Not every CIA always requires both and whether a Systems Review is conducted may be conditional upon results and findings of Transactions Reviews. However, each type of review is directly related to either outcome or process and hence in line with the OIG’s CPGs. The OIG’s original compliance program guidance for hospitals stated that “[t]he existence of benchmarks that demonstrate implementation and achievements are essential to any effective compliance program.”[4] In other words, an effective program has to a) have demonstrable processes in place that implement the program and b) be able to demonstrate via outcomes the achievements that are made by applying these processes.
Systems Reviews Follow Transaction Reviews
Implementation deals with process and related procedures that can be assessed and described in terms of a system of internal controls. Well-implemented healthcare compliance programs rely on well-specified processes, i.e., existence of internal controls, including metrics and threshold or target rates for compliance that keep the processes within desired limits. These metrics and thresholds are then used in monitoring. If the process fails or underperforms, a Systems Review is conducted to find out “why” and the root causes of failure are examined. The Systems Review tries to detect and correct aberrant patterns, not singular events. Achievements, on the other hand, deal with outcome or performance and typically are reviewed by auditing samples and looking at results of singular items or events at a point in time or for a certain review period. A “snapshot” is taken. Transaction Reviews take these snapshots and are meant to examine specific outcomes, they are audits. They check ‘what” the state of compliance for a universe of cases or items in a given review period is. Transaction Reviews involve auditing specific claims, physician arrangements, patients in clinical trials, enrollment plans, FDA-approvals, etc. IRO reviews can be grouped and associated as follows:
Transaction Review assesses Outcome:
- Review of Achievements or Performance
- Keyword: Auditing, State of Compliance/Findings, Review Period
- What happened?
- Unit of Analysis: Sampling Item
Systems Review assesses Process (= Implementation):
- Review of Implementation or Design
- Keyword: Monitoring, Internal Controls, Workflow, Root Cause
- Why did it happen?
- Unit of Analysis: Pattern of Process, Functional Relationships
A simple analogy to these concepts is that of a machine (process) in a production line and the result (product, output). Both process and product must be good, otherwise the production system is not effective. It is the same with compliance programs, and these two types of reviews address those same two aspects of an effective system. If the product is faulty, it typically leads to a machine inspection!
Systems Reviews look at the bigger picture, i.e., patterns and the process as a whole. They are often triggered by Transaction Reviews. That is, audits detect a significant number of non-compliant items or system failures and a System Review therefore follows. The OIG defines the purpose of the Systems Review most clearly in the context of a claim processing system and billing violations: ”The purpose of the Systems Review is to identify problems and weaknesses that resulted in overpayments. A Systems Review is a ‘walk through’ of the systems(s) and process(es) that generated the sampling unit in error.[5]” It is not the particular findings and individual audit sampling unit that are of interest, but the process itself that led to the errors, i.e., the root-causes. In summary, the Systems Review relies on root cause analysis that includes examining:
- work flow or process flow (walking through);
- written policies and procedures;
- communication of procedures;
- internal controls;
- training and guidance;
- technical configurations, software settings for approval and override, such as in HIM, EMR, patient accounting and billing systems; and
- metrics that trigger alerts or sound “alarm.”
Systems Reviews can be complex, quite time-consuming, and potentially very costly, as they are meant to identify and quantify internal controls and map out a process to trace the flow of decisions and documents to the origins of actual or potential control failures by looking at the entire workflow.
The OIG does not define the concept of the Transaction Review as explicitly as the System Review. However, from examining the numerous CIA documents published by the OIG[6], one may conclude that these types of reviews are best described as audits that rely on statistical sampling of individual items from a universe of items in the system, such as claims, patients in clinical trials, plan members, pre-market approvals for drugs or devices, physician arrangements, etc. Transaction Reviews focus on the particular results the processes generated and not on the design or number of internal controls. For example, in a revenue cycle system one might review the number of claims billed in error or the amount paid with error, or the denial rate. In a contract management system an arrangement review might assess the percentage of contracts violating the FMV standards, and in a FDA compliance assessment the percentage of inappropriately billed drugs for off-label use might be reviewed as part of a Transaction Review. As Transaction Reviews deal with performance and outcomes, they aim to count faulty items, calculate error rates or failure rates, and check whether the process stays within acceptable tolerance levels, i.e., whether it is well controlled.
Metrics of Effectiveness
The OIG has put forth one such tolerance level for use in Transaction Reviews dealing with claims payments. The OIG has defined a metric to measure overpayment errors: the “financial” error rate, i.e., net overpayment error rate. The rate is calculated from a statistical discovery sample of claims (at least 50). The OIG has set forth a threshold level for a well controlled claims process: the financial error rate at 5%. If this tolerance level is exceeded in a Transaction Review examining paid claims on an entity under a CIA, it typically triggers a Systems Review by the IRO, including a full sample claims review.[7] The 5% rate is taken as an indicator for a systemic problem in the claims process that warrants further investigation and requires getting to the root of the problem. Another compliance metric that relates to revenue cycle integrity is the coding accuracy rate. Although not mandated in CIAs, the best practice threshold accuracy rate is 95% according to AHIMA’s best practice guidelines. This coding accuracy can serve as target compliance metric for use in Transaction Reviews conducted by any entity. Similarly, compliance training completion or participation rates, such as 90% or 95% are other metrics that can define a well-functioning compliance program and be confirmed in audits. Another example is the deviation of physicians’ profiles from national norms of Evaluation &Management (E&M) level bell curves for their respective specialties. For example, deviations of more than two standard deviations from the normal profile could be set as thresholds and trigger alarm and a Systems Review as follow on. In summary, Transaction Reviews audit against defined standards and compliance metrics.
Transaction Reviews can therefore be understood as audits that serve as a risk identification and assessment measure and are part of a risk management approach for the compliance program. This is so even if the risk assessment is performed by an external entity, such as the IRO! Transaction Reviews involve reviewing:
- Samples or items from a larger set, such as claims universe, arrangements database, patient list, trainees, etc;
- Projections such as overpayment extrapolation or percentage/occurrence rate projection;
- Deviations from thresholds and triggers; (error rate, accuracy rate, completion rate, etc.);
- Check lists with characteristics of measure to determine if an item is compliant or correct; and
- “Pass/Pass with Findings/Fail” results.
Failure or not meeting targets understandably would lead to further review. It is noteworthy that several recent CIAs are more stringent and require conducting Systems Review(s) regardless of the outcome of a Transaction Review, i.e., not only when a Transaction Review fails, which is the more traditional approach in CIAs focused on provider claims reviews in the past.[8] It appears that the OIG pursues a strategy aggressively preventing risk of improper payments and compliance violations by an entity under a CIA. They make the IRO not just focus on outcomes and evidence of a “clean claims” universe via sampling, but require examining the processes that led to the billed claims, even if the financial error rate does not exceed 5%.
Supporting Internal Auditing & Monitoring
Compliance Officers may want to apply the Systems Reviews and Transaction Reviews as another tool in his/her toolbox to ensure that the compliance program is effective and to prevent or successfully pass any government audits. These IRO-type reviews can expand and support internal auditing and monitoring, i.e., one of the seven core elements of an effective compliance program. Especially organizations under a CIA, managed care plans, and others with mandatory programs and reporting requirements may benefit from developing such structured and formalized review capabilities in-house and mimic what the IRO does in preparation for the annual IRO checkups and beyond. Understanding the difference between the two types of reviews is also important if a compliance office seeks outside help and outsources reviews. It needs to specify the scope of the engagement clearly and state essentially if it seeks process reviews or audits.
Furthermore, when developing and expanding Transaction Review capabilities to support internal auditing and monitoring, especially statistical auditing and extrapolation methods should be formalized and added to the compliance program’s audit and transaction review protocols. Sampling is one of the most powerful and cost-effective method to gain insight into a risk area with limited resources. Well designed probe and discovery samples can go a long way in checking if the regulatory compliance program is fairly well controlled or deviations from metrics, such as the financial error rate or coding accuracy rate, indicate potential systemic patterns. Detecting these deviations through Transaction Reviews, and then confirming or refuting them using in depth Systems Reviews will lead to workflow changes and internal control improvements as part of effective compliance risk mitigation strategies. Getting into the habit of using both review types wisely may very well be the best preparation for any form of certification of compliance program effectiveness in the coming years.
[1] – See Sec 6201. This provision is consistent with recent state developments that have made compliance programs mandatory for Medicaid providers. See also Lewis Morris, Chief Counsel to the IG, Testimony to Subcommittees on Health and Oversight of the U.S. House Ways and Means Committee on Reducing Fraud, Waste and Abuse in Medicarehttp://www.oig.hhs.gov/testimony/docs/2010/morris_testimony61410.pdf
[2] – Thomas E. Herrmann, Independent Review Organizations Must Meet GAO “Yellow Book” Standards, Journal of Health Care Compliance, Vol. 12, No. 2, March- April 2010
[3] – All current CIAs are posted on the HHS OIG website on the Corporate Integrity Agreements Documentation List: http://oig.hhs.gov/fraud/cia/cia_list.asp
[4] – HHS, Publication of the OIG Compliance Program Guidance for Hospitals, Federal Register / Vol. 63, No. 35 / Monday, February 23, 1998 / Notices; http://oig.hhs.gov/authorities/docs/cpghosp.pdf, p. 8988
[5] – http://oig.hhs.gov/fraud/cia/faq.asp
[6] – http://oig.hhs.gov/fraud/cia/cia_list.asp
[7] – A full sample is larger than a discovery sample and allows for more precise estimates. Usually this is accomplished by reviewing more sampling items.
[8] – Some CIAs require a systems review in year 1 and year 4 (e.g., CIAs for AstraZeneca or Spectranetics), regardless of transaction review results.
