Q&A on Evidencing Compliance Program Effectiveness
Q&As from the August 16, 2022, Webinar presented by Richard P. Kusserow and hosted by SAI360
- How important is it to evidence compliance program effectiveness?
There are many reasons why it is important to be able to evidence compliance program effectiveness. The following provides some reasons:
- It is the right thing to do.
- Many organizations find that their leadership and Boards are seeking such evidence.
- The Department of Health and Human Services Office of Inspector General (OIG) considers it a mitigating factor in the assignment of liabilities and terms of a Corporate Integrity Agreement (CIA).
- The Department of Justice (DOJ) notes the absence of evidence of an effective compliance program negates the argument of the fault being an individual “rogue employee” rather than the organization.
- Failure to evidence an effective program increases executive and board liability exposure.
- As the Centers for Medicare and Medicaid Services completes promulgating regulations implementing the Affordable Care Act, evidencing compliance program effectiveness becomes mandatory.
- Effective programs can reduce costly errors and high employee turnover.
2. Is there an assessment tool available when vetting an expert for the external review process?
The following are some suggestions to consider:
- Examine the history and leadership of prospective firms.
- Ask the firm how many evaluations it has conducted and ask for references.
- Avoid those that use a common checklist approach, which produces poor process results.
- Engage a firm that can evidence being expert on compliance.
- Ask if the firm will certify meeting General Accounting Office operational review standards.
- Require a tort liability insurance policy for their work ($1-5 million).
- Check the credentials of the individuals conducting the review.
- Ensure those who perform the review are real experts and avoid bait-and-switch.
- Ensure the evaluation includes (1) a full document review, (2) an audit of operations, activities, and functions, (3) interviews with executives/board members/key employees, and (4) focus group meetings.
- Be sure the review focuses on auditing and monitoring high-risk areas.
See also SMS’s blog discussing alternative methods for conducting program evaluations and best practice tips. It includes a lot of additional guidance related to this question.
3. Can an internally developed and administered survey provide credible evidence of compliance program effectiveness?
The problem of internally developed and administered surveys is largely the credibility of results:
- Most likely this type of survey is not professionally developed, tested, or validated.
- It won’t be benchmarked against other organizations in the healthcare sector.
- For outside authorities like the DOJ and OIG, credible evidence comes from independent sources outside the organization’s control.
- Many may suspect an internally developed survey is designed to provide biased results.
- There is the problem of ensuring respondent anonymity.
- Developing and managing surveys internally is more costly than using a vendor service.
- Results are not analyzed and assessed by outside experts.
4. Can you provide examples of what an independent compliance program evaluator would look at/ask about? I am guessing there are no standard assessments/questionnaires that they would use, correct?
Questionnaires can be very useful for a compliance officer in the ongoing monitoring of the operation of the compliance program. They can produce output information, often in terms of metrics, but do not provide credible evidence of how well that process has been in achieving the desired outcome (effectiveness). The OIG and the Health Care Compliance Association’s (HCCA’s) joint publication, “Measuring Compliance Program Effectiveness: A Resource Guide,” provided a list of 401 possible metrics (“What to Measure”) and 630 recommendations for how to measure against the metrics (“How to Measure”) that compliance officers can use to review their compliance program. However, the output of process reviews lends itself to metrics, but outcome seldom can be evidenced numerically.
5. How has the new DOJ guidance regarding Chief Compliance Officer (CCO) certification impacted this information?
On March 25, 2022, in remarks to NYU Law’s Program on Corporate Compliance Enforcement, Assistant Attorney General Kenneth A. Polite Jr., announced a new policy requiring CCOs to sign off on certain agreements with the DOJ, stating that the policy is meant to “empower” the CCO, to ensure that the CCOs are “in the room” and reporting to the board directly about “what has or has not gone on in the course of fulfilling the company’s obligations,” and to promote the concept that “the business is taking ownership of its role in the compliance program and the Officer receives all relevant compliance-related information and can voice any concerns prior to certification.” In carrying out such mandates, the CCO should be prepared to evidence meeting these requirements. Having an Independent Evaluation of the Compliance Program that documents the progress and accomplishments of the program with ideas by which the program can be enhanced would go a very long way to providing this evidence. Such reviews should address the hundreds of prosecutor questions in the DOJ’s June 2020 “Evaluation of Corporate Compliance Programs.”
6. How often should Independent/Outside Compliance evaluations be performed?
Governmental authorities have not provided a frequency for having an Independent/Outside Compliance Program Effectiveness Review other than referencing the general term “periodic.” I don’t believe it is cost-effective or worthwhile having an annual review of such a magnitude unless the organization is under a CIA with a mandate for a Compliance Expert to provide such a review. Having said that, many larger organizations that have annual reviews conducted are often driven by decisions at the Board level. We have found the best practice is to have a comprehensive review every two or three years with an employee survey using a professionally developed instrument administered independently and anchored to a large data universe. This permits understanding the program’s status when compared to that universe.
7. What are the best means by which Industry Comparative Data can be used to benchmark a Compliance Program?
Two types of healthcare sector compliance surveys permit individual organizations to benchmark their results against a large industry universe. The Compliance Program Knowledge Survey© is a dichotomous survey designed to assess the effectiveness of an organization’s compliance program from feedback from employees. Results are benchmarked against the universe of those who have completed this same survey. The Compliance Program Benchmark Survey©, a compliance culture survey, has been used since 1993 to measure employees’ perceptions and attitudes in evidencing the compliance culture of an organization using a Likert-based survey. Results of this survey are also benchmarked against the universe of hundreds of organizations that have used this same survey.
8. Is it being suggested that the HCCA Measuring Effectiveness Guidance document is inadequate to measure effectiveness?
The OIG and HCCA’s January 2017 “Measuring Compliance Program Effectiveness: A Resource Guide” is a 54-page document that provides a list of 401 possible metrics for measuring a compliance program with 630 recommended actions to assist in evaluating compliance programs. It is very useful for compliance ongoing monitoring but focuses primarily on the operation and program management processes, and process evaluation is just the first level in determining how truly effective a program is in achieving desired goals. Measuring process outputs lend well to metric evidence, however, outcome measurement seldom can be evidenced numerically. Using surveys was mentioned over 60 times as one area relating to evidencing outcome.
9. What do you think are the highest risk areas to monitor for behavioral health in 2023 in that sector is growing very quickly?
The fact is that the mental health services sector is rapidly growing because of the Affordable Care Act, and the COVID-19 pandemic loosened some health care service rules to enable faster delivery of needed services. This rapid growth has also led to a rise in fraud and abuse issues leading to OIG and DOJ enforcement. Among the most common areas have been (a) improperly prescribed unnecessary psychotropic medications, (b) fabricated patient files and services, (c) patient recruiters involved in kickbacks, (d) billing for patients not referred by health care facilities, (e) billing for patients without diagnoses mental health issues, (f) any claims for services medically unnecessary, (g) services never provided, (h) services for ineligible individuals, etc. It is noteworthy that the OIG is continuing its review of the appropriateness of Medicare payments for partial hospitalization programs and community mental health centers for psychiatric services. However, if you are looking for the highest risk area, focusing on any arrangements for recruiting patients should be at the top of the list, followed by claims processing.
10. What are the differences in cost and benefit of having a Compliance Program Effectiveness Review, Compliance Program Gap Analysis, and Compliance Program Survey of Employees?
Compliance Program Effectiveness Review is the most expensive, generally running between $50-$100,000 depending on the size and scope of the work. It involves a full review of all compliance-related documents (in effect the same process as a Gap Analysis); examination of the program elements in operation; and debriefing of the Compliance Officer and staff, interviews with the CEO, other executive leadership, board members, and key employees. It may also include focus group meetings with selected employees. A key focus is reviewing and verifying that ongoing monitoring and ongoing auditing of high-risk areas are operating effectively.
Compliance Program Gap Analysis is primarily a documentary checklist review that is supplemented by some interviews. Its cost, depending on the size of the organization and scope of work, normally runs less than half of a full Compliance Program Effectiveness Review. It is a process review that only tells you what is in place (outputs) but not how well it is functioning or how effective the program is in meeting its objectives (outcome). This is primarily for organizations where the program is just getting started and has not been fully implemented. It is of very limited value for fully established programs.
Compliance Program Knowledge Survey© or Compliance Program Benchmark Survey© of employees can provide a measure of effectiveness and is recommended by the OIG. These surveys can test compliance knowledge, understanding, attitudes, and perceptions of employees. When using a professionally developed and independently administered survey linked to a larger universe to compare results with other organizations in the health care sector, is extremely valuable. The cost of such a survey is far less than a Compliance Program Gap Analysis and roughly only about 10-15% of the cost of a full Compliance Program Effectiveness Evaluation.
11. What is the least costly method to evidence Compliance Program effectiveness?
Without a doubt, the easiest, fastest, and least expensive way to provide convincing evidence of Compliance Program effectiveness is the use of a validated healthcare compliance survey. It is one of the two suggested methods by the OIG.
12. Who do you recommend for conducting Interviews during a Compliance Program Effectiveness Evaluation?
If an organization engages a firm to perform such an evaluation, the key is that all those involved in the review have compliance experience and expertise. This, in turn, means that those conducting the interviews be experts on debriefing and garnering relevant information, rather than someone just following a checklist of questions.
13. Do you have any information on CMMC compliance and adherence?
The Cybersecurity Maturity Model Certification (CMMC) is not focused on the healthcare sector. It is a program established by the US Department of Defense (DoD) to secure and protect Federal Contract Information (FCI) and Controlled Unclassified Information (CUI) by requiring the certification of external contractors across 17 different domains. However, it deals with similar issues as presented in the webinar in terms of process and resulting practices. The focus is largely on compliance maturity levels.
14. What about automated systems?
Automated systems can be a very good method to monitor compliance. The key point is monitoring and that relates to process and outputs, but not necessarily outcome which evidence effectiveness of the process.
Frequently Asked Questions (FAQs)
For more information on this subject, contact Richard Kusserow at [email protected].
Keep up-to-date with Strategic Management Services by following us on LinkedIn.
