Annual Compliance Program Effectiveness Evaluation

The Department of Health and Human Services (HHS) Office of Inspector General (OIG) and other regulatory bodies have stressed that an organization’s Compliance Program (CP) must demonstrate that it is fully effective. The Compliance Officer, like any program manager, is responsible for ongoing monitoring of the CP. But Compliance Officers cannot independently audit their own operations. Therefore, the OIG has recognized two alternative auditing methods: (1) a full evaluation by outside experts; and (2) an independent compliance survey of employees. However, the first option is cost prohibitive. Two types of compliance surveys exist—one tests the organization’s culture and the second tests employee knowledge of the CP. However, surveys are more limited in scope and depth.

Independent Compliance Program Evaluation

An independent evaluation by experts is the best way to evidence an organization’s CP effectiveness. However, organizations should avoid using a simple checklist approach to determine whether the seven elements of a CP are present. Rather, they should look for expert firms and check the credentials of the individuals conducting the review. This type of review is complex, as effectiveness is related to outcome metrics, rather than process output. Assessments should draw direction from a number of authorities, including the Sentencing Commission Guidelines, OIG Compliance Guidance, JCAHO, CMS, and overlapping requirements from HIPAA and the Sarbanes-Oxley Act. Experts should focus on how well the CP’s infrastructure and operations are functioning, the existing metrics that assess CP effectiveness, and the effectiveness of ongoing monitoring and auditing of the major high-risk areas as identified by the OIG. Assessment results should provide an organization with an in-depth analysis of the CP’s status, identify opportunities for improvement, and specify recommendations for overcoming deficiencies. A detailed assessment report is typically 50-75 pages in length.

Culture And Knowledge Surveys

Surveys are among the best and most inexpensive means for evaluating, evidencing, and benchmarking CP effectiveness. They are one of the two methods suggested by the OIG in its Compliance Program Guidance for Hospitals and Supplemental Guidance for Hospitals. The OIG noted that “as part of the review process, the compliance officer or reviewers should consider techniques such as…questionnaires developed to solicit impressions of a broad cross-section of the hospital’s employees and staff.” The OIG also recommends that organizations evaluate all elements of a compliance program through “employee surveys.” Results from a professionally and independently administered survey can provide a very powerful, approximately 30-page report for compliance oversight committees, and credible independent evidence for any outside authority questioning the program. Surveys can also identify the CP’s relative strengths, and areas requiring special attention.

Although quite different in approach, a culture survey or knowledge survey can provide great insight into the CP’s effectiveness, and can signal strengths and potential weaknesses warranting attention. These surveys have the added benefit of signaling to employees that (1) their opinions are valued; (2) the organization is committed to them as individuals; and (3) that the organization is using their input to make positive changes. Both types of surveys must be a reasonable length and generally take no more than 20 to 30 minutes to complete. Longer surveys may cause employees to become apathetic and respond in a careless fashion.

Compliance Culture Surveys are useful in measuring the CP’s outcomes and examining the extent to which individuals, coworkers, supervisors, and leaders demonstrate commitment to compliance. These surveys are presented in a Likert Scale format and offer a gradation of answers. The survey asks respondents whether they “Strongly Disagree,” “Disagree,” are “Neutral,” “Agree,” or “Strongly Agree,” with the statement presented in each item. They may also offer a sixth option, “Don’t Know,” for respondents who feel they lack the knowledge needed to answer the particular question.

Compliance Knowledge Surveys are used most often with maturing CPs to learn about their progress. These surveys test employee knowledge of CP structure and operations, including their understanding of the Compliance Officer’s role, the hotline function, and other elements. They also provide empirical evidence of program knowledge, understanding, and effectiveness.  Compliance Knowledge Surveys use only dichotomous questions with “Yes,” “No,” and “I don’t know” answer choices. This approach creates the simplest of all closed-ended questions and is therefore extremely easy for respondents to answer. As such, all the questions on the survey are referred to as nominal, and binomial statistics are not applied.

Tips on Surveying

1. Use professionally developed and validated surveys for reliable, valid, and credible results.
2. Use an independent party to administer the survey to avoid questions of bias and reliability.
3. Use only valid and independently administered surveys tested over many organizations.
4. Look for surveys providing comparative data to others who have taken the same survey.
5. Surveys must ensure participant confidentiality.
6. Using a firm specializing in health care compliance is surprisingly inexpensive.
7. Professionally developed/tested surveys are less costly than internally developed ones.

Organizations should strongly consider employing all three approaches, rotating full evaluations with the two types of surveys. The result will be an annual independent report on CP effectiveness with each report measuring different dimensions of the CP.