Using Surveys to Evidence Compliance Program Effectiveness

Over the last decade compliance officers and programs have matured and become more sophisticated.  The importance of such programs has been underscored by a number of mandated compliance laws and regulations passed by Congress, such as the Sarbanes Oxley Act and more recently the Affordable Care Act.  The latter legislation includes provisions mandating compliance programs as a condition of enrollment and most likely continued participation in Medicare, Medicaid, and the Child Maternal Act programs. CMS is charged with developing these new requirements and will likely build off the voluntary compliance standards of the DHHS Office of Inspector General (OIG).

With executive leadership having to certify compliance under these new mandates, this is likely to create a sea change in terms of how executive leadership in many organizations will view their compliance officers.  Those executives who will have to make the certification will be forced to rely upon metrics and evidence of compliance program effectiveness, generated by their compliance officers.  As a result, more and more attention is being given to what types of metrics are needed.  This article deals with one type of metric evidence of compliance program effectiveness: employee surveys.

The OIG in their compliance guidance makes reference to this type of evidence when they recommended compliance officers to take a “snapshot” of their operations from a compliance perspective. This “snapshot,” often used as part of benchmarking analyses, becomes a baseline for the compliance officer and other managers to judge the organization’s progress in reducing or eliminating potential areas of vulnerability.

They note that “[t]he existence of benchmarks that demonstrate implementation and achievements are essential to any effective compliance program.” The OIG further indicates that “organizations [should be] evaluating all elements of a compliance program through employee surveys, management assessments, and periodic review of benchmarks… and other employee feedback.”

They suggest one way this can be done is through the use of questionnaires that solicit impressions of a broad cross-section of the employees and staff as part of ongoing compliance program monitoring.  They suggest using these techniques that permit the compliance officer to identify and review variations from an established baseline.  Significant variations from the baseline should trigger a reasonable inquiry to determine the cause of the deviation and to take corrective action measures, such as new or revised written guidance and additional focused compliance training.

Compliance program effectiveness surveys, conducted regularly, and acted upon promptly; provide an effective means of gauging and improving compliance programs.  There is the added benefit of employers demonstrating they value and are interested in their employees’ views.  Surveys obtain quantifiable data on workforce knowledge and/or perceptions and attitudes that can be applied to benchmark and monitor progress against their own results (i.e., trending historical company survey data), as well as possibly allowing for comparisons to industry peers and national averages.

There are two common types of surveys: knowledge surveys and culture or attitude surveys.  The former measures knowledge of various elements of a compliance program; and the latter the general compliance cultural environment.

A knowledge survey asks employees about the compliance program and questions should be designed to gather actionable information and insight from employees, identifying opportunities for compliance program improvements. It can measure employee knowledge and experience under the compliance program, whereas the compliance culture surveys measure employee attitudes and perceptions of the compliance culture. The knowledge surveys are very useful in assessing progress and effectiveness of the compliance program.  It should be used only by organizations that have a compliance program that has been in operation for over a year.

For this type of survey, it is recommended that the questions be dichotomous; in that they are simply questions with Yes/No answer choices. They are the simplest of all the closed-ended questions, and as such prove extremely easy to answer.  What you are really asking is their personal knowledge of certain key elements of the compliance program.

The questions should address what the U.S. Sentencing Commission’s Guidelines for Organizations and Entities (“Guidelines”) and the various guidance documents of the OIG note that affected parties should experience or know. Employing this type of survey can be a highly effective means of determining how well the compliance program is understood and implemented in its various elements, as well as insight as to which action steps are needed to improve overall program effectiveness.

Compliance culture or attitude surveys can assess the compliance cultural and work environment of an organization and assists in the design of compliance and improvement of programs. The results can provide important insights as to organization strengths on which to advance the compliance culture and areas warranting special attention.  Unlike the knowledge survey, the focus is on perception and attitudes.

Most such surveys use a Likert scale to record degree of agreement.  Using a Likert type scale of one to five makes it impossible to determine at what level the score is positive versus negative.  A score of three may be good, bad or neutral.  Comparing scores to a large universe of many organizations and large survey populations will cure that deficiency. Under such circumstance, an organization is able to determine how they rank in the industry as a whole and by any of the six compliance dimensions measured.

The anonymous feedback gathered by such a survey enables management to make informed decisions about making organization, cultural, and management changes essential for creating an environment that is conducive to ethical behavior and compliance with all applicable rules. Results from surveys can provide a lot of benefit and value, including:

• Identifying and reviewing key areas of the compliance program where special focus and attention may be needed for overall program improvement
• Supporting compliance monitoring for the purpose of disclosing in a timely manner any deviations or deficiencies in program goals that enable taking corrective actions

• Establishing a benchmark for trend analysis and the study of positive or negative deviations over time
• Evaluating and evidence the effectiveness of the compliance program with respect to addressing employee level of knowledge and buy in of the program
• Providing feedback that assist in designing future compliance training programs
• Providing a benchmark for comparison by future surveying

Surveys should be designed so that respondents will need only about 20 minutes to complete them.  Selecting the most appropriate survey administration mode depends on factors such as the number of respondents desired, how fast the information must be collected and analyzed, and the degree of anonymity required. Today, it is clear that using an on-line survey is the most efficient method.

In deciding on the possible use of surveys, keep in mind that it must be credible to an outside third party. For the results to be accepted as evidence of program effectiveness, it must meet certain professional standards.  They should be designed professionally, tested and validated, and provide meaningful results.  The best surveys are those that have undergone such a process and have been deployed over a long period of time to many organizations. This will amplify and clarify meaning of scores.

For more information, contact Jillian Bower, MPA V.P. of Business Development and Marketing the Compliance Resource Center complianceresource.com. Her contact information: (703) 683-9600, x. 405 or email [email protected]