Conducting an Ethical Climate Assessment Survey for your Organization

The U.S. Sentencing Commission explicitly recognized the significance of “culture” in its 2004 amendments to the Federal Sentencing Guidelines, stating that businesses must “promote an organizational culture that encourages ethical conduct and a commitment to compliance with the law.” [1]  The Federal Sentencing Guidelines call for organizations to regularly evaluate the “effectiveness” of their compliance and ethics program.

The Importance of an Organizational Climate Survey for Compliance

Thus, not only must organizations demonstrate the presence of specific program elements, they must also provide evidence that these programs are actually working.  This is almost identical to what the Department of Health and Human Service Office of Inspector General (OIG) calls for. The OIG recommends that organizations should evaluate all elements of a compliance program through “employee surveys, management assessments, and periodic review of benchmarks established for audits, investigations, disciplinary action, overpayments, and employee feedback.”[2]

The Shift Towards Using a Company Climate Survey to Evidence Compliance Effectiveness

It is still common for Compliance Officers to evidence the effectiveness of their compliance programs (CP) on implementation of basic program elements, such as the amount, attendance and type of compliance training, as well as how effectively these elements are put into action.  However, the move is toward having other more convincing evidence of program effectiveness. To assess whether such an environment exists, organizations are increasingly making use of employee perception and knowledge surveys. And more and more Board of Directors, given their oversight duties and personal exposure, are requesting such surveys be undertaken.

These surveys are used to obtain quantifiable data on workforce knowledge and perceptions that can be applied by companies to benchmark and monitor progress against their own results (i.e., trending historical company survey data), as well as possibly allowing for comparisons to industry peers and national averages.

What Do Organizational Climate Surveys Measure?

Some survey can measure ethical outcomes, or the “impact” of their program activities. However, assessing an ethical climate and assessing program effectiveness are not one in the same. Often, the two are confused and creating tools that confound the measures. The difference between “culture” and “climate” is commonly confused, and the terms are frequently and erroneously used interchangeably (even in the Federal Sentencing Guidelines). They are, however, derived from very different ontological traditions and represent distinguishable perspectives. And while the two constructs are complementary, for the purposes of quantitatively assessing organizational ethics, we are best to conceptualize an ethical climate.

Organizational Climate vs. Culture

Culture is commonly defined as the beliefs and values which guide the thinking and behavior of an organization’s members. Traditionally, culture was assessed by qualitative methods such as case studies, observation, storytelling and unstructured interviews to examine symbolism, sense-making, and social norms in an organization. Culture research is often concerned with the evolution of social systems over time, and has inconsistent results when predicting outcomes of interest.

In contrast, climate researchers place a greater emphasis on an individual’s perceptions of observable practices and procedures in an organization, and the categorization of these perceptions into analytical dimensions that can be measured. Climate, rather than culture, reflects employee’s perceptions of the organizational structures that help them to make sense of their world.

Questions pertaining to the possible effect of program activities should be distinguished from items that seek to measure a climate for ethics. Questions that ask employees about the awareness and impact of training concepts, their use of various program resources, and perceptions regarding compliance and ethics communications, etc., are valuable measures of program impact. In addition, there may be value in analyzing the data for correlations between program inputs and ethical climate items.

Why Organizational Climate Surveys?

Surveys have examined the extent to which individuals, coworkers, supervisors, and leaders demonstrate commitment to compliance by setting a good example, keeping promises and commitments, and supporting others in adhering to compliance standards. Surveys can be extremely useful tools for assessing the current state of the compliance climate of an organization. Surveys also communicate a strong message to employees.

Ideally, surveys can signal to employees, customers, or other stakeholders that their opinions are valued, that the organization is committed to them as individuals, and that their input is being used to make positive changes.  These messages can have a powerful influence on increased compliance, reduced violations and heightened integrity.

Standardized surveys can be supplemented with organization specific questions and/or open-ended questions designed to provide more dimension to the information gathered. Also the survey can be linked to focus group meetings and interviews to shed additional light on the reason why there may be a problem as might be suggested in survey response to certain questions. This can assist if certain issues require deeper probing and more nuanced evaluation.

Before Beginning the Ethical Climate Assessment Survey Process

Before beginning the survey process, it is a good idea to form a survey committee. The committee is responsible for developing a strategy to manage all phases of the survey project. The process of designing, implementing, and reporting the data from an organizational survey is as important, if not more important, than the actual results. Consider the following components.

Organizational Climate Survey Objectives

Before getting started, it is helpful to be able to answer the following questions:

1. Why is this survey being conducted?
2. What is to be accomplished by this survey?
3. What will the organization do with the survey results?

Once this is developed, the survey team should be able to articulate the survey objectives clearly.

Organizational Climate Survey Target Audience

The population to target in any survey research is the group of individuals who possess knowledge and views pertinent to the survey content. In ethical surveys, the target population will be all members of the organization. Obviously, you want a response rate (i.e. the percentage of people who complete the survey) high enough to feel confident the survey results are generalizable to the organization as a whole. There is no ideal percentage to aim for, as the response rate will depend on such things as the length of the survey, the effectiveness of the survey communication, and how effective any previous survey research has been in your organization. However, a 50 percent response rate is considered acceptable.

Mode of Climate Survey Administration

Selecting the most appropriate survey administration mode depends on factors such as the number of respondents desired, how fast the information must be collected and analyzed and the degree of anonymity required. Today, most companies are opting for on-line surveys for ease; however, no evidence supports that the response rate is different from the traditional paper survey. Whatever the method of administration, it is critical that the target population is assured of their anonymity, as this will increase the likelihood of participation.

Invitation to Participate in Organizational Climate Surveys

Any survey should include a description of the purpose of the project. This description should address the need for the project and attempt to overcome participant anxieties about the purpose of the survey. Employees will likely want to know why they were chosen to participate, what will be done with the findings, and how the survey will impact them personally. Specific language answering these questions should be included in the invitation.

The invitation to participate should also include assurances of anonymity, so participants understand that individual respondents cannot be identified. Employees should be reminded that the survey is not a test—there are no wrong answers and honest answers are best. They also must be assured that only group-level data will be examined, not individual data. For example, the data is analyzed in terms of how the entire accounting department responded, not in terms of any accountant’s individual responses. Another set of fears employees may have is that the survey will be used to punish them or their co-workers in some way.

These anxieties are likely to influence employees’ willingness to complete questionnaires and may affect the honesty of their responses, if they choose to participate. If possible, provide assurances that the project has no malevolent intent. Supervisors should also communicate the nature of the survey to their subordinates, and, if relevant, provide examples of how similar surveys have been used in the past.

Instructions and Response Scale

 You should include clear instructions for accessing and completing the survey. The instructions can include tips like where to write responses and whether the questions refer to the employee or to employees in general. The choice of a response scale is a function of a number of statistical and practical concerns. The most important point is to choose a response scale that matches the information requested in the question. In one of the commonly used ethical climate surveys, the scale range is anchored from “Completely False” to “Completely True.”

In general, there are two rules for response scales. First, use an odd number of response options (3, 5, 7). This is particularly relevant where a middle value can be used to assess “no opinion or neither agree nor disagree.” Offering a “don’t know” or “no opinion” option for perceptual questions is sometimes perceived as losing information, forcing people to express an opinion when they genuinely have no opinion, or are unsure, may lead to misinterpretations of the data. Second, use no more than seven points in the scale, because little or no increase in precision is gained beyond seven points.

Wording of  Climate Survey Questions

A good guideline for organizational research is that “You only get what you measure” and, conversely, “If you don’t want to know, don’t ask.” Here are some guidelines to keep in mind when constructing questions:

1. Keep questions short, unless a detailed explanation or clarification of an issue is absolutely necessary. For example, “Successful people in my company operate by the book.”
2. Avoid ambiguity. Questions should be specific, avoiding generality. For example, “On a scale from 1 to 5, how popular is the company’s ethics program?” which begs the question, “popular with whom?” Some respondents will answer an item like this in terms of how they, personally, would rate the program; others will answer in terms of popularity with their peers.
3. Avoid unfamiliar terms and jargon. Employees may not understand what is being asked in “The CRO reports to the (1) CRO (2) CEO (3) BOD (4) Board Level Risk Committee.” Terms such as CRO may not be well known by typical employees. When a term that is not in common usage must be used, it is wise to precede the item with a brief explanation.
4. Avoid requiring inaccessible information. Avoid asking employees for information that they would not know or do not have access to. For example, “My company has a superior ethics program as compared to what is offered by other companies” assumes that employees are familiar with both their employer’s ethics program and those offered by other companies.
5. Avoid hypothetical questions. Questions that begin with “What would you do if…” (e.g., “What would you do if someone committed an unethical act?”) are difficult for employees to answer. They require greater mental effort on the part of the employee and take longer to answer.
6. Write questions in the present tense. People’s ability to recall the past is limited. The more current and specific the question is, the better. If a recall question is necessary (e.g., “Over the past six months, I have witnessed an unethical act at work”), the time frame should be as recent as possible, and not over six months unless the reference is to a major event, such as the compliance program roll-out or mandatory ethics training.
7. Avoid causing anxiety. Some questions may cause employees to feel anxious about their futures, (e.g., “I would feel nervous if my organization was firing employees for occasionally using their company laptop to check personal e-mail.”) Where possible, avoid these questions.
8. Do not use “double-barreled” questions. Double-barreled questions tap more than one topic and can introduce ambiguities into the responses. For example, “The organization has created strong awareness about its ethics and compliance guidelines and principles and has trained employees how to use them.” If employees disagree with this statement, they may be answering about the awareness, the training, or both. Generally, the use of conjunctions like “and” or “or” should be carefully considered. Double-barreled questions should be separated into two questions.
9. Do not use leading questions. Leading questions are sometimes used deliberately to achieve personal goals. For example, the ethics training director might ask “Most people feel that ethics training is extremely important for career advancement. Do you agree?” Be aware of this potential issue, and eliminate these questions.
10. Watch for potential cultural bias. When conducting a cross-cultural survey, just translating it into a different language may not be sufficient. Keep in mind that U.S. benchmarks may not be appropriate. Consult a local expert when exporting your survey to a different culture.
11. Use multiple measures of the same topic. If a survey is designed to assess attitudes or opinions about issues that are subject to some interpretation by employees (as is the case with a climate survey), you should include multiple questions on each topic. This ensures that the responses are a function of the employee’s perceptions of the issue and not a function of the idiosyncrasies of a single question. When multiple questions are asked, the average of the answers to these questions produces a “total score” for that topic. For example, if the topic of interest is supervisors setting an ethical tone, the questions might be “My supervisor talks about the importance of ethics and doing the right thing in the work we do”, “My supervisor sets a good example of ethical behavior”, and “My supervisor supports me in following our Code of Ethics and Conduct.”
12. Keep the survey to a reasonable length. Generally, the survey should take no more than 20 to 30 minutes to complete. If it takes longer, the employee may become bored and begin responding in a careless fashion. Also, management may not fully endorse the survey, because of concerns about loss of productivity.
13. Open-Ended Questions. It is advisable to include at least a few open-ended questions in any survey. At the minimum, a “comments” section will provide respondents the opportunity to voice their opinions on issues that they feel are important, but that were not addressed in the survey. Specific open-ended questions such as, “What advice would you give your CEO to improve the ethical climate of the company?” are more effective at eliciting specific and useful answers than vague or general questions. Open-ended questions are often used as a follow-up to important structured items. An open-ended question can point out unanticipated interpretations of questions and variations in the actual meanings in responses.
14. Background (Demographic) Questions. In many cases, it is useful to include questions on employee characteristics. These questions concern personal characteristics (e.g., age, gender, marital status) or job/work characteristics such as geographical location, functional unit (e.g., sales, marketing, finance), supervisory unit, and hierarchical level. These questions are useful to divide the respondents into sub-groups of interest. For instance, you might be interested in whether reactions differ by factors such as gender, shift, location, exempt vs. non exempt, full-time vs. part time, and job classification. These issues need to be identified in the planning stage because, if they are not, the organization will be unable to identify the characteristics of a given response group during the data analysis. However, the survey should measure employee characteristics only if they are important for a particular survey purpose. Background measures are often seen as intrusive or unnecessary and may be viewed by employees as a way to identify them (particularly if many different characteristics are used). Generally speaking, it is best to avoid asking any question that deals with race, color, physical appearance, disability, citizenship/national origin, or religion. Such questions can be interpreted as offensive and as an invasion of privacy. Survey questions should be constructed on a need-to-know basis. That is, any question that is asked should have a specific, defensible purpose. Survey questions that touch on any of the above topics should be examined by an attorney familiar with labor law at local, state, and federal levels.

Sample Questions Reflecting Different Types of Response Scales

Ethical Climate

My supervisor sets a good example of ethical behavior.

• Completely false
• I don’t know.
• Completely true

What advice would you give the CEO to improve the climate for ethics around here? Select all that apply.

• Better communication
• Hire more ethical people
• Reward ethical behavior
• Deliver ethics training
• Set ethical tone at the top
• Other (box for comments)

Program Impact

The training I receive helps me comply with legal requirements.

• Strongly disagree
• Disagree
• Neither agree nor disagree
• Agree
• Strongly agree

There is a means for employees to communicate, anonymously if so desired, other than through a direct supervisor.

• Yes
• No

Once the survey is designed, you will ideally conduct a pilot study to pre-test the survey. This procedure can help avoid numerous problems later in the process, and save many additional hours of work. After refining the survey, you will want to administer it to your selected population, being sure to communicate effectively during the roll-out and data collection phase of the survey. Upon completion of the data collection phase, the challenging but rewarding task of analyzing the data begins.

This should be followed by feedback sessions with leadership and management about the results and recommendations, and finally communication of the endorsed action plans to the organization as a whole. Part of this action plan usually involves a follow-up survey scheduled for some point in the future.

Organizational Climate Survey in Review

In summary, not only must you consider the effectiveness of your formal compliance programs, you must also address a little thing called culture. Survey design teams should know how to begin thinking about what creates a climate for ethics and how to measure it appropriately. Because survey research has become increasingly popular as a method to assess organizational life, it is important to address some of the fundamental principles of survey design.

Knowing the design fundamentals is helpful, whether you choose to draft your own survey or employ an existing measure. Remember—this is only the beginning. The data collection, analysis, and the feedback and sharing of results warrant separate articles all on their own.

About the Author

Richard P. Kusserow established Strategic Management Services, LLC, after retiring from being the DHHS Inspector General, and has assisted over 3,000 health care organizations and entities in developing, implementing and assessing compliance programs.