What You Don’t Know Can Hurt You [Part 2]

Published Winter, 2004, HR Matters, pg. 26-28

As an HR executive, your top priority is the recruitment and retention of top employees. Of course, keeping your top employees is a function of keeping them satisfied, yet few HR executives know how to accurately measure, track and benchmark the all-important metric of employee satisfaction.

Many executives rely on anecdotal indicators of employee sentiment, like “water cooler gossip” or exit interviews. Relying on this type of feedback can be dangerous, however, because it is not necessarily representative of employee sentiments overall. In researcher-speak, we would say that such anecdotal indicators are plagued with “respondent bias” and “insufficient sample size”. That is, information gathered from these sources can be faulty because it comes from a small subset of employees, which is not statistically valid.

Luckily, there is a tool in existence, accessible to every organizational size and budget, that can accurately measure employee satisfaction with true statistical validity and provide actionable results: the employee satisfaction survey.

Employee survey results are empowering to HR executives. Often, employee survey results allow HR executives to prove to top management “hunches” that they’ve had all along. Employee surveys show not only how satisfied employees feel overall, but also point to specific areas of strength and weakness for the organization.

Based on our experience executing hundreds of employee satisfaction surveys for leading companies across the country, our firm has distilled the survey process into five easy steps that any company can take to design and execute an effective employee satisfaction survey program. If your company chooses to outsource this process, full-service professional employee survey solutions can be purchased starting around $2500.

Step One: Buy-In Phase

An otherwise well-planned and executed customer survey program can be rendered useless without the proper level of organizational buy-in. Everyone in your organization must be made aware of the employee survey program and educated as to its benefits it will bring them.

Experience shows that a memo released from the company’s CEO touting the importance of the survey program is an effective way to get employees on board. This also helps facilitate a high survey response rate.

In this memo, employee anonymity must be guaranteed, along with management’s commitment to act upon the survey results. If a third party is hired to conduct the survey, this fact should be highlighted as studies show that employee surveys administered by a third party have higher response rate and cause employees to feel more secure about their anonymity.

Step Two: Survey Design

A well-designed employee satisfaction survey is essential the collection of valid, reliable, actionable data. Typical customer satisfaction surveys are designed to measure:

  • overall satisfaction
  • corporate culture
  • supervisor relations
  • training
  • pay and benefits
  • work environment
  • communications

A survey instrument is a measuring tool – and as with any measuring tool, the measurements it takes are only as accurate as the tool itself. Since measuring customer sentiments can be tricky, we consider the questionnaire design stage the most “sensitive” in the process.

Most employee satisfaction surveys are composed of a series of positive statements in which customers rate their level of agreement on a five-point Likert scale (e.g., agree strongly, agree somewhat, neither agree nor disagree, disagree somewhat, disagree strongly).

Common pitfalls to avoid in the questionnaire design stage are vague or double-barreled questions. A vague question is one that may be interpreted differently by different respondents, and therefore yields less reliable data. A double-barreled question is one that asks more than one thing at the same time.

Step Three: Survey Administration

The main goals of the survey administration stage are to achieve the maximum response rate possible while protecting respondent anonymity.

Response rate is calculated by dividing the number of completed surveys into the total number of potential survey respondents. If a company invites 100 customers to take a survey and 60 of them complete it, they have achieved a 60% survey response rate. Although employee survey responses rates will vary depending on a number of factors, t he industry average response rate for an employee survey is around 60-65 percent. Bringing in a third party to conduct the employee survey can help, as can sending e-mail reminders and offering multiple survey completion options (e.g., online and paper). Using targeted e-mail reminders, real-time reporting by department, and multiple survey completion options, our firm typically achieves employee survey response rates around 90-95 percent.

Respondent anonymity is important because employees will only be straightforward and honest in their survey responses if they know that their identity is being protected. Employees must understand that survey results will only be analyzed in aggregate. We have a policy against reporting survey results by any employee subset with fewer than five individuals.

Step Four: Survey Analysis

The survey results are in. Now what? Blending the feedback of hundreds or thousands of employees into a single coherent voice can be tricky, but is of vital importance if meaningful conclusions are to be reached.

The most common method of identifying trends in employee survey responses is through a range of mathematical techniques called statistical analysis. We are all familiar with the statistical concept of a mean or average – a value calculated by adding a series of numbers and dividing by their quantity.

The mean analysis is useful, but represents only one tool in a good tackle box of statistical analysis methods. Other handy statistical techniques include the standard deviation analysis, frequency distribution analysis, and regression analysis.

Standard deviations are useful for calculating the level of agreement that respondents exhibited to each survey question. The lower the standard deviation, the closer responses tended to hover around the mean. For example, if Question X had a mean response of “3” with a standard deviation of “.5,” and Question Y also had a mean response of “3” but with a standard deviation of “1,” we know that employees felt a similar level of satisfaction with both issues but there was more agreement when it came to Question X.

Frequency distribution analysis simply counts the number of times respondents answered each question each possible way. This technique is useful not only for diving deeper into Likert scale responses, but also analyzing verbatim questions. One little-known use of the frequency distribution technique is to “code” verbatim comments into categories and then count the number of times each type of comment occurred. This presents a handy way to convert cumbersome qualitative data into succinct statistical conclusions.

Regression analysis measures how often certain survey responses tend to correlate, or move in tangent. Looking at such correlations can tell us things about employees’ sentiments that they may not have even been consciously aware of. For example, if we notice that respondents who are very satisfied overall tend to also indicate an excellent relationship with their immediate supervisor, we might infer that positive supervisor relationships lead to satisfied employees. Of course you must bear in mind the old statistical adage, “Correlation does not imply causation,” and test such correlations with other survey data before drawing positive conclusions.

Step Five: Results Implementation

Too many well-designed, administered, and analyzed employee surveys fail to yield positive change due to a simple lack of results implementation. We recommend that if an organization is not ready to make changes for the better, they should save time and money by avoiding the employee survey process altogether. The results implementation stage is where the proverbial tire meets the road.

Every report summarizing survey findings must end with a list of actionable recommendations. Intended changes must be realistic, achievable in the short-medium term, and most importantly, backed by data.

Many organizations find that survey results confirmed their existing suspicions. Even if this is the case with 100 percent of the results, however, the survey project was not done in vain. Quantitative results in the form of graphs and numbers are much more compelling and actionable than hunches or antidotal evidence.

It is important to remember that an employee satisfaction survey is not an isolated event, but rather the beginning of a continual improvement cycle. Even while the results of the first employee survey are being implemented, plans must be made for the next survey administration. We recommend that surveys be re-administered at least every six to 12 months. Subsequent survey administrations are always smoother and easier than the first as they are simply a matter of repeating an already established process.

In addition to testing your organization’s success in implementing the survey results, subsequent survey administrations also test the “pulse” of employee satisfaction over time. By plotting how satisfaction measurements change year by year, organizations can identify trends and plan accordingly.

Companies who have committed themselves to an ongoing employee satisfaction survey process will often tell you that it’s one of the best investments they have ever made. Many can’t imagine how they would get by without the quick and actionable feedback that their surveys provide.

In the long term, those companies who conduct ongoing employee surveys enjoy higher employee retention levels than their peers, often saving hundreds of thousands of dollars in recruitment costs – dwarfing the cost of the employee survey program itself.

Jared Heyman is a senior project manager at Infosurv, Inc. (www.infosurv.com), a full-service online employee satisfaction survey provider established in 1998 and located in Atlanta. Mr. Heyman has overseen employee satisfaction surveys for a long list of Fortune 500 corporations and major government agencies located across the country.