5 Reasons Why DIY Employee Surveys Are Bad Practice

 I recently came across an article on Vox.com with an intriguing headline.

Jeff Bezos says 94% of Amazon workers would recommend their job to a friend.

Photo by Hello I'm Nik on Unsplash

Complete bullshit, I thought. In over 20 years of surveying employees in thousands of companies we have never, ever encountered a score anywhere close to that high for that particular question. How on earth could Amazon then, when stories of the often grueling conditions in their warehouses paint a very different picture? Well there's a pretty obvious reason. The number comes from an internal survey of employees, and Amazon, like many companies these days, have opted to conduct the survey themselves in-house. The article goes on to explain it:

Amazon CEO Jeff Bezos recently said warehouse employees like working for his company so much that “94% say they would recommend Amazon to a friend as a place to work.” But some of his own employees aren’t buying that statistic.

The 94 percent number was gleaned from an employee survey program at Amazon called Connections, which asks Amazon employees to answer a single question each day before they can start working on their company computer or their warehouse workstation. Bezos cited the stat in mid-April in his final letter to shareholders as Amazon’s CEO.

But in interviews with Recode over the past two weeks, a half-dozen Amazon employees and managers, two of whom are familiar with the inner workings of the Connections program, said that many Amazon employees have widespread concerns about the Connections program and the accuracy of its data and insights.

Really? Concerns about the accuracy of the data? In truth, there's a simple explanantion why Amazon employees have concerns and it doesn't take a genius to figure it out. If this survey is being conducted in-house by HR themselves, then it's not anonymous. And if workers believe they are being tracked via the use of company logins and their responses are monitored by HR they will simply not be honest.

As I wrote in a LinkedIn post a few years ago, anonymity is critical for the success of any kind of employee feedback system. That's still true today, even with the proliferation of DIY survey tools out there, using a trusted 3rd party to conduct your employee survey is the only way to guarantee that you are getting accurate data.

Being able to trumpet a 94% recommendation rate might make you feel good about your company and probably looks great on a shareholder statement and in your press releases, but you're only fooling yourself if you think it's the truth.

So with that in mind, here are my 5 reasons why in-house DIY surveys are a lousy idea.

1. They are not anonymous
2. They are not anonymous
3. They are not anonymous
4. They are not anonymous
5. They are not anonymous

According to some of the Amazon employees who participate in the survey and spoke to Recode: "There is a common concern among Amazon’s employee base that their answers will not remain anonymous. It is a persistent concern that responses aren’t confidential/anonymous". 

Amazon also allows managers to view aggregated reults from as few as 4 team members. WOW! That's a ridiculously low number and makes it far too easy for a manager to figure out who might have made negative comments and (if the manager is like some managers I've known) make an effort to punish them for their honesty.

The article continues:

Two sources said that warehouse workers often choose the top answer, which seems to frequently be the most positive choice, just to get on with their day. Others, on small teams, fear that even if their name is not tied to their survey answers, managers may be able to take an educated guess at who responded negatively based on prior interactions and retaliate against them in some way. Managers of teams of more than four employees can view aggregate survey results from their staff, but those who lead teams smaller than that can’t, the Amazon spokesperson said.

“Depending on the size of team, people used to be able to figure out who said what,” according to a former Amazon employee familiar with the inner workings of the program. “So after a while, some employees decide, ‘I’m not going to be honest.’”

Color me shocked!

So why do an in-house survey in the first place? Many do it as a money saving excercise. But the small amount of money you might save is simply not worth it in the long run if the data you collect paints an inaccurate picture of your organization's culture. Much worse, some companies will do it as a way to deliberately distort the picture by first designing a self-serving survey and then encouraging employees to answer in the most positive terms to sugarcoat the results. Either way it's just a terrible, awful, counterproductive idea. 

And to any Amazon employees who would like to weigh-in on this topic, please leave us a comment below. (anonymously, of course).