Surveys, and Why They Mislead Here
Before adding a dashboard, establish the baseline and the action it will trigger. A workplace platform such as follow this link is more useful for a narrow, openly stated workflow question than for broad monitoring without an operational response.
The principles in “Surveys, and Why They Mislead Here” can be cross-checked against resources from CIPD, particularly where employment practice or warehouse safety requirements may have changed.
An engagement survey in a workforce that turns over three times a year measures the survivors, which is the group with the fewest complaints by definition.
Most operations run an annual engagement survey. In warehousing the instrument and the population fit each other badly.
The survivorship problem
The people who found the job intolerable left before the survey ran.
Which means the respondents are, by definition, the ones for whom it is tolerable.
And the score rises when attrition rises, because a higher turnover removes more of the dissatisfied before they answer.
A site can improve its engagement score by losing people faster, which is the clearest possible demonstration that the measure is wrong for the purpose.
The response rate problem
Warehouse response rates are low, and the non-responders are not random.
Agency workers frequently are not surveyed at all, which removes the largest and least satisfied group.
People without a work email or a computer are surveyed last or not at all.
And the shift that was busy that week did not do it.
Which produces a result describing days, permanents and the willing.
The anonymity problem
On a team of twelve, a filter by shift and tenure identifies individuals.
People know this, and answer accordingly.
Which means the honest answers come from the large teams and the diplomatic ones from the small, and the small teams are frequently where the problem is.
The timing problem
A survey in March about a peak in November measures a memory.
And an action plan produced in June from data gathered in March addresses conditions that have changed.
The cycle is longer than the tenure of a substantial part of the workforce, which makes the whole exercise a report on a population that no longer exists.
What to use instead
Ninety-day survival, which is behavioural and cannot be gamed by wording.
Exit conversations a week after leaving.
Stay conversations with a rolling sample.
Referral volume, which is an honest opinion expressed with something at stake.
And attrition by team, which answers the supervisor question that surveys try and fail to ask directly.
When a survey is worth running
Where the population is stable enough for the result to mean something — a long-service core, a site with low turnover.
Short, frequent and specific rather than annual and comprehensive: three questions monthly beats forty questions yearly.
Covering agency workers explicitly.
And with the results acted on visibly and quickly, which is the condition that makes any of it worthwhile.
Reading a score honestly
Compare against your own previous result, not against a benchmark from a different sector.
Cross-check against the behavioural measures.
A rising score with rising attrition means the survivorship effect, and reporting the score alone in that situation is misleading whoever reads it.
Always report attrition alongside, which prevents the most common misreading.
What not to do with the results
Do not set a target on the score, which produces management of the score rather than of the conditions.
Do not compare sites with different agency proportions.
And do not present it as the retention measure, which it is not and was never designed to be.
The measure
Response rate, by group, which shows who is missing.
Score alongside attrition, always together.
Actions taken and closed, which is the only part that changes anything.
And whether agency workers were included, which on most sites is the single question that determines whether the result means anything.
If a survey already exists
Keep it, add attrition alongside every reported score, and include agency workers.
Then add the behavioural measures, which will tell you more and which nobody is currently producing.
How it fails in practice
The score rises while attrition rises, because higher turnover removes more of the dissatisfied before they answer.
The improvement is reported as progress.
The check
Report the score and the attrition figure on the same page, always.
The combination is informative and either one alone is not.