How to Overcome Judgement Noise
As human beings, our judgements can be strongly influenced by irrelevant factors. Even professionals with rich experience on the job treat identical cases quite differently at different time points, according to studies.
Kahneman et al. (2016) termed the chance of such variability as
(judgement) noise, which has been identified by their study as one of the most prevalent but overlooked problems that harm companies' bottom line.
WHAT IS NOISE?
Like bias, noise (aka random error, random variation, variability) is
a type of error in judgement and decision-making. However, biases cluster together while noises widely scatter when both are compared with what the accurate one is (variability). An increase in noise always harms accuracy when there is no bias in system. When bias is presented, increasing noise may actually cause a lucky hit. Of course, no organization would put its trust in luck, therefore noise is always undesirable.
HOW TO MEASURE NOISE
Unlike bias, noise is noticeable even without knowing what accuracy would be. Kahneman et al. (2016) believe judgment noise can be measured by
noise audits, which involve a process of independent evaluation on a few realistic cases that cover the range of problems typically encountered. The aim of the audits is to work out a
noise index on each individual case, allowing executives to recognize if the judgments of their professionals are unreliable to an extent that can not be tolerated.
HOW TO DIAL DOWN THE NOISE
The most radical solution to judgment noise is to replace human judgment with formula rules. Although in many situations such formula rules won’t be practical, in other situations developing and implementing reasoned rules can be surprisingly easy and effective if the conditions are right. Often reasoned rules can be used as an intermediate source of information for professionals to make their final decisions. Managers must monitor and adjust the rules for occasional changes in the population of cases, keep an eye on individual decisions, and have the authority to override the rules in clear-cut cases.
Controlling noise is hard, but for many companies conducting a noise audit and evaluating the cost of noise in monetary term will help them to recognise that reducing variability is worth the effort.
Source: Kahneman, D., Rosenfield, A., Gandhi, L. and Blaser, T. (2016). Noise: How to Overcome the High, Hidden Cost of Inconsistent Decision Making. Harvard Business Review, 94(10), pp.38 - 46.
