The Economist has an interesting report about seven scientists in Italy who are on trial on charges of manslaughter. The prosecution follows their recommendation, on 31 March 2009, that the tremors felt around the Italian city of L’Aquila posed ‘no danger’. Six days later there was a series of geological events that resulted in 308 deaths.
The scientists say that when they said there was no danger, they meant that the danger was at its normal level for a city in an earthquake zone, not that there was NO danger. They are suggesting that the prosecution is invalid because people who live in an earthquake zone understand risk.
A second point that is being made is that the scientists were probably (or is that possibly) wrong in their comments. The Economist reports that other scientists have suggested that the background risk of a major earthquake in L’Aquila is about 1-in-200,000, but that after the minor tremors the risk had increased to 1-in-1000. Which raises the question whether a city should be evacuated every time it has a 1-in-1000 risk? If yes, then there will be losses, wasted time, and road deaths. If no, then 1-in-1000 times there will be a disaster.
Although this affair is more important and more tragic than market research, there are, nevertheless, two useful lessons for market researchers that arise from it.
1) People tend not to understand statistics. When I say people, I mean both experts and the public. Do not assume that your client has grasped the statistical interferences in your findings, make them tangible and test whether you have communicated the important knowledge (not just the facts).
2) Don’t appear more certain than you are. Clients do not want debriefs that just regurgitate statistics, they want insight and they want recommendations. But they do not want to be told they are safe when they are not! Rather than say “the data show”, tell the truth, such as “I believe this means …”, “We think it is likely that ….”, or “The most likely explanation is …”
This is not a new problem for market researchers. Back in 1987, Yankelovich Clancy was sued by Beecham because it forecast sales growth that did not materialise. Market research needs to convey its findings in a way that allows the client to make a better decisions, the research should not seek to replace the client’s responsibility for the decision. Without resorting to statistical devices, such as error margins and distributions, researchers need to ensure clients understand the degree of trust that they should put in the information they are being presented with.