Category Archives: Blog article

What can the trial of Italian Scientists teach market researchers?

The Economist has an interesting report about seven scientists in Italy who are on trial on charges of manslaughter. The prosecution follows their recommendation, on 31 March 2009, that the tremors felt around the Italian city of L’Aquila posed ‘no danger’. Six days later there was a series of geological events that resulted in 308 deaths.

The scientists say that when they said there was no danger, they meant that the danger was at its normal level for a city in an earthquake zone, not that there was NO danger. They are suggesting that the prosecution is invalid because people who live in an earthquake zone understand risk.

A second point that is being made is that the scientists were probably (or is that possibly) wrong in their comments. The Economist reports that other scientists have suggested that the background risk of a major earthquake in L’Aquila is about 1-in-200,000, but that after the minor tremors the risk had increased to 1-in-1000. Which raises the question whether a city should be evacuated every time it has a 1-in-1000 risk? If yes, then there will be losses, wasted time, and road deaths. If no, then 1-in-1000 times there will be a disaster.

Although this affair is more important and more tragic than market research, there are, nevertheless, two useful lessons for market researchers that arise from it.

1) People tend not to understand statistics. When I say people, I mean both experts and the public. Do not assume that your client has grasped the statistical interferences in your findings, make them tangible and test whether you have communicated the important knowledge (not just the facts).

2) Don’t appear more certain than you are. Clients do not want debriefs that just regurgitate statistics, they want insight and they want recommendations. But they do not want to be told they are safe when they are not! Rather than say “the data show”, tell the truth, such as “I believe this means …”, “We think it is likely that ….”, or “The most likely explanation is …”

This is not a new problem for market researchers. Back in 1987, Yankelovich Clancy was sued by Beecham because it forecast sales growth that did not materialise. Market research needs to convey its findings in a way that allows the client to make a better decisions, the research should not seek to replace the client’s responsibility for the decision. Without resorting to statistical devices, such as error margins and distributions, researchers need to ensure clients understand the degree of trust that they should put in the information they are being presented with.

It’s time for market research to join 21st Century

There are a large number of discussions and consultations going on at the moment about initiatives from ESOMAR, CASRO, the MRS and others to try to regulate how social media research should be conducted, especially social media monitoring. The general thrust of the new guidelines is to try and fit the new world into the traditional values and ideas of market research. I think this is the wrong way to go about the change.
I think we need to change the whole of commercial market research to match the 21st Century, rather than try to keep shoehorning the new world into the old constructs.
My feeling is that there will soon be a schism in market research, between those trying to hang onto the past and those embracing the new.
The benefits of traditional market research ethics were that they allowed some exemptions to laws (e.g. data protections laws, laws about multiple contacts, laws about phoning people who were on ‘no call’ lists), increased public trust, and allowed market research to get close to a scientific model – for example to use concepts such as random probability sampling and statistical significance. Complying with codes of ethics incurred extra costs, but they also brought commercial benefits. The ‘proper’ market research companies could do things the non-research companies could not- so there was a commercial argument in favour of self-regulation, codes of conduct, and professional conduct bodies.
However, in several areas, ‘new’ market research is at odds with the traditional guidelines. Examples of where NewMR is at odds with the traditional ethics includes: the brand-related incentives for members of communities, the brand advocacy of community members, the changes wrought by deliberative research, and most of social media monitoring research. Other areas where research is drifting away from the classic model of anonymity include a growing amount of customer satisfaction and most of enterprise feedback systems.
Traditional market research is based on a) anonymity and b) informed consent. Large parts of new market research cannot deliver anonymity and in the area of social media research (and behavioural data integration) informed consent cannot be reliably assumed either.
If market research companies abide by the old ethics, in particular anonymity and informed consent, they will not be able to compete for business in most areas where market research is growing. This is because there will be no commercial benefits that will accrue to sticking to rules and ideas that nobody else does. To stick to out-dated rules simply provides a worse service for clients. Rules have costs, they only work when they also confer benefits.
The view of people like the UK’s MRS is that all of the ‘stuff’ that does not match the traditional view of market research should be done as “NOT market research”. The problem with this solution is that it will soon classify the majority of market research as “NOT market research” which is clearly nonsense.
My remedy is that commercial market research should be split from genuine social research (by social research I mean the stuff that is not done principally for commercial reasons, such as some of the research by Governments, academics, and NGOs). Social research should keep the traditional values of ethics and commercial market research should fully embrace the new world. The ethics of NewMR should be based on:
  1. The law
  2. Not doing things likely to outrage the public
  3. Creating high standards (and that can include charter marks and ISOs for those interested)
  4. Emphasising the need to be open and honest
Note the case for charter marks and ISOs should not be based on theoretical arguments, but simply on whether they confer commercial benefits. If signing up to an ISO means that market research companies are able to win more work, then the ISO is s a good thing. If the ISO simply makes the industry feel better about itself, the ISO is a bad thing.
These four principles would, for example, mean that if a company told respondents that the study was anonymous and that they would not be contacted, then it would have to be anonymous and there would have to be no follow-up contact – that would be the law in many countries (because a contract has been entered into) and failing to stick to a promise would outrage the public.
Similarly, the four principles would outlaw using a false identity to access a closed community (for example PatientsLikeMe) and surreptitiously scraping comments to be sold to a third party – i.e. the Nielsen scrape-gate case. I suspect that not only would this outrage the public and damage the value of the company, but it could easily fall foul of civil suits, where members of the community could sue for damages.
This model of market research ethics changes the balance of who determines what can and should be done. In the traditional market research model the rules were set by the wise market researchers, to protect respondents and brands. My suggestion is that respondents should determine what can and should be done with their data, and that citizens should set the framework.
What do you think?

Running the Saints Way for Coppafeel

Saints Way

Saints Way

Every year I like to set myself a physical challenge and I like to utilise that challenge to raise money for charity. This year I am going to run the Saints Ways in Cornwall, from Padstow on the North coast of Cornwall to Fowey on the South Coast. We are still tweaking the route but it is likely to be just under 30 miles (about 48 KM).

The charity I will be running for is CoppaFeel, a breast cancer awareness charity organised by two friends of my daughter. Even if you do not follow my progress, even if you do not dontate, please read Kris’s story, read how she developed breast cance at 23 and the problems she had getting her GP to believe her. Kris and her sister Maren created CoppaFeel to raise awareness of breast cancer amongst younger people.

I am going to be running the Saints Way on 17 August, and I will be blogging and Tweeting about my progress, and hoping to persuade my longsuffering family to support me, again.

You can make a donation via my page on JustGiving