Category Archives: Ethics

Is the MRS courting doom with its privacy proposals?

The MRS has been consulting on new guidelines for online privacy, particularly with a focus on the issues arising out of social media. Some of the ideas being floated by the MRS have caused a shocked and angry response from market researchers, some of which were reported by Research-Live.

Earlier this week I met a wide range of UK researchers at the ESOMAR Congress in Amsterdam and was amazed by the anger that many researchers expressed about the some of the ideas that the MRS appeared to be promoting. Indeed, several researchers said that if the ‘worst’ of the ideas in the consultation ended up in the code they would leave the MRS and they would encourage their employers to stop funding MRS membership.

If this convenience sample turns out to be representative (and there is no statistical reason why it should) the MRS could be in real trouble if it continues with its current direction.

My position is more nuanced. I think the ideas in the consultation are crazy in places and harmful to our industry without being good for anybody. However, I will still be in the MRS, even if they are adopted. But, I will be describing most of what I do as “not market research”, which is a shame.

What can the trial of Italian Scientists teach market researchers?

The Economist has an interesting report about seven scientists in Italy who are on trial on charges of manslaughter. The prosecution follows their recommendation, on 31 March 2009, that the tremors felt around the Italian city of L’Aquila posed ‘no danger’. Six days later there was a series of geological events that resulted in 308 deaths.

The scientists say that when they said there was no danger, they meant that the danger was at its normal level for a city in an earthquake zone, not that there was NO danger. They are suggesting that the prosecution is invalid because people who live in an earthquake zone understand risk.

A second point that is being made is that the scientists were probably (or is that possibly) wrong in their comments. The Economist reports that other scientists have suggested that the background risk of a major earthquake in L’Aquila is about 1-in-200,000, but that after the minor tremors the risk had increased to 1-in-1000. Which raises the question whether a city should be evacuated every time it has a 1-in-1000 risk? If yes, then there will be losses, wasted time, and road deaths. If no, then 1-in-1000 times there will be a disaster.

Although this affair is more important and more tragic than market research, there are, nevertheless, two useful lessons for market researchers that arise from it.

1) People tend not to understand statistics. When I say people, I mean both experts and the public. Do not assume that your client has grasped the statistical interferences in your findings, make them tangible and test whether you have communicated the important knowledge (not just the facts).

2) Don’t appear more certain than you are. Clients do not want debriefs that just regurgitate statistics, they want insight and they want recommendations. But they do not want to be told they are safe when they are not! Rather than say “the data show”, tell the truth, such as “I believe this means …”, “We think it is likely that ….”, or “The most likely explanation is …”

This is not a new problem for market researchers. Back in 1987, Yankelovich Clancy was sued by Beecham because it forecast sales growth that did not materialise. Market research needs to convey its findings in a way that allows the client to make a better decisions, the research should not seek to replace the client’s responsibility for the decision. Without resorting to statistical devices, such as error margins and distributions, researchers need to ensure clients understand the degree of trust that they should put in the information they are being presented with.

It’s time for market research to join 21st Century

There are a large number of discussions and consultations going on at the moment about initiatives from ESOMAR, CASRO, the MRS and others to try to regulate how social media research should be conducted, especially social media monitoring. The general thrust of the new guidelines is to try and fit the new world into the traditional values and ideas of market research. I think this is the wrong way to go about the change.
I think we need to change the whole of commercial market research to match the 21st Century, rather than try to keep shoehorning the new world into the old constructs.
My feeling is that there will soon be a schism in market research, between those trying to hang onto the past and those embracing the new.
The benefits of traditional market research ethics were that they allowed some exemptions to laws (e.g. data protections laws, laws about multiple contacts, laws about phoning people who were on ‘no call’ lists), increased public trust, and allowed market research to get close to a scientific model – for example to use concepts such as random probability sampling and statistical significance. Complying with codes of ethics incurred extra costs, but they also brought commercial benefits. The ‘proper’ market research companies could do things the non-research companies could not- so there was a commercial argument in favour of self-regulation, codes of conduct, and professional conduct bodies.
However, in several areas, ‘new’ market research is at odds with the traditional guidelines. Examples of where NewMR is at odds with the traditional ethics includes: the brand-related incentives for members of communities, the brand advocacy of community members, the changes wrought by deliberative research, and most of social media monitoring research. Other areas where research is drifting away from the classic model of anonymity include a growing amount of customer satisfaction and most of enterprise feedback systems.
Traditional market research is based on a) anonymity and b) informed consent. Large parts of new market research cannot deliver anonymity and in the area of social media research (and behavioural data integration) informed consent cannot be reliably assumed either.
If market research companies abide by the old ethics, in particular anonymity and informed consent, they will not be able to compete for business in most areas where market research is growing. This is because there will be no commercial benefits that will accrue to sticking to rules and ideas that nobody else does. To stick to out-dated rules simply provides a worse service for clients. Rules have costs, they only work when they also confer benefits.
The view of people like the UK’s MRS is that all of the ‘stuff’ that does not match the traditional view of market research should be done as “NOT market research”. The problem with this solution is that it will soon classify the majority of market research as “NOT market research” which is clearly nonsense.
My remedy is that commercial market research should be split from genuine social research (by social research I mean the stuff that is not done principally for commercial reasons, such as some of the research by Governments, academics, and NGOs). Social research should keep the traditional values of ethics and commercial market research should fully embrace the new world. The ethics of NewMR should be based on:
  1. The law
  2. Not doing things likely to outrage the public
  3. Creating high standards (and that can include charter marks and ISOs for those interested)
  4. Emphasising the need to be open and honest
Note the case for charter marks and ISOs should not be based on theoretical arguments, but simply on whether they confer commercial benefits. If signing up to an ISO means that market research companies are able to win more work, then the ISO is s a good thing. If the ISO simply makes the industry feel better about itself, the ISO is a bad thing.
These four principles would, for example, mean that if a company told respondents that the study was anonymous and that they would not be contacted, then it would have to be anonymous and there would have to be no follow-up contact – that would be the law in many countries (because a contract has been entered into) and failing to stick to a promise would outrage the public.
Similarly, the four principles would outlaw using a false identity to access a closed community (for example PatientsLikeMe) and surreptitiously scraping comments to be sold to a third party – i.e. the Nielsen scrape-gate case. I suspect that not only would this outrage the public and damage the value of the company, but it could easily fall foul of civil suits, where members of the community could sue for damages.
This model of market research ethics changes the balance of who determines what can and should be done. In the traditional market research model the rules were set by the wise market researchers, to protect respondents and brands. My suggestion is that respondents should determine what can and should be done with their data, and that citizens should set the framework.
What do you think?