The survey found that UK adults expressed very negative views of Lloyds Bank in response to a number of questions – with 60% of all respondents saying the bank should be stopped from using the line “By your side” in its ad campaign.
But a number of the questions asked in the survey include background information that Kellner said could produce different results to a question asked without such information.
On the advertising slogan, the full question asked was:
“British Airways changed its advertising slogan – ‘The world’s favourite airline’ – because it could no longer substantiate the claim. Noel Edmonds recently complained to the Advertising Standards Authority, asking for Lloyds’ adverts to be suspended as they misrepresented the truth about the brand, specifically the slogan ‘By your side’. What would be your decision if you were asked whether Lloyds should be allowed to use the line in ads?”
Kellner said: “The normal purpose of a survey is to measure public opinion as it is, given the degree of knowledge respondents have before the survey. As soon as the question includes information, especially the kind of extensive and detailed information in a number of questions in this survey, this ceases to be the case.
“The results cannot be presented as the views of the general public, because the very fact that people are responding to information they did not have before, means they become different from the general public.
“This does not mean it is always wrong to provide respondents with information. But, on any contentious issue, the results are very likely to reflect the choice and presentation of the facts in the question. Even minor difference in the information given can produce significantly different responses.”
Kellner added that the survey had “broken no hard-and-fast rules about questionnaire design; but great care needs to be taken in the way the results are used.” He called ComRes a “good company” that he expected would warn clients about the limitations of this approach.
Edmonds and Therium, his litigation funder, worked with Mark Palmer, founder of independent change consultancy Maverick Planet, on the survey. Palmer, who has a background in media agencies, gave a vigorous defence of ComRes and the survey after it was called into question by Fiona Blades of Mesh Experience.
“Comres is a major and highly regarded independent research company”, Palmer said. “They had input on the methodology and the questionnaire design to ensure it wasn’t leading. They respect and adhere to the Marketing Research Society Code, and the research went out on this basis.
Palmer added that they had been “totally transparent about the questions asked”, and pointed out that all information given in the survey was factually correct.
He defended the decision to include the information about British Airways in the question on Lloyds’ slogan, saying that as consumers don’t necessarily understand the process used by Advertising Standards Authority, it made sense to provide a relevant example. BA was chosen because it was a “fairly well known brand and strapline”, he said.
