Science fiction: The creeping threat of survey abuse
The perfect sitcom (that’s (rd+vf/a+s, in case you were wondering), the perfect day (quality = O+NS+Cpm/T+He) and the perfect rugby kick (the formula’s too long to bother with here) have two things in common. The first is that they are basically made-up research with no statistical basis. The second is that that didn’t stop them getting extensive mainstream press coverage.
Now Simon Singh, doctor of physics, MBE, author of several of the UK’s bestselling science books and occasional journalist, is taking aim at the ‘experts’ who give birth to these spurious research findings and the journalists who give them notoriety. In a lecture for British Science Week on Sunday he announced a (so far unspecified) prize for the most ridiculous equation that he discovers in 2010. He’s inviting you to email your submissions to voys@senseaboutscience.org and, in that open way that scientists have with debate, to suggest a name for the award.
“Everybody seems happy with these equations,” he said in his lecture, “but for me, they are a real problem.”
The reason he is unhappy, and the reason it concerns all of us, is not just that the public are getting a false impression of any type of research, it’s that they are becoming unable to distinguish potentially important, well-researched insight from made-up garbage that’s designed only to sell a product or an idea. Singh quotes a fellow scientist, Len Fisher, who comes up with some of these equations (how to pour gravy is one) – but only when he believes there is real intellectual content. Fisher is continually asked to provide pseudoscience solely to market a product, which he refuses to do. “Some of them [the PR companies] seemed genuinely astonished that prejudging issues and then providing proof is not science,” he says.
The same goes for the ‘surveys’ that make the news, complete with the ‘experts’ who validate them with sensational quotes which may or may not be supported by the evidence. Or the sensational internet polls that ‘prove’ some pre-determined editorial point. For many members of the public, just as the formula for the perfect chip (commissioned by Tesco) is science, so these surveys are market research. In this, the market research industry has common cause with Singh and his fellow academics: when MR is so easily traduced, then important research will inevitably be lost in the noise.
So who does Singh blame? Not, you may be surprised to learn, the PR companies that give life to this. Or, indeed, their clients who have paid for the pre-determined marketing-based result. He blames the people who lend their names and their doctorates to the press release.
“It would not happen if there were more mathematically literate people working in newsrooms who could put a stop to it,” he says. “But when someone comes up with this we all know it’s not real.” Therefore, he reasons, the only way to stop it is to stop it at source.
For mathematicians, who (most of the time) value the respect of their peers and their integrity above a cheap headline and a cheque from a PR company, this might be possible. But for market research, where anyone with an internet connection can produce a survey with alluring statistics and pie charts attached, the brake is harder to apply. Can anyone hold back this tide?
Tim Phillips is a Research contributor and writer of the Talk Normal blog.

We hope you enjoyed this article.
Research Live is published by MRS.
The Market Research Society (MRS) exists to promote and protect the research sector, showcasing how research delivers impact for businesses and government.
Members of MRS enjoy many benefits including tailoured policy guidance, discounts on training and conferences, and access to member-only content.
For example, there's an archive of winning case studies from over a decade of MRS Awards.
Find out more about the benefits of joining MRS here.
1 Comment
John Grono
16 years ago
Research shows that 117.8% of all qualified, mathematicaly literate, researchers agree with Tim. Seriously though, I applaud Tim's effort and approach. We're burdened with this issue down here in Australia as well - especially in the online sphere - and vigilant debunking on various blogs and posts seems to be the only way currently to counter this plague.
Like Reply Report