Archive for November, 2012

“Chocolate Consumption, Cognitive Function, and Nobel Laureates”
Franz H. Messerli, M.D.
N Engl J Med 2012; 367:1562-1564

All Journals like publicity. it increases readership, impact, subscriptions, reputation. Presumably this is why the New England Journal of Medicine published a piece of statistical nonsense about the link between chocolate consumption and Nobel prizes.

The NEJM is one of the medical world’s top journals, with a high impact rating. It is difficult, and prestigious, to get published in it. That is presumably why it felt unable to give space (even a letter of 175 words) to any of the replies it received to this piece of statistical garbage.

Either the editors of the NEJM failed to notice statistical howlers any half competent undergraduate could be relied upon to spot, or they took a cynical decision to devote precious space to a piece which they knew well was scientific rubbish, believing (absolutely correctly as it turned out) that it would bring them a publicity windfall.

Is it churlish to criticise what could or should have been seen as a ‘bit of fun’ or lighthearted nonsense? I donĀ“t think so. One only needs to google the story to see that the papers which carried it – even the serous or broadheet titles – usually took it at face value: it was written by a doctor after all and published in a scientific journal. (The inability of journalists to discern good data from bad is a another story for another day.) This has a corrosive and insidious effect. It helps cultivate the perception that statistics are something that can prove anything, because that is what bad statistics can always claim to do. This in turn helps to reinforce the perception that all statistics are rubbish, including the robust ones produced by careful research, of the kind NEJM depends upon on all its other pages, and upon which we all ultimately rely not only in evidence based medicine, but evidenced based everything.

The NEJM should be ashamed of debasing statistics in this way. However this cloud has one, small, silver lining. The article is wonderful example that can be used to teach students about the ecological fallacy, prior variables, causation and correlation and measurement error. The article has these in spades. A paper based upon a (rejected) reply to the NEJM by colleagues at Edinburgh (ChocolateSerialKillers_WintersRoberts) goes through some of them, and inter alia points out that similar evidence points to a link between chocolate consumption and serial killers. I’ve also produced a NEJM study comments that can be used in tutorials or seminars.

Read Full Post »