William Briggs, Ph.D. (a statistician and meteorologist) gives a talk on the replication problem and how much bad Science is done.
The problem, I think, is not that Science produces incorrect results but that media spreads “a new study says” long before we know the problems of the paper. There is no “recall” in the media – we know from persuasion/propaganda that the first thing someone hears is what they remember, even if that is subsequent shown as not to be true.
Reporters are mostly BAs in English, Political science or journalism, and some have an MA in Journalism or an MFA in creative writing – and some lack even these credentials. Only a few have the knowledge needed to skeptically question the science and the experts. Instead, many reporters nearly worship science and are unable to question their deities.
The problem then isn’t “bad science” (in fact, bad results can lead to better studies) – the problem is university public relations departments, staffed typically by people trained in journalism, who highlight sound bites, exaggerate the importance of the study and conclusions, and ignore the limitations – and that it might even be all wrong. These press releases are then picked up and regurgitated by reporters eager to exaggerate results (especially if really scary!), neglect limitations and doubts, and reporters’ inability to question studies critically and skeptically.
This talk is fascinating and parts are hilarious. I’ve tagged this as a “Follow up” as it is relevant to the topics covered under Follow Up – that is – expert predictions of the future routine turn out wrong, and this is, to some extent, why that happens.
