The ability of statistics to accurately represent the world is declining. In its wake, a new age of big data controlled by private companies is taking over – and putting democracy in peril.
begins William Davies tale of woe in the Guardian. Unfortunately, he confuses credible statistics with modern state-istics*; and seems impervious to the idea that Joe Sixpack has wised up to the fact that there are “lies, damned lies, and statistics,” and that most of these are peddled by the Leviathan State and its corporate cronies. Usually to Joe’s detriment.
Statistics in industry and scientific research is doing quite well, thank you. The Big Data movement is still immature and riddled with snake-oil salesmen; it will eventually spot them, possibly by applying its methodologies reflexively.
Tip from that same O’Reilly Newsletter. Finally, I got on a sucker list that’s interesting!
*Where did you think the word came from?
…can be done in 10 minutes or less, using the Jadad score. There’s a full explanation in the original paper, but suffice it to say, it’s pretty easy to identify sketchy studies using this method. Aaron Carroll, writing in the New York Times, shows how this affects the credibility of nutrition research. For those who want to try this at home, here’s the scorecard from the paper:
- Was the study described as randomized? (YES/NO)
- Was the study described as double blind? (YES/NO)
- Was there a description of withdrawals and dropouts? (YES/NO)
Give 1 point for each YES, and 0 points for each NO, with no partial credit. Then assess these
For question 1, GIVE 1 additional point if the method to generate the sequence of randomization was described and it was appropriate (table of random numbers, computer generated, etc.) Otherwise, DEDUCT 1 point if the method to generate the sequence of randomization was described and it was inappropriate (patients were allocated alternately, or according to date of birth, hospital number,etc.)
- For question 2, GIVE 1 addtional point if the method of double blinding was described and it was appropriate (identical placebo, active placebo, dummy, etc.). Otherwise, DEDUCT 1 point if the study was described as double blind but the method of blinding was inappropriate (e.g., comparison of tablet vs. injection with no double dummy).
Hey, it’s not perfect, but then neither is the APGAR score, and look where that’s gotten us.
Tip from Andrew Gelman’s often contrarian Statistical Modeling, Causal Inference, and Social Science blog.
Beware the semi-erudite who thinks he is an erudite. He fails to naturally detect sophistry.
Nicholas Taleb off on a delicious rant.
Tip from Gary Jones, who did make the smallest peep of a comment (so I know he likes it).
*I’m more of an intellectu’all. And I DO deadlift.
I suppose men standing around the barbecue burning meat and drinking beer is just another ritual of the Patriarchy.
Mark Twain was hip to this sort of thinking over a century ago:
In the space of one hundred and seventy-six years the Lower Mississippi has shortened itself two hundred and forty-two miles. That is an average of a trifle over one mile and a third per year. Therefore, any calm person, who is not blind or idiotic, can see that in the Old Oolitic Silurian Period, just a million years ago next November, the Lower Mississippi River was upwards of one million three hundred thousand miles long, and stuck out over the Gulf of Mexico like a fishing-rod. And by the same token any person can see that seven hundred and forty-two years from now the lower Mississippi will be only a mile and three-quarters long. . . . There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact.
–Life on the Mississippi
Writing in The New Atlantis, Daniel Sarewitz says “Science, pride of modernity, our one source of objective knowledge, is in deep trouble.” The public has swallowed the myth of scientism and Vannevar Bush’s self-serving rationalization for federally-funded Big Science:
Scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown.
Sarewitz quotes Richard Horton, editor-in-chief of The Lancet, who puts it like this:
The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.
There’s much, much more, including Alvin Weinberg’s decades-old identification of the problem of trans-science. Read the whole thing.
Tip from Gary Jones, who was apparently so flabbergasted at the completeness of this article that he (uncharacteristically) didn’t even comment.
Looks like potatoes are back on the OK to Eat This list, and the usual thumbsuckers are outraged.
Nutritionist Marion Nestle and other progressive reformers called foul, denouncing the change. “Really?” Nestle scoffed. “I have a hard time believing that WIC recipients are suffering from lack of potatoes in their diets.” Several watchdog groups and the national WIC advocacy group opposed the change, too. “It’s disappointing that politics has trumped science,” Margo Wootan, director of nutrition policy at the Center for Science in the Public Interest, told reporters.
It seems that much of what
our betters the Feds have been telling us about foods and nutrition is a bunch of Just-So Stories:
Rather it’s that the David-and-Goliath narrative of science versus Big Ag may be blinding us to another, even bigger problem: the fact that there is often very little solid science backing recommendations about what we eat.
Most of our devout beliefs about nutrition have not been subjected to a robust, experimental, controlled clinical trial, the type of study that shows cause and effect, which may be why Americans are pummeled with contradictory and confounding nutritional advice.
Any day now, I expect to hear that I should add a shot of tequila to my ideal breakfast of steak and (whole) eggs…and potatoes.
Update (14 April): There’s evidence to suggest that we can improve the good carbs-bad carbs ratio by changing the way we prepare starchy foods. I’ve replaced mashed and roasted potatoes with cooked-then-chilled potatoes (mmm, spicy potato salad).
David Warren, like me, is tired of being beaten about the head and shoulders with observational studies masquerading as “settled science.”
According to the latest research, he writes facetiously, coffee may be good for your heart….Actual science would show the mechanism by which a specific constituent in coffee, such as caffeine, operates within the human metabolism to produce specific reactions in a long, very specific chain, leading to a specific result. … The rest is, to be perfectly colloquial, bullshit,
How many times must we tell the hoi polloi, CORRELATION IS NOT CAUSATION?
Tip from American Digest.