Gee Whiz, GraphViz!

Sometimes you need to draw a simple network diagram, like this Hasse diagram

SimpleSplitPlot

but you don’t have a good graph drawing tool.  Get Graphviz!  Easy to learn, scriptable, and FREE.

Advertisements

The Fourier Transform, explained beautifully

At the Better Explained blog, Kalid Azad hits another home run with An Interactive Guide to the Fourier Transform.

Here’s a plain-English metaphor:

  • What does the Fourier Transform do? Given a smoothie, it finds the recipe.
  • How? Run the smoothie through filters to extract each ingredient.
  • Why? Recipes are easier to analyze, compare, and modify than the smoothie itself.
  • How do we get the smoothie back? Blend the ingredients.

Here’s the “math English” version of the above:

  • The Fourier Transform takes a time-based pattern, measures every possible cycle, and returns the overall “cycle recipe” (the amplitude, offset, & rotation speed for every cycle that was found).

smoothie-to-recipe

Tip from Kotke, who has a cool Fourier Transform video.

Whether to ask the question invites an answer

The Justice Department and the Census Bureau are engaged in a kerfuffle over the 2020 Census.  It’s all about a question of citizenship: “What country are you a citizen of?”  With the inevitable congressional reapportionment that occurs based on the Census, this is a question that many states really don’t want to know the answer to.

My take:  the Census Bureau has been crying poor for years now.  The Trump Administration should jawbone Congress into increasing the Bureau’s funding, but only if they ASK THE QUESTION (and report the answers).

Update:  Now folks should really be worried.  Combine citizenship data with Google location data (“we have ways to make you opt in”), and some dedicated data miners could find every Android-using illegal alien in the country.

Tips from the Instapundit, where the signal-to-noise ratio seems to be increasing lately.

Update:  It has come to my attention that at least one other branch of the federal government already ASKS THE QUESTION, to wit, the Justice Department’s Bureau of Alcohol, Tobacco, Firearms, and Explosives* E-Form 4473, Firearms Transaction Record has Questions 12 and 13:

ATFE4473_AskTheQuestion

In other words, you cannot exercise your 2nd Amendment right to own a firearm unless you ANSWER THE QUESTION.

* Or what I call a Redneck Hedge Fund.

Our National Blind Spot

Want to save the planet?  How about starting by saving the birds.  Here’s a Pareto graph that gives a strong hint of where to start:

BirdMortality

That’s right, get the cat population under control.  Eradicate feral cat colonies, and euthanize cat collections (oh, and institutionalize obsessive cat ladies).  The whole country needs to grow up and get that “cute little kitty” lie out of their heads, and replace it with something more realistic, like “bird murderer.”

Tip from Bird Note, by way of Sarah Hoyt at the Instapundit.

Update:  One Dallas suburb is infested with feral cats, protected by a well-connected cat lady.

Science is getting exciting!

Five very interesting articles recently popped up on the web, suggesting that current science is much more interesting than the average Joe might think:

    • At FiveThirtyEight*, Christie Aschwanden’s Science Isn’t Broken gives a great exposition on scientific fraud, p-hacking, and why science is much more difficult than most folks realize.
    • Robert Matthews, writing in UAE’s The National, says Lone researchers with radical ideas may hold the keys to science’s unanswered questions.  One of those “loners” is “Eleonora Troja, an astronomer at NASA’s Goddard Space Flight Center who studies X-rays, had hoped for years to detect the light from a neutron-star merger, but many people thought she was dreaming.”  
    • FiveThirtyEight’s Rebecca Boyle,  in Two Stars Slammed Into Each Other And Solved Half Of Astronomy’s Problems. What Comes Next?, describes that dream coming true and a revolution in astronomy that occurred in just 3 weeks this past August.
    • In The Serial-Killer Detector, the New Yorker’s Alec Wilkinson tells the story of Thomas Hargrove’s one-man Big Data project to categorize and analyze murders in the United States (751,785 since 1976) with the goal of tracking down serial killers.  From the description, is appears Hargrove has done yeoman’s work combining Small N and Big Data techniques with great success. “Hargrove thinks … that there are probably around two thousand serial killers at large in the U.S.”  Yikes!
    • Want to get in on the action?  At ScienceAlert.com, Mike McRae tells how Now You Can Build Your Very Own Muon Detector For Less Than $100, and possibly contribute to a Big Data project supporting stellar astronomy.

*ESPN’s website that analyzes sport statistics, election polling, and (apparently) anything else that catches their analysts’ eyes.

 

When all you have is a hammer…

…everything looks like a nail.

Daniel Lakens, the 20% Statistician, takes a rare but easy shot at statisticians and null hypothesis significance testing.

Our statistics education turns a blind eye to training people how to ask a good question. After a brief explanation of what a mean is, and a pit-stop at the normal distribution, we jump through as many tests as we can fit in the number of weeks we are teaching. We are training students to perform tests, but not to ask questions

He defines

…the Statisticians’ Fallacy: Statisticians who tell you ‘what you really want to know’, instead of explaining how to ask one specific kind of question from your data.

My favorite is the two-tailed test of the difference of two means, which can provide evidence that the two are different, but not that they are (nearly) the same.  My runners up are goodness-of-fit tests, which do no such thing.  Sometimes I feel like I’m selling the researcher’s version of Snake Oil, rather than teaching sound data analysis and interpretation.

Lakens closes with an excellent addendum, a reference to David Hand’s Deconstructing Statistical Questions,  which goes into much more detail.

Seven Pillars

Wisdom hath built her house, she hath hewn out her seven pillars.  –Proverbs 9:1

I just finished Stephen Stigler’s The Seven Pillars of Statistical Wisdom, and I’m daunted–and embarrassed that I waited so long to read it.  Stigler gives us a structure and taxonomy to statistical thinking* that gives us the “big picture” of statistics.

StiglerSevenPillars

Quite a difference from the descriptives-to-inference-to-models approach that most textbook authors follow.  This is making me rethink how I approach my introductory courses, especially those for statistics majors.  I’m starting with a baby step: adding the (inexpensive, paperbound) book as a required reading in my statistical research methods class.

*the 7 pillars: aggregation, information, likelihood, intercomparison, regression, design, and residual (and that’s just the table of contents!)