This sounds like a great innovation, using Raman spectroscopy to detect melanoma. But let’s look a little deeper at the preliminary results.

First, the developers claim that in 274 known cases of melanoma, the device detected all 274 of them, which gives and estimated test **sensitivity** of 100%, which is patent bullshit; nothing in this life is guaranteed perfect. A reasonable Bayesian shrinkage estimator might be 275/276 = 0.9964, which is still pretty good. Second, the developers conveniently omitted any statistics on **specificity** (how well cases of NO melanoma are correctly identified); I’m sure this was simple oversight*. Third, and finally, there’s no mention of the **prevalence** of melanoma in the general population. Without all three of these numbers, we have no way of evaluating the utility of the device.

But what the heck, let’s do a back of the envelope** calculation. We have the estimate for sensitivity, and we’ll be optimistic and assume that the other 726 subjects were all melanoma-free and tested negative, so we’ll get a shrinkage estimate of 727/728=0.9986 (which should make it a pretty good test). Then look up the prevalence of melanoma (also called incidence rate) to get 1/5074 = 0.000197. Then plug into Bayes’ Theorem to get the posterior predictive value:

PPV = (sensitivity x prevalence) / (sensitivity x prevalence + (1-specificity)x(1-prevalence)) = 0.1251

So if you test positive with this gizmo, you have about a 12.5% chance ( 1 in 8 ) of having melanoma. I think we need more test data before buying any stock.

*Tip from the Instapundit*, who should calm down.

* I’m sure there’s an Easter Bunny, too.

** OK, back of the spreadsheet

**Update** (8 February). Jan Nordgren has the perfect quote to describe this situation. I wish I had used it for my title.

### Like this:

Like Loading...