The breakthrough test

This one’s been dipped in the fountain of Zum Zum, too.

The Telegraph has moved on from wonder drugs and fountains of youth to a breakthrough test that ‘detects prostate cancer in less than 15 minutes.’  

Overall, it's been a bad few weeks for health reporting with the overhyping of Lecanemab wonder drug status and Ozempics offering as the “fountain of youth.” 

However, it's crucial to approach this latest news story critically. While the prospect of a new test that could detect prostate cancer in less than 15 minutes is promising, it also highlights the importance of understanding test results and the frequency of overhyped diagnostic accuracy claims. 

According to The Telegraph, ‘The “breakthrough” blood test was able to diagnose men with prostate cancer to 90 per cent accuracy in a new study.’

Before delving into the details here in the TTE office, it's worth mentioning that we have an extensive research team that has noticed many things over time. In 2010, CH wrote in The Guardian about "Why autism can't be diagnosed with brain scans."

The TTE office wants to take you back in time, as little has escaped our extensive researcher teams' notice.

In 2010, CH wrote in the Guardian about Why autism can't be diagnosed with brain scans.

Both reports have the same 90% accuracy. However, they both make a common mistake in interpreting diagnostic test accuracy.

They confuse sensitivity with positive predictive value. Sensitivity answers the question, "If I have the disease, do I have a positive test?" On the other hand, positive predictive value answers the question, "If I have a positive test, do I have the disease?"

The study (which is hard to find in the Telegraph) reports the results in Table 3, indicating a sensitivity of approximately 90% and a specificity of 90% (the proportion of people without the disease who test negative).

Stick with us; this is all a bit confusing. We’ll explain further what the test results mean. 

Let's take a 65-year-old as an example. The prevalence of prostate cancer for this age group is about 1 in 20. With this information, we can calculate the test's positive predictive value.

If we imagine 1,000 65-year-olds, 50 (5%) will have prostate cancer. Out of these 50, 45 will have a positive test, and five will be missed and have a negative test, leading to the reported 90% accuracy by the media.

For the 950 who do not have the disease, 95 (10%) will test positive, and 855 (90%) will test negative. This leads to 140 positive tests, of which only 45 will have the disease, resulting in a positive predictive value of 32%. It's not quite a breakthrough when you consider how many people will be falsely identified as positive for the disease.

In addition, a diagnostic study needs to include a wide range of disease severity, from mild to severe cases. Prostate cancer can vary from aggressive forms (you die of prostate cancer) to slowly progressing forms (you die with prostate cancer after living in fear for X time). Most studies also require an unbiased comparison of test results with a reference standard among a consecutive series of patients. None of this occurs in the study as it is done in a lab.

Fourteen years ago, we wrote to the author of the original autism study, seeking clarification. We understand that your expertise is crucial in understanding the study, as you stated: "It is currently unknown how these values generalise to the entire population, and across all dimensions of the autistic spectrum, which is why we have clearly stated that we are not yet ready to make our approach available in the NHS just yet."

No breakthrough in autism diagnosis has occurred as predicted by the media, and the same simple error in test accuracy continues to befuddle journalists. Once you put it all together, there is no breakthrough. 

This post was written by two old geezers who drowned in the Styx, trying to reach the waters of Zum Zum.

We shall post a short video explainer later today for those still confused by test accuracy.