Know Your Priors
Vol #48 - July 30th, 2021
Percentages, rates, and ratios seem designed to be misleading.
At the same time as they are maddeningly precise, they are also deviously misleading.
The first thing I think about when I hear a statistic is - ‘why could this be wrong?’.
It’s true with markets, medicine, and everything in between. The world is awash with data scientists and machine learning, but simple statistics can be particularly confounding.
An obvious example in financial markets is when journalists quote how many points the market increased by. It’s fun to read a headline that says “Dow rose 500 points today”, but even I usually have to look up what the actual index value is today to get an idea of what that means.
With the pandemic stretching long into its second year, we’re bombarded with numbers tracking the course of the virus. As vaccinations roll out and new variants of COVID-19 crop up, there are dozens of facts and figures about efficacy, infection and reinfection rates.
Public health officials are struggling to clearly communicate policy for a number of reasons, and counter-intuitive statistics don’t help matters.
One of the most important principles in statistics is Bayes Theorem. It is a concept of conditional probability, and how incorporating new information can update forward looking probabilities.
Take an example of a drug test that's known to be 98% accurate. If someone tests positive, what are the chances they’re actually a drug user?
Our system one brain wants to jump to conclusions and say, “98%!”. But there’s an important fact missing. What is the actual usage rate of the drug in the broader population?
If only 5% of people are users of the drug, then even with a positive test in hand, the reality is only about a 19.7% chance. Knowing that only one in twenty people actually use the drug, we have to discount the results of that drug test by our prior knowledge.
The below headline about positive COVID tests is a perfect example of a misleading statistic. It’s meaningless unless you know the actual vaccination distribution across Los Angeles.
“Over One Quarter of Positive COVID Cases in LA Are Fully Vaccinated People”
Thinking about statistics (or options pricing) at the extreme is a useful tool for quickly considering the dynamics at play. If one quarter of the cases are coming from fully vaccinated people, but 99% of the population was vaccinated, then the 1% of unvaccinated people would make up 75% of the cases - that sounds pretty effective to me.
The actual vaccination rate in LA is hovering a little below 60%. While not nearly as poignant as the extreme example, it does suggest that the significant majority of cases are coming from a minority of the population.
The probability of one event occurring given another event occurring, must be taken conditionally.
This framework for thinking is particularly useful in investing and performing scenario analysis. We might think about how changes in interest rates affect stock prices, or how commodity price inflation impacts profitability.
Bayes Theorem is even credited for helping to win World War II. The famous German Enigma code was uncrackable for years. Some of the country's best mathematicians and cryptographers puzzled at the infinite possibilities it produced.
Alan Turing had the brilliant insight to apply probabilistic models to the outcomes and use prior knowledge to inform their efforts. For example, messages from U-boats were likely to be about shipping and weather, and so they were able to vastly narrow the field of outcomes.
No matter what type of judgement you’re asked to make, it’s useful to think in probabilities. The base case has a powerful effect on the actual chances.