While I write thousands of words every day in the form of blogs, newsletters, and e-mails, it’s been quite a few years since I wrote my longest piece ever.
Even running the Tufts Daily News Department alongside a full load of Poli Sci and Economics classes, I never wrote anything longer than 2500 words through college. Trading didn’t exactly encourage the craft after graduation.
Yet the capstone project for the high school International Baccalaureate was the minimum 4,000 word “extended essay”. Perhaps the only way to convince a teenager to dedicate their time to this is to let them choose the topic.
I chose the bomb. More specifically, how different media sources covered the first explosion of the hydrogen bomb. Only a few years after Nagasaki and Hiroshima were leveled, the US split the atom even finer by annihilating an atoll in the Marshall Islands with the first thermonuclear explosion.
The detonation fused atoms and fractured opinions. Long before we blamed Fox News and MSNBC for stretching the Overton window to its breaking point, conservative and liberal news sources took wildly different views of a literal seismic event.
Digging up clippings from 1952 was not particularly easy. I managed to scrape together some microfilm of magazines like “The New Republic” and “The Nation”. The Chicago Tribune was my relatively conservative newspaper, and the good old Grey Lady was the liberal daily. Despite her seeming ubiquity, she was the most difficult one to find.
I went all the way to the New York Public Library to get that week’s papers, and it was missing from their archives. They had sequential numbered files for the week of October 26th and November 9th. I’d like to believe it was some psyops that “lost” the week of November 2. I ended up buying a PDF from the Purdue University library to get the article I needed.
I parsed these primary sources down by which adjectives they used and where they placed the coverage. Was this a glowing celebration of American might or a doomsday escalation of Dr. Strangelove’s worst nightmare? We all watched the same movie but came away with wildly different interpretations. (You too can watch Operation Mike right here.)
The words our scribes choose lay very different sheens on the same event. While it can be described as biased when every witness in Rashomon precisely curates distinct elements of the murder, there are important reasons for understanding history through different lenses. For the amateur historian, my job was to paint a picture from all these different sources that captured the true zeitgeist in a way that elucidated forward looking meaning.
Different attributes of history matter for data magicians too. Controlled scientific tests seem wildly subjective compared to the cold steel objectivity of market data. Trade prices represent facts, deals, and are perfectly quantitative representations of transpired events. Yet they can tell any story you want.
Just as we learn from history to repeat its mistakes in rhyming ways, we look at market data in the hopes that the past will reveal some future insight about up, down, or sideways. The weakest form of efficient market hypothesis suggests that this is all incorporated into current price, yet technicians abound with Ichimoku cloud formations and Fibonacci retracements.
While I only believe the most basic technical analysis (mostly the importance of round numbers to investor psychology), just as much fiction can be written in Excel and Python as historical volatilities are calculated.
If the most important part of the options pricing model is the implied volatility level, it’s only natural that the most important piece of data to calculate is the amount of volatility that stock has seen in the past. Knowing what something has done is a pretty good start for a prediction of what it will do in the future.
There are a thousand devils of detail in the words “what something has done in the past”. A simple first question might be how long is the past. Ten days? Thirty days? Ten years? We know of course that more recent history is likely to be more informative than events long past, but what’s the balance between focusing on the latest observations and capturing enough different possible scenarios?
Now that we have a time frame, we can grab the prices - but which ones and at what frequency? Is the daily close price enough granularity, or should we get intraday data? There might be good reasons to smooth out noise with weekly returns. Parkinson’s historical volatility model emphasizes the High and Low price on the day - not just where it ended up. “Close to Close” models do exactly what they say - which incorporates overnight gaps. Comparing these highlights when the movement is happening.
Close isn’t even a simple definition. I once spent several days debugging historical earnings move calculations because of what turned out to be a snapshot timing question. If you get prices at 4:02pm instead of 4:00pm, sometimes all a stock needs to move a lot is the first few seconds of a transcript.
In calculating the differences period to period, unfortunately basic arithmetic won’t do. We need to consider the changes in a lognormal framework, which accounts for the non negativity and proportionality of financial asset price returns, as well as having the convenient property of being aggregable over different time frames.
Now that we’ve traversed all the basic questions of how to decide which data points to use and how to calculate the returns, we can take this data curation even further. Rather than choose between different time frames, what if we could incorporate weights into our data? Science, meet art.
If the more recent is more informative, what is the degree? Determining the appropriate factor for calculating an exponentially weighted moving average is far more difficult than the arithmetic to calculate it. Short term weights provide faster signals but more false positives. Waiting for a long term trend to manifest risks missing out all together.
One of the more common models in options historical volatility calculations is the “GARCH” framework. The acronym stands for Generalized Autoregressive Conditional Heteroskedasticity, and is a mouthful that broadly means the math takes into account the memory of volatility. The two main parameters regulate how much of an impact large moves in the past have on the current estimate, as well as how much of an impact the current environment has.
We know that higher volatility begets higher volatility. Volatility also clusters. Weeks of calm followed by a few days of storms. That’s the heteroskedasticity part. Autoregressive and conditional mean; if price were a human, how much do your surroundings and past trauma impact what your next move will be?
As SQL therapists massage meaning out of a fuzz of numbers, they are confronted with endless parameterization decisions. If I overfit the polynomial my R-squared makes Robert Parker jealous, but I’m left with an absolutely useless divining rod. A stock’s past bouts with volatility and the current VIX regime will set a bountiful table, but it’s always anyone’s guess which personality shows up to dinner.
These inputs and endless decisions about which data to use and how are born from the desire for a nuanced understanding. We have to look through different facets of the prism to understand some useful truth about the behavior of a stock’s price. Neither GARCH, EGARCH or no GARCH is the uniquely right volatility model. Parkinsons and Close to Close have their shortcomings too.
A blend of these gets to be more informative, as you incorporate different elements of behavior in your model. Then we get to the meta question of how to blend your blends. Here’s where the complexity stops for me- Occam’s razor that into equal parts and accept that truth is an estimate.