Category: Uncategorized

One natural question we should ask is whether stocks can be grouped by how they behave, rather than just by the sector labels someone assigned them decades ago. For example, here’s one of many things sector labels don’t capture: a tech company and a utility might sit in completely different industries, yet move in near-perfect lockstep during a market downturn. That’s why in this article, we’ll use KK-mean clustering to classify a universe of stocks based on their return and volatility characteristics. Of course, there are likely more robust ways of choosing factors for the task, e.g., including other features such…

Predicting asset returns is a difficult game where the usual rules of regression are stress-tested by noisy signals, correlated features, and the ever-present risk of overfitting to market microstructure. While ordinary least squares (OLS) gives us a clean starting point, penalized regression methods like Ridge, LASSO, and Elastic Net offer a principled way to impose structure on that noise. Note that due to the violation of assumptions, the performances of the models will suck a lot. The goal is therefore to understand how regularization changes what a model learns, what it discards, and how we should think about bias-variance tradeoff…

In the previous article, we introduced Hidden Markov Models (HMMs) as a way to capture volatility regime-switching in SPY returns. By decomposing returns into distinct states, i.e., low, medium, and high volatility, we’re able to uncover meaningful structure that a single continuous model like GARCH could not explicitly represent. However, HMMs assume that the market operates in a finite set of discrete regimes, forcing what is inherently a continuous process into a small number of fixed states. In reality, volatility and other latent dynamics tend to evolve smoothly over time, exhibiting gradual transitions rather than abrupt switches. To explore this…

Previously, we saw that modeling variance, rather than the mean, provides a much more effective way of capturing financial time series dynamics. GARCH models, in particular, are able to reproduce volatility clustering and persistence, making them a strong baseline for volatility modeling. However, GARCH comes with an important structural assumption, which is that volatility evolves smoothly as a deterministic function of past shocks and past variance. In reality, financial markets often appear to operate in distinct regimes, with periods of sustained low volatility punctuated by sudden transitions into high-volatility environments, such as during crises. So we simply ask: what if…

In the previous article, we fit AR, MA, ARMA, and ARIMA models to SPY log returns and watched them systematically fail in a very specific way. The residuals showed volatility clustering, the QQ plots showed fat tails, and ACF plot on squared residuals confirmed that the variance itself was autocorrelated. ARIMA models the conditional mean. The problem is in the conditional variance. That’s what ARCH and GARCH are built for, and that’s we’ll explore today. *** The data we’ll be using here is exactly the same as what we’ve used in the previous article — SPY log returns from the…

This is (hopefully) a beginner-friendly tutorial in attempting to model SPY data using linear time series models. Specifically, we take a look at basic properties of the data, the fitness of AR, MA, ARMA, and ARIMA models, and, spoiler alert, why they suck at the job. *** A good refresher from my article on linear time series models reminds us that unlike static data, the error term ε\varepsilon in time series data exhibits autocorrelation, which violates the assumption of independent ε\varepsilon for static models. Therefore, we use time series models because they’re capable of explicitly removing autocorrelation from the error…

Every ARIMA model that we write carries an assumption we might not have explicitly stated. When we consider the error term εt\varepsilon_t, we assume that where σ2\sigma^2 is a constant. We call this homoskedasticity, and for many applications, it’s honestly reasonable enough. However, some data (e.g., financial time series) have a well-documented habit of violating it. Take a look at the plot for the S&P 500’s log returns above. It’s clear that the magnitude of daily swings isn’t constant across time. We observe that markets go through stretches of relative calm (e.g., mid-2023 to mid-2024) and then abruptly enter periods…

In our Black-Scholes post, we saw that the log-return over any interval assumed by the model is normally distributed with variance proportional to σ2T\sigma^2 T, where σ\sigma is a constant. This implies three things that empirical data consistently contradict: no volatility clustering (large moves tend to follow large moves in real markets), no mean-reversion (volatility spikes during crises and subsides afterward), and symmetric returns (market exhibits negative skew). These limitations give rise to the Heston model. Unlike the Black-Scholes, this model lets variance vt=σt2v_t = \sigma_t^2 evolve as its own stochastic process, correlated with the stock price process. The correlation…

he Black-Scholes model is inarguably one of the most important formulas to ever exist. Lots of people have seen it and memorized it, and some have applied it derivatives pricing to actually accumulate wealth. Personally, I’m deeply interested in how it came to be simply because fully understanding the system gives a level of intuition unlike anyone blatantly memorizing the formula can have. So, unlike my other posts, I’ve decided to comprehensively show the derivation (abstracting non-trivial calculations) of the Black-Scholes formula, and only later explain the intuition of the model that many know. *** How should we model a…

If there’s anything we’ve learnt after spending time with hidden Markov models (HMMs), it’s that HMMs are based on a powerful idea: the world has a hidden state that evolves over time, and all we ever get to observe is noisy, indirect measurements of that state. HMMs gave us a clean framework for reasoning about this under discrete state spaces. But, then, what happens when the hidden state is continuous? What about the exact position of a moving car, or the true temperature of a chemical reactor? Enumerating every possible state is no longer feasible, and so we need a…