Category: Uncategorized

This is a story of cointegration: of common misconceptions about the relationship between multiple time series and how cointegration brings a new perspective to this. Much of the concept of cointegration I’ve encountered comes with in-depth technical details and derivations that often makes it more challenging than it looks, so I thought I’d like to present this by weaving a more storytelling approach that, fingers crossed, still captures (with some rigor) the main ideas behind the topic. *** Let’s start by defining two completely independent random walks, generated as ARIMA(0,1,0) processes as where εt∼i.i.d.(0,σε2)\varepsilon_t \sim \text{i.i.d.}(0, \sigma_\varepsilon^2) and νt∼i.i.d.(0,σν2)\nu_t \sim…

Most statistical models begin with some assumption about the world. In linear regression, we assume the relationship between inputs and output is a weighted sum, while in logistic regression, we assume the log-odds of a class is linear in the features. However, while they do give us interpretable coefficients and elegant closed-form solutions, they’re still assumptions. And reality often doesn’t cooperate. Decision trees take a different philosophical stance. Rather than imposing a functional form on the data, they ask a simpler question at every step: what is the best way to split this data into two groups such that each…

My fiancée has this supernatural ability she calls gut feeling where she’s able to somewhat accurately able to sense a hidden truth. The other day, she told me out of the blue that she felt a little nauseous, and then out of the blue, that perhaps so-and-so we’re broken up. Then we’d stalk their socials and find her words true. Okay, now I’m not so insane that every single life experience becomes a lesson on statistics, but I thought this would make a smooth segue (fun fact: you probably thought this was spelled segway) into Hidden Markov Models (HMMs), one…

Previously, we’ve taken a look at the continuous Fourier transform (FT), which is a powerful tool for decomposing a signal into its constituent frequencies. However, as we’ve briefly mentioned in the conclusion of that article, in practice, we never actually observe a continuous signal. Therefore, the tool is useless and we end our discussion here. Just kidding. This is where the discrete Fourier transform (DFT) comes in. Essentially, the DFT is a discretized version of the continuous FT, which provides a much more realistic and applicable approach for applying the transformation on discrete data (although actually still not the one…

In my previous post, we delved quite deep into time series models like AR, MA, ARMA, and ARIMA. Essentially, by capturing different aspects of a series’ memory, these models usually effectively extract autocorrelation out of data into their structural parameters. I actually have to apologize—to simplify definitions, I intentionally omitted an important label: univariate. These aforementioned models are univariate time series models, meaning they excel at tasks where all relevant information is contained within a single series. Thing is, real-world scenarios are rarely so simple. Because univariate models only capture relationships within a single series, they completely ignore the effects…

As a brief break from my usual, more rigorous content, I thought it’d be fun to explore some games. Of course, to stay consistent with the themes of my blogs, these games will still be rooted in statistics. As we’ll soon see, the puzzles revolve around how uncertainty behaves and how small changes can dramatically change what good strategies look like. This is, in fact, exactly the kind of thinking that appears in probability theory: We’re given a system governed by randomness, and the challenge is to reason our way through it! I thought card decks make particularly good playgrounds…

One of my favorite topics in time series analysis is forecasting, which is the art of modeling memory. What’s memory? In a previous post, we explored this concept in depth through the lens of autocorrelation. On top of understanding how it’s measured, we established that it’s basically a non-negotiable heartbeat for any time series dataset. Now, if autocorrelation is the paint (that represents the raw material of memory), then time series models are the brush we use to bring a forecast to life. Essentially, different models allow us to stroke the canvas in different ways, and each accesses a unique perspective of said…

In the context of time series analysis, as statisticians, we’re more than comfortable with thinking about data in the time domain. Here, we have values which evolve across time, and the questions we’re interested in often follow this perspective: Is the series trending? Is it noisy? Is today’s data dependent on yesterday’s? This view is extremely useful but also has its limits. Real data is noisy. Sometimes, what looks like chaotic noise is actually the result of multiple cyclical patterns layered on top of each other. When these signals combine, the resulting time-series can look so tangled that the underlying…

The irony of the random variable (r.v.) is that although it takes on an “unpredictable” value every time, it’s not exactly random if we understand the shape of its distribution. This is why descriptive statistics matters a lot—they define the boundaries of the set of values an r.v. can take, otherwise known as, again, the shape of its distribution. There isn’t just one descriptive statistics, so it sure would be nice if we had a way to systematically identify these instead of applying different formulas for each one. Well. Actually, there is. It’s called moments. *** Before we even delve…

One assumption we discussed for linear regression is the independence of error terms. In that setting, we were typically dealing with cross-sectional data, where we assumed that observations don’t influence each other. Time series data is a little special. Over time, observations are rarely ever independent. If we observe that today’s stock price is high, there’s a pretty good chance that yesterday’s price is high. In other words, values in a time series tend to be correlated with their past. We call this dependence across time autocorrelation. As if to smoothen the transition between the two, many time series models…