Category: Uncategorized

One of the most important assumptions for statistical models to work is the notion of consistency. This means that statisticians often drool with excitement when they find out that their data has approximately stable statistical properties, because they can finally unlock the cabinet of unused dusty models. In time series analysis (and several other disciplines), this consistency is coined stationarity. *** Stationarity is often defined in two ways: strictly and weakly. Strict stationarity requires that the joint distribution of a set of values be the same for all time points. Mathematically, given the set of values at time tt, strict…

We’ve briefly talked about how pp-values should be interpreted. It’s crucial to understand that a pp-value of 0.01 doesn’t mean that there is a 1% chance of some null hypothesis being true. Instead, it implies a 1% chance of observing data as extreme or more extreme than the current data under the condition that the null hypothesis is true. pp-values form the foundation of frequentist statistics, and rightfully so because it determines whether a certain event (alternative hypothesis) is probably true by thresholding it against a determined significance level and repeating the experiment many times. However, it’s precisely because the…

The normal distribution is one of statistics’ most precious models of reality. It’s analytically tractable, computationally simple, and provides a universal language for uncertainty. As such, it definitely deserves an in-depth exploration of its characteristics, properties, and significance. And then, we’ll explode in, Game of Thrones style, to ruin the perfect rainbow world of normality by introducing the real world we live in. The normal (Gaussian) distribution is the most famous continuous distribution with a bell-shaped p.d.f. φ\varphi, defined by only two parameters: mean μ\mu and standard deviation σ\sigma. Given a normal r.v. X∼𝒩(μ,σ2)X \sim \mathcal{N}(\mu, \sigma^2), Having two descriptive…

There was a time when I used to apply linear regression to some data and if the resulting metrics (R2R^2, RMSE, MAE, etc.) were unsatisfactory, I simply concluded that the regression wasn’t a good fit and I should probably instead look at other models like gradient boosting or neural networks. If you don’t think this was a problem, see it in this way: a baker judges whether a cake is baked properly just by looking at the clock, without ever inserting a toothpick to check inside the cake. The issue is that we tend to only use metrics to determine…

I always knew Principal Component Analysis (PCA) as a dimensionality reduction technique. Way too many features? PCA. Need to visualize clustering? PCA. Exploratory data analysis? PCA. It’s definitely one of my go-to analysis back then, but not because of its usefulness—it was one of the few tools I knew existed, so might as well, I thought. When I ventured deeper into statistics, this was one of the first few things I mastered because it was a technique I used so often. Like I mentioned, PCA is a powerful tool for dimensionality reduction, feature engineering, and data visualization. Essentially, it linearly…

During my early days of learning statistics, I encountered a pretty interesting phenomenon while reading a (relatively) ancient article. The story goes like this: In the fall of 1973, a study on gender bias among graduate school admissions to University of California, Berkeley made headlines. The reason for this was that the admission figures showed that out of almost 13,000 applicants, the men had a 44% chance of being admitted while the women only had a 35% chance. This stark difference sent a clear message to the masses—Berkeley was simply discriminating women and prioritized education for men. Or was it?…

Back in university, my genetics professor introduced the concept of chi-square tests like it was a magical instrument. Before I was ever interested in any statistics, I always had a script from the lecture notes that ran some code on R, and all I had to do was reject either the hypothesis that my two variables were related or unrelated. The me back then wouldn’t ever fathom a version of myself so deeply invested in learning the foundations of many statistical models. Like, did you know that there are two types of chi-square tests? They are the: Although the two…