<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Statistics on LCS.Developer</title><link>https://www.lcs-dev.org/tags/statistics/</link><description>Recent content in Statistics on LCS.Developer</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><copyright>LCS.Developer</copyright><lastBuildDate>Tue, 08 Apr 2025 20:00:00 +0000</lastBuildDate><atom:link href="https://www.lcs-dev.org/tags/statistics/index.xml" rel="self" type="application/rss+xml"/><item><title>The Central Limit Theorem: Formulation, Intuition, and Statistical Inference</title><link>https://www.lcs-dev.org/posts/the-central-limit-theorem---formulation-intuition-and-statistical-inference/</link><pubDate>Tue, 08 Apr 2025 20:00:00 +0000</pubDate><guid>https://www.lcs-dev.org/posts/the-central-limit-theorem---formulation-intuition-and-statistical-inference/</guid><description>&lt;h2 id="universal-convergence-to-the-gaussian"&gt;Universal Convergence to the Gaussian
&lt;/h2&gt;&lt;p&gt;Why does the Gaussian bell curve $\phi(z) = \frac{1}{\sqrt{2\pi}}e^{-z^2/2}$ govern phenomena as disparate as thermal noise in silicon, measurement errors in astronomy, biological heights, and poll margins of error?&lt;/p&gt;
&lt;p&gt;The mathematical answer is the &lt;strong&gt;Central Limit Theorem (CLT)&lt;/strong&gt;: whenever an observable quantity is the additive aggregate of many small, independent random fluctuations with finite variance, its standardized distribution converges toward a normal distribution—&lt;strong&gt;regardless of the shape of the individual fluctuations&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;First observed for binomial coin flips by Abraham de Moivre (1733), generalized by Pierre-Simon Laplace (1812), and proven in full generality by Aleksandr Lyapunov (1901), Jarl Lindeberg (1920), and Paul Lévy (1925), the CLT is the pillar connecting probability theory to statistical inference.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="formal-statement-lindeberglévy-clt"&gt;Formal Statement (Lindeberg–Lévy CLT)
&lt;/h2&gt;&lt;p&gt;Let $X_1, X_2, \dots, X_n$ be independent and identically distributed (i.i.d.) random variables with finite mean and positive, finite variance:&lt;/p&gt;
&lt;p&gt;$$\mu = \mathbb{E}[X_i], \qquad 0 &amp;lt; \sigma^2 = \text{Var}(X_i) &amp;lt; \infty.$$&lt;/p&gt;
&lt;p&gt;Define the partial sum $S_n = \sum_{i=1}^n X_i$ and the sample mean $\bar{X}_n = \frac{1}{n}S_n$. Because expectation is linear and variance is additive over independent variables:&lt;/p&gt;
&lt;p&gt;$$\mathbb{E}[S_n] = n\mu, \qquad \text{Var}(S_n) = n\sigma^2, \qquad \mathbb{E}[\bar{X}_n] = \mu, \qquad \text{Var}(\bar{X}_n) = \frac{\sigma^2}{n}.$$&lt;/p&gt;
&lt;p&gt;Standardizing $S_n$ (or equivalently $\bar{X}_n$) by subtracting its mean and dividing by its standard deviation defines the dimensionless pivot:&lt;/p&gt;
&lt;p&gt;$$Z_n = \frac{S_n - n\mu}{\sigma\sqrt{n}} = \frac{\bar{X}_n - \mu}{\sigma / \sqrt{n}}.$$&lt;/p&gt;
&lt;h3 id="theorem-central-limit-theorem"&gt;Theorem (Central Limit Theorem)
&lt;/h3&gt;&lt;p&gt;As $n \to \infty$, the standardized sum $Z_n$ converges in distribution to the standard normal random variable $Z \sim \mathcal{N}(0, 1)$:&lt;/p&gt;
&lt;p&gt;$$Z_n \xrightarrow{d} \mathcal{N}(0, 1), \qquad \text{i.e.,} \qquad \lim_{n \to \infty} \mathbb{P}(Z_n \le z) = \Phi(z) = \int_{-\infty}^{z} \frac{1}{\sqrt{2\pi}}e^{-u^2/2},du \quad \forall z \in \mathbb{R}.$$&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="why-sqrtn-is-the-exact-scaling-window"&gt;Why $\sqrt{n}$ Is the Exact Scaling Window
&lt;/h2&gt;&lt;p&gt;It is instructive to compare what happens to $S_n - n\mu$ under different normalizations as $n \to \infty$:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Dividing by $n$ (The Law of Large Numbers Scale)&lt;/strong&gt;:
$$\frac{S_n - n\mu}{n} = \bar{X}_n - \mu \xrightarrow{p} 0.$$
At scale $1/n$, the variance is $\sigma^2 / n \to 0$: all randomness collapses to a single deterministic spike at $0$.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Dividing by $1$ (Unnormalized Sum)&lt;/strong&gt;:
The variance $n\sigma^2 \to \infty$: the distribution spreads out flat over the entire real line.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Dividing by $\sqrt{n}$ (The CLT Fluctuation Scale)&lt;/strong&gt;:
$$\text{Var}!\left(\frac{S_n - n\mu}{\sqrt{n}}\right) = \sigma^2.$$
Dividing by $\sqrt{n}$ acts as a microscope zoomed to the exact scale where the variance remains constant ($\sigma^2$) for all $n$, revealing the universal Gaussian shape of the random fluctuations around the mean.&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;h2 id="convolution-smoothing-in-action"&gt;Convolution Smoothing in Action
&lt;/h2&gt;&lt;h3 id="1-sums-of-discrete-uniform-variables-dice"&gt;1. Sums of Discrete Uniform Variables (Dice)
&lt;/h3&gt;&lt;p&gt;Let $X_i \in {1, 2, 3, 4, 5, 6}$ be the outcome of a single fair die ($\mu = 3.5$, $\sigma^2 = 35/12 \approx 2.917$). The probability mass function (PMF) of $X_1$ is completely flat.&lt;/p&gt;
&lt;p&gt;Adding a second independent die $S_2 = X_1 + X_2$ computes the &lt;strong&gt;discrete convolution&lt;/strong&gt; of two flat rectangular distributions, which is a triangle peaking at $7$. Adding a third die $S_3$ convolves the triangle with a rectangle, producing a piecewise-quadratic bell curve (&lt;strong&gt;B-spline&lt;/strong&gt; smoothing). By $n = 10$ dice, the standardized PMF is visually indistinguishable from the continuous Gaussian density $\mathcal{N}(0, 1)$.&lt;/p&gt;
&lt;h3 id="2-de-moivrelaplace-binomial-approximation"&gt;2. De Moivre–Laplace Binomial Approximation
&lt;/h3&gt;&lt;p&gt;Let $X_1, \dots, X_n \sim \text{Bernoulli}(p)$ be independent coin flips with success probability $p \in (0, 1)$. Their sum $S_n \sim \text{Binomial}(n, p)$ has mean $np$ and variance $np(1-p)$. By the CLT:&lt;/p&gt;
&lt;p&gt;$$\frac{S_n - np}{\sqrt{np(1-p)}} \xrightarrow{d} \mathcal{N}(0, 1), \qquad \text{so} \qquad S_n \approx \mathcal{N}!\left(np,, np(1-p)\right).$$&lt;/p&gt;
&lt;p&gt;&lt;em&gt;(When approximating a discrete integer sum $S_n$ with a continuous normal integral for moderate $n$, applying the $\pm 0.5$ &lt;strong&gt;continuity correction&lt;/strong&gt; $\mathbb{P}(a \le S_n \le b) \approx \Phi!\left(\frac{b + 0.5 - np}{\sqrt{np(1-p)}}\right) - \Phi!\left(\frac{a - 0.5 - np}{\sqrt{np(1-p)}}\right)$ substantially improves accuracy).&lt;/em&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="statistical-inference-and-confidence-intervals"&gt;Statistical Inference and Confidence Intervals
&lt;/h2&gt;&lt;p&gt;In empirical data analysis and Monte Carlo simulation, we observe a sample $X_1, \dots, X_n$ from an unknown distribution $F$ and want to estimate its true mean $\mu = \mathbb{E}[X]$ with a rigorous error bar.&lt;/p&gt;
&lt;p&gt;Even though we do not know $F$, the CLT guarantees that for large $n$, the pivot $\frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}$ is approximately $\mathcal{N}(0, 1)$. Furthermore, if the true standard deviation $\sigma$ is unknown, replacing $\sigma$ with the consistent sample standard deviation&lt;/p&gt;
&lt;p&gt;$$s_n = \sqrt{\frac{1}{n-1}\sum_{i=1}^n (X_i - \bar{X}_n)^2}$$&lt;/p&gt;
&lt;p&gt;preserves asymptotic normality by &lt;strong&gt;Slutsky&amp;rsquo;s Theorem&lt;/strong&gt; (since $s_n \xrightarrow{p} \sigma$):&lt;/p&gt;
&lt;p&gt;$$\frac{\bar{X}_n - \mu}{s_n / \sqrt{n}} \xrightarrow{d} \mathcal{N}(0, 1).$$&lt;/p&gt;
&lt;h3 id="asymptotic-1---alpha-confidence-interval"&gt;Asymptotic $(1 - \alpha)$ Confidence Interval
&lt;/h3&gt;&lt;p&gt;Let $z_{\alpha/2} = \Phi^{-1}(1 - \alpha/2)$ denote the upper $\alpha/2$ quantile of $\mathcal{N}(0, 1)$ (for a $95%$ confidence level, $\alpha = 0.05$ and $z_{0.025} \approx 1.96$). Then:&lt;/p&gt;
&lt;p&gt;$$\lim_{n \to \infty} \mathbb{P}!\left(\mu \in \left[\bar{X}&lt;em&gt;n - z&lt;/em&gt;{\alpha/2}\frac{s_n}{\sqrt{n}}, ;; \bar{X}&lt;em&gt;n + z&lt;/em&gt;{\alpha/2}\frac{s_n}{\sqrt{n}}\right]\right) = 1 - \alpha.$$&lt;/p&gt;
&lt;p&gt;Notice the &lt;strong&gt;$\sqrt{n}$ law of precision&lt;/strong&gt;: the width of the confidence interval is proportional to $1/\sqrt{n}$. To reduce estimation uncertainty by a factor of $10$, a Monte Carlo simulation or clinical trial must collect &lt;strong&gt;$100\times$ as many samples&lt;/strong&gt;.&lt;/p&gt;</description></item></channel></rss>