Note to Self: Norm of a Gaussian Random Vector

Let $g$ be a standard Gaussian vector—that is, a vector populated by independent standard normal random variables. What is the expected length $\mathbb{E} \|g\|$ of $g$ ? (Here, and throughout, $\|\cdot\|$ denotes the Euclidean norm of a vector.) The length of $g$ is the square root of the sum of $n$ independent standard normal random variables

$\|g\| = \sqrt{g_1^2 + \cdots + g_n^2},$

which is known as a $\chi$ random variable with $n$ degrees of freedom. (Not to be confused with a $\chi^\mathbf{2}$ random variable!) Its mean value is given by the rather unpleasant formula

$\mathbb{E} \|g\| = \sqrt{2} \frac{\Gamma((n+1)/2)}{\Gamma(n/2)},$

where $\Gamma(\cdot)$ is the gamma function. If you are familiar with the definition of the gamma function, the derivation of this formula is not too hard—it follows from a change of variables to $n$ -dimensional spherical coordinates.

This formula can be difficult to interpret and use. Fortunately, we have the rather nice bounds

(1) $\sqrt{n-1} < \frac{n}{\sqrt{n+1}} < \mathbb{E} \|g\| < \sqrt{n}.$

This result appears, for example, page 11 of this paper. These bounds show that, for large $n$ , $\mathbb{E} \|g\|$ is quite close to $\sqrt{n}$ . The authors of the paper remark that this inequality can be probed by induction. I had difficulty reproducing the inductive argument for myself. Fortunately, I found a different proof which I thought was very nice, so I thought I would share it here.

Our core tool will be Wendel’s inequality (see (7) in Wendel’s original paper): For $x > 0$ and $0 < s < 1$ , we have

(2) $\frac{x}{(x+s)^{1-s}} < \frac{\Gamma(x+s)}{\Gamma(x)} < x^s.$

Let us first use Wendel’s inequality to prove (1). Indeed, invoke Wendel’s inequality with $x = n/2$ and $s = 1/2$ and multiply by $\sqrt{2}$ to obtain

$\frac{\sqrt{2} \cdot n/2}{(n/2+1/2)^{1/2}} < \sqrt{2}\frac{\Gamma((n+1)/2)}{\Gamma(n/2)} = \mathbb{E}\|g\| < \sqrt{2}\cdot \sqrt{n/2},$

which simplifies directly to (1).

Now, let’s prove Wendel’s inequality (2). The key property for us will be the strict log-convexity of the gamma function: For real numbers $x,y > 0$ and $0 < s < 1$ ,

(3) $\Gamma((1-s)x + sy) < \Gamma(x)^{1-s} \Gamma(y)^s.$

We take this property as established and use it to prove Wendel’s inequality. First, use the log-convexity property (3) with $y = x+1$ to obtain

$\Gamma(x+s) = \Gamma((1-s)x + s(x+1)) < \Gamma(x)^{1-s} \Gamma(x+1)^s.$

Divide by $\Gamma(x)$ and use the property that $\Gamma(x+1)/\Gamma(x) = x$ to conclude

(4) $\frac{\Gamma(x+s)}{\Gamma(x)} < \left( \frac{\Gamma(x+1)}{\Gamma(x)} \right)^s = x^s.$

This proves the upper bound in Wendel’s inequality (2). To prove the lower bound, invoke the upper bound (4) with $x+s$ in place of $x$ and $1-s$ in place of $s$ to obtain

$\frac{\Gamma(x+1)}{\Gamma(x+s)} < (x+s)^{1-s}.$

Multiplying by $\Gamma(x+s)$ , dividing by $(x+s)^{1-s}\Gamma(x)$ , and using $\Gamma(x+1)/\Gamma(x) = x$ again yields

$\frac{\Gamma(x+s)}{\Gamma(x)} > \frac{\Gamma(x+1)}{\Gamma(x)} \cdot \frac{1}{(x+s)^{1-s}} = \frac{x}{(x+s)^{1-s}},$

finishing the proof of Wendel’s inequality.

Notes. The upper bound in (1) can be proven directly by Lyapunov’s inequality: $\mathbb{E} \|g\| \le (\mathbb{E} \|g\|^2)^{1/2} = n^{1/2}$ , where we use the fact that $\|g\|^2 = g_1^2 + \cdots + g_n^2$ is the sum of $n$ random variables with mean one. The weaker lower bound $\mathbb{E} \|g\| \ge \sqrt{n-1}$ follows from a weaker version of Wendel’s inequality, Gautschi’s inequality.

After the initial publication of this post, Sam Buchanan mentioned another proof of the lower bound $\mathbb{E} \|g\| \ge \sqrt{n-1}$ using the Gaussian Poincaré inequality. This inequality states that, for a function $f : \real^n \to \real$ ,