Back to Math Tools
Back to notes

# Gaussian Integral and Tricks

The main theme of this note is about the Gaussian distribution defined as follows.

$$\phi(x) = \frac{1}{\sqrt{2\pi}}e^{-x^2/2}\, .$$

It’s not hard to see that it takes non-negative value on the real line, and indeed it forms a distribution as it integrates to 1 (see below if you want to know how to prove this).

Note that it is equivalent to show that $\int e^{-x^2}dx=\sqrt{\pi}$ and indeed there are multiple ways to prove this. Here we will use a simple proof via switching to polar coordinates. Observe that \begin{align} \left(\int e^{-x^2}dx\right)^2 &= \left(\int e^{-x^2}dx\right)\cdot\left(\int e^{-y^2}dy\right)\\
&= \int e^{-(x^2+y^2)}dxdy \, . \end{align} Note that this becomes an integral over the $\Real^2$ plane and we can switch to the polar coordinates $(r,\theta)$ due to the rotational symmetry. Concretely, we have $x=r\cos\theta$ and $y=\sin\theta$ and by the substitution rules of integrations, we have $dxdy=rdrd\theta$ and hence \begin{align} \int e^{-(x^2+y^2)}dxdy = \int e^{-r^2}rdrd\theta = \frac{2\pi}{2} = \pi \end{align} as desired.

## Basic properties

### Derivatives

$$\frac{d\phi(x)}{dx} = -x\phi(x) \, .$$

## Multivariate Gaussians

Let $n$ be an integer, $\mu$ be an $n$-dimensional vector, and $\Sigma$ be an $n\times n$ positive definite matrix. The corresponding multivariate Gaussian distribution is defined as follows. $$f(\bx) = \frac{1}{\sqrt{(2\pi)^n\det(\Sigma)}} e^{-(\bx-\mu)^\top\Sigma^{-1}(\bx-\mu)/2} \, .$$

## Useful approximations

### Gaussian tail bounds

When $a>0$, we have the following Gaussian tail bound. $$\left(\frac{a^2}{1+a^2}\right)\cdot\frac{\phi(a)}{a} \leq \int_{a}^{\infty}\phi(x)dx \leq \frac{\phi(a)}{a}$$

Let’s start with the upper bound: observe that \begin{align} \int_{a}^{\infty}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}dx &\leq \int_{a}^{\infty}\frac{x}{a}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}dx\\
&= \frac{1}{a\sqrt{2\pi}}\int_{a}^{\infty}e^{-x^2/2}dx^2/2\\
&= \frac{1}{a\sqrt{2\pi}}\int_{a^2/2}^{\infty} e^{-u}du\\
&= \frac{1}{a\sqrt{2\pi}} \left(-e^{-u}\right)|_{a^2/2}^{\infty}\\
&= \frac{e^{-a^2/2}}{a\sqrt{2\pi}} = \frac{\phi(a)}{a} \end{align} as desired.

Next, let’s move on to the lower bound: first we utilize the basic fact that $\frac{d\phi(x)}{dx} = -x\phi(x)$ and have \begin{align} \int_{a}^{\infty}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}dx &= \int_{a}^{\infty}-\frac{d\phi(x)}{x}dx\\
&= \left(-\frac{\phi(x)}{x}\right)|_{a}^{\infty} + \int_{a}^{\infty}-\frac{\phi(x)}{x^2}dx \\
&= \frac{\phi(a)}{a} + \int_{a}^{\infty}-\frac{\phi(x)}{x^2}dx \, . \end{align} By adjusting the two sides, we have \begin{align} \frac{\phi(a)}{a} &= \int_a^{\infty}\frac{1+x^2}{x^2}\phi(x)dx \leq \frac{1+a^2}{a^2}\int_a^{\infty}\phi(x)dx \end{align} where we get the desiring lower bound by dividing both side by $(1+a^2)/a^2$.

Note that when $a>0$ is very large, we have $$\int_{a}^{\infty}\phi(x)dx\approx \frac{\phi(a)}{a} \, .$$

## Some tricks

### Dirac delta function

Dirac delta function, $\delta(x)$, is an operator with the following two properties: (i) $\delta(x)=0$ for every $x\neq0$ and (ii) $\int \delta(x-a)f(x)dx=f(a)$ for any reasonable function $f$ and real number $a$. Of course, there exists rigorous mathematical treatment on Dirac delta function but that’s not the focus of this post.

In physics, one often replace the Dirac delta function in an equation with an integral representation so that further tricks/manipulations are possible. The following is the most common one. $$\delta(x-a) = \int\frac{1}{2\pi}e^{i(x-a)t}dt$$

For any reasonable function $f$, consider applying Fourier transform and inverse Fourier transform in a sequence: \begin{align} f(a) &= \frac{1}{\sqrt{2\pi}}\int \hat{f}(t)e^{ita}dt \\
&= \frac{1}{\sqrt{2\pi}}\int \left( \frac{1}{\sqrt{2\pi}}\int f(x)e^{-ixt}dx\right)e^{ita}dt\\
&= \frac{1}{2\pi} \int \left(\int e^{it(a-x)} dt\right)f(x) dx \end{align} where the inner parenthesis of the last equation behaves like the Dirac delta function. Indeed this can be shown to be the case rigorously. And here we treat $i$ and $-i$ interchangeably.

### Hubbard–Stratonovich transformation

$$\int e^{ixa}\phi(x)dx = e^{-a^2/2}$$