Normal Distribution

Normal Distribution

Normal distribution is one of the most fundamental distribution in Statistics. It is also known as Gaussian distribution.

Definition of Normal Distribution

A continuous random variable $X$ is said to have a normal distribution with parameters $\mu$ and $\sigma^2$ if its probability density function is given by

$$ \begin{equation*} f(x;\mu, \sigma^2) = \left\{ \begin{array}{ll} \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2\sigma^2}(x-\mu)^2}, & \hbox{$-\infty < x < \infty$,} \\ & \hbox{$-\infty < \mu < \infty$, $\sigma^2 > 0$;} \\ 0, & \hbox{Otherwise.} \end{array} \right. \end{equation*} $$

where $e= 2.71828...$ and $\pi = 3.1425926...$.

The parameter $\mu$ is called the location parameter (as it changes the location of density curve) and $\sigma^2$ is called the scale parameter of normal distribution (as it changes the scale of density curve).

In notation it can be written as $X\sim N(\mu,\sigma^2)$.

Graph of normal distribution

Following is the graph of probability density function of normal distribution. Here the means are different ($\mu = 0, 1,2$) while standard deviations are same ($\sigma=1$).

Normal distribution with different mean
Normal distribution with different mean

Graph of normal distribution

Following is the graph of probability density function of normal distribution. Here the means are same ($\mu = 0$) while standard deviations are different ($\sigma=1, 2, 3$).

Normal distribution with different variance
Normal distribution with different variance

Mean Normal distribution

The mean of $N(\mu,\sigma^2)$ distribution is $E(X)=\mu$.

Proof

The mean of normal $N(\mu,\sigma^2)$ distribution
$$ \begin{eqnarray*} \text{Mean } &=& E(X) \\ &=& \int_{-\infty}^\infty x f(x)\; dx\\ &=& \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty xe^{-\frac{1}{2\sigma^2}(x-\mu)^2} \; dx. \end{eqnarray*} $$

Let $\dfrac{x-\mu}{\sigma}=z$ $\Rightarrow x=\mu+\sigma z$.

Therefore, $dx = \sigma dz$. And if $x = \pm \infty$, $z=\pm \infty$. Hence,

$$ \begin{eqnarray*} \text{Mean } &=& \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty (\mu+\sigma z)e^{-\frac{1}{2}z^2}\sigma \;dz\\ &=& \frac{\mu }{\sqrt{2\pi}}\int_{-\infty}^\infty e^{-\frac{1}{2}z^2}\; dz+\frac{\sigma}{\sqrt{2\pi}}\int_{-\infty}^\infty ze^{-\frac{1}{2}z^2} \;dz\\ & & \qquad (\text{ Second integrand is an odd function of $Z$})\\ &=& \frac{\mu }{\sqrt{2\pi}}\int_{-\infty}^\infty e^{-\frac{1}{2}z^2}\; dz+\frac{1}{\sqrt{2\pi}}\times 0\\ &=& \frac{\mu }{\sqrt{2\pi}}\sqrt{2\pi}\quad (\because \int_{-\infty}^\infty e^{-\frac{1}{2}z^2}\; dz = \sqrt{2\pi})\\ &=& \mu. \end{eqnarray*} $$

Thus the mean of normal distribution $N(\mu,\sigma^2)$ is $\mu$.

Variance of Normal distribution

Variance of $N(\mu,\sigma^2)$ distribution is $V(X)=\sigma^2$.

Proof

The variance of $N(\mu,\sigma^2)$ distribution is

$$ \begin{eqnarray*} \text{Variance } &=& E(X-\mu)^2 \\ &=& \int_{-\infty}^\infty (x-\mu)^2 f(x)\; dx\\ &=& \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty (x-\mu)^2e^{-\frac{1}{2\sigma^2}(x-\mu)^2} \; dx. \end{eqnarray*} $$

Let $\dfrac{x-\mu}{\sigma}=z$ $\Rightarrow x=\mu+\sigma z$.

Therefore, $dx = \sigma dz$. And if $x = \pm \infty$, $z=\pm \infty$. Hence,

$$ \begin{eqnarray*} \text{Variance } &=& \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty \sigma^2 z^2e^{-\frac{1}{2}z^2}\sigma \;dz\\ &=& \frac{2\sigma^2}{\sqrt{2\pi}}\int_{0}^\infty z^2e^{-\frac{1}{2}z^2}\; dz\\ & & \qquad (\text{Integrand is an even function of $Z$}) \end{eqnarray*} $$

Let $z^2=y$, $z=\sqrt{y}$ $\Rightarrow dz = \frac{dy}{2\sqrt{y}}$ and $z=0 \Rightarrow y=0$, $z=\infty\Rightarrow y=\infty$. Hence,

$$ \begin{eqnarray*} \text{Variance } &=& \frac{2\sigma^2}{\sqrt{2\pi}}\int_{0}^\infty ye^{-\frac{1}{2}y}\; \frac{dy}{2\sqrt{y}}\\ &=& \frac{\sigma^2}{\sqrt{2\pi}}\int_{0}^\infty y^{\frac{1}{2}}e^{-\frac{1}{2}y}\; dy\\ &=& \frac{\sigma^2}{\sqrt{2\pi}}\int_{0}^\infty y^{\frac{3}{2}-1}e^{-\frac{1}{2}y}\; dy\\ &=&\frac{\sigma^2}{\sqrt{2\pi}}\frac{\Gamma(\frac{3}{2})}{(\frac{1}{2})^{\frac{3}{2}}}\\ &=&\frac{\sigma^2}{\sqrt{2\pi}}\frac{\frac{1}{2}\Gamma(\frac{1}{2})}{(\frac{1}{2})^{\frac{3}{2}}}\\ &=&\frac{\sigma^2\sqrt{\pi}}{\sqrt{\pi}}=\sigma^2. \end{eqnarray*} $$

Hence, variance of normal distribution is $\sigma^2$.

Central Moments of Normal Distribution

All the odd order central moment of $N(\mu,\sigma^2)$ distribution are $0$.

Proof

The $(2r+1)^{th}$ order central moment is

$$ \begin{eqnarray*} \mu_{2r+1} &=& E(X-\mu)^{2r+1} \\ &=& \int_{-\infty}^\infty (x-\mu)^{2r+1} f(x)\; dx\\ &=& \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty (x-\mu)^{2r+1}e^{-\frac{1}{2\sigma^2}(x-\mu)^2} \; dx. \end{eqnarray*} $$

Let $\dfrac{x-\mu}{\sigma}=z$ $\Rightarrow x=\mu+\sigma z$.

Therefore, $dx = \sigma dz$. And if $x = \pm \infty$, $z=\pm \infty$. Hence,

$$ \begin{eqnarray*} \mu_{2r+1} &=& \frac{\sigma^{2r+1}}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty z^{2r+1}e^{-\frac{1}{2}z^2}\sigma \;dz\\ &=& 0 \qquad (\because \text{Integrand is an odd function of $Z$}). \end{eqnarray*} $$

Hence, all the odd order central moments of normal distribution are zero, i.e., $\mu_1=\mu_3=\mu_5= \cdots = 0$.

Even order central moments of $N(\mu,\sigma^2)$ distribution

The $(2r)^{th}$ order central moment of $N(\mu,\sigma^2)$ distribution is

$$ \begin{equation*} \mu_{2r}= \frac{\sigma^{2r}(2r)!}{2^r (r)!}. \end{equation*} $$

Proof

The $(2r)^{th}$ order central moment of $N(\mu,\sigma^2)$ distribution is

$$ \begin{eqnarray*} \mu_{2r} &=& E(X-\mu)^{2r} \\ &=& \int_{-\infty}^\infty (x-\mu)^{2r} f(x)\; dx\\ &=& \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty (x-\mu)^{2r}e^{-\frac{1}{2\sigma^2}(x-\mu)^2} \; dx. \end{eqnarray*} $$
Let $\dfrac{x-\mu}{\sigma}=z$ $\Rightarrow x=\mu+\sigma z$.

Therefore, $dx = \sigma dz$. And if $x = \pm \infty$, $z=\pm \infty$. Hence,

$$ \begin{eqnarray*} \mu_{2r} &=& \frac{\sigma^{2r}}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty z^{2r}e^{-\frac{1}{2}z^2}\sigma \;dz\\ &=& 2 \frac{\sigma^{2r}}{\sigma\sqrt{2\pi}}\int_{0}^\infty z^{2r}e^{-\frac{1}{2}z^2}\sigma \;dz\\ & & \qquad (\because \text{Integrand is an even function of $Z$}). \end{eqnarray*} $$

Let $z^2=y$, $z=\sqrt{y}$ $\Rightarrow dz = \frac{dy}{2\sqrt{y}}$ and $z=0 \Rightarrow y=0$, $z=\infty\Rightarrow y=\infty$. Hence,

$$ \begin{eqnarray*} \mu_{2r} &=& \frac{2\sigma^{2r}}{\sqrt{2\pi}}\int_{0}^\infty y^re^{-\frac{1}{2}y}\; \frac{dy}{2\sqrt{y}}\\ &=& \frac{\sigma^{2r}}{\sqrt{2\pi}}\int_{0}^\infty y^{r-\frac{1}{2}}e^{-\frac{1}{2}y}\; dy\\ &=& \frac{\sigma^{2r}}{\sqrt{2\pi}}\int_{0}^\infty y^{r+\frac{1}{2}-1}e^{-\frac{1}{2}y}\; dy\\ &=&\frac{\sigma^{2r}}{\sqrt{2\pi}}\frac{\Gamma(r+\frac{1}{2})}{(\frac{1}{2})^{r+\frac{1}{2}}}\\ &=&\frac{\sigma^{2r}}{\sqrt{2\pi}}\frac{(r-\frac{1}{2})(r-\frac{3}{2})\cdots \frac{3}{2}\frac{1}{2}\Gamma(\frac{1}{2})}{(\frac{1}{2})^{r+\frac{1}{2}}}\\ &=&\frac{\sigma^{2r}}{\sqrt{\pi}}\frac{(r-\frac{1}{2})(r-\frac{3}{2})\cdots \frac{3}{2}\frac{1}{2}\sqrt{\pi}}{(\frac{1}{2})^{r}}\\ &=&\sigma^{2r}(2r-1)(2r-3)\cdots 3\times 1\\ &=& \sigma^{2r}\frac{2r(2r-1)(2r-2)(2r-3)\cdots 3\times 2\times 1}{2r(2r-2)(2r-4)\cdots 4\times 2} \\ \mu_{2r} &=& \frac{\sigma^{2r}(2r)!}{2^r (r)!}. \end{eqnarray*} $$

Hence,

$$ \begin{equation*} \mu_2 = \sigma^2,\; \mu_4 = 3\sigma^4. \end{equation*} $$

Coefficient of Skewness

The coefficient of skewness of $N(\mu,\sigma^2)$ distribution is $0$.

Proof

The coefficient of skewness of $N(\mu,\sigma^2)$ distribution is
$$ \begin{aligned} \beta_1 &= \frac{\mu_3^2}{\mu_2^3}\\ &= \frac{0}{\sigma^6} \\ &=0. \end{aligned} $$

Hence normal distribution is symmetric.

Kurtosis of Normal distribution

The coefficient of kurtosis of $N(\mu,\sigma^2)$ distribution is $3$.

Proof

The coefficient of kurtosis of $N(\mu,\sigma^2)$ distribution is

$$ \begin{aligned} \beta_2 &= \frac{\mu_4}{\mu_2^2}\\ &= \frac{3\sigma^4}{\sigma^4}\\ &=3. \end{aligned} $$

Hence, normal distribution is mesokurtic.

M.G.F. of Normal Distribution

The m.g.f. of $N(\mu,\sigma^2)$ distribution is

$$ \begin{equation*} M_X(t)=e^{t\mu +\frac{1}{2}t^2\sigma^2}. \end{equation*} $$

Proof

The moment generating function of normal distribution with parameter $\mu$ and $\sigma^2$ is

$$ \begin{eqnarray*} M_X(t) &=& E(e^{tX}) \\ &=&\frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty e^{tx}e^{-\frac{1}{2\sigma^2}(x-\mu)^2} \; dx. \end{eqnarray*} $$

Let $\dfrac{x-\mu}{\sigma}=z$ $\Rightarrow x=\mu+\sigma z$.

Therefore, $dx = \sigma dz$. And if $x = \pm \infty$, $z=\pm \infty$. Hence,

$$ \begin{eqnarray*} M_X(t) &=& \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty e^{t(\mu+\sigma z)}e^{-\frac{1}{2}z^2} \; \sigma dz\\ &=& \frac{e^{t\mu}}{\sqrt{2\pi}}\int_{-\infty}^\infty e^{t\sigma z}e^{-\frac{1}{2}z^2} \; dz\\ &=& \frac{e^{t\mu}}{\sqrt{2\pi}}\int_{-\infty}^\infty e^{-\frac{1}{2}(z^2-2t\sigma z)} \; dz\\ &=& \frac{e^{t\mu}}{\sqrt{2\pi}}\int_{-\infty}^\infty e^{-\frac{1}{2}(z^2-2t\sigma z+t^2\sigma^2-t^2\sigma^2)} \; dz\\ &=& \frac{e^{t\mu}}{\sqrt{2\pi}}\int_{-\infty}^\infty e^{-\frac{1}{2}(z-t\sigma)^2} e^{\frac{1}{2}t^2\sigma^2} \; dz\\ &=& \frac{e^{t\mu +\frac{1}{2}t^2\sigma^2}}{\sqrt{2\pi}}\int_{-\infty}^\infty e^{-\frac{1}{2}(z-t\sigma)^2} \; dz. \end{eqnarray*} $$

Let $z-t\sigma=u$ $\Rightarrow dz=du$ and $z=\pm \infty\Rightarrow u=\pm \infty$.

$$ \begin{eqnarray*} M_X(t) &=& \frac{e^{t\mu +\frac{1}{2}t^2\sigma^2}}{\sqrt{2\pi}}\int_{-\infty}^\infty e^{-\frac{1}{2}u^2} \; du.\\ &=&\frac{e^{t\mu +\frac{1}{2}t^2\sigma^2}}{\sqrt{2\pi}}\sqrt{2\pi}\\ &=&e^{t\mu +\frac{1}{2}t^2\sigma^2}. \end{eqnarray*} $$

C.G.F. of Normal Distribution

The cumulant generating function (CGF) of $N(\mu,\sigma^2)$ distribution is

$$ \begin{equation*} K_x(t)= \mu t + \frac{1}{2}t^2\sigma^2. \end{equation*} $$

Proof

The moment generating function of normal distribution with parameter $\mu$ and $\sigma^2$ is

$$ \begin{eqnarray*} M_X(t) &=& e^{\mu t + \frac{1}{2}t^2\sigma^2}. \end{eqnarray*} $$

Then the cumulant generating function of normal distribution is given by

$$ \begin{eqnarray*} K_x(t) &=& \log_e M_X(t) \\ &=& \mu t + \frac{1}{2}t^2\sigma^2. \end{eqnarray*} $$

Cumulants

$\kappa_1 = \mu_1^\prime =$ coefficient of $t$ in the expansion of $K_X(t)$ = $\mu$ = mean.

$\kappa_2 = \mu_2 =$ coefficient of $\dfrac{t^2}{2!}$ in the expansion of $K_X(t)$ = $\sigma^2$ = variance.

$\kappa_3 = \mu_3 =$ coefficient of $\dfrac{t^3}{3!}$ in the expansion of $K_X(t)$ = 0.

$\kappa_4 = \mu_4-3\kappa_2^2 =$ coefficient of $\dfrac{t^4}{4!}$ in the expansion of $K_X(t)$ = 0.

$\Rightarrow \mu_4 =0+3 (\sigma^2)^2 = 3\sigma^4$.

The coefficient of skewness is

$$ \begin{aligned} \beta_1 &= \frac{\mu_3^2}{\mu_2^3}\\ &= 0. \end{aligned} $$

Hence normal distribution is symmetric.

The coefficient of kurtosis is

$$ \begin{aligned} \beta_2 &= \frac{\mu_4}{\mu_2^2}\\ &= \frac{3\sigma^4}{\sigma^4}\\ &=3. \end{aligned} $$

Hence, normal distribution is mesokurtic.

Characteristics function of normal distribution

The characteristics function of $N(\mu,\sigma^2)$ distribution is

$$ \begin{equation*} \phi_X(t)=e^{it\mu -\frac{1}{2}t^2\sigma^2}. \end{equation*} $$

Proof

The characteristics function of normal distribution with parameter $\mu$ and $\sigma^2$ is

$$ \begin{eqnarray*} \phi_X(t) &=& E(e^{itX}) \\ &=& M_X(it)\\ &=&e^{it\mu +\frac{1}{2}(it)^2\sigma^2}\\ &=&e^{it\mu -\frac{1}{2}t^2\sigma^2}\\ & & \qquad (\because i^2 = -1). \end{eqnarray*} $$
(replacing $t$ by $it$ in moment generating function of $X$).

Median of Normal Distribution

The median of $N(\mu,\sigma^2)$ distribution is $M =\mu$.

Proof

Let $M$ be the median of the distribution. Hence

$$ \begin{eqnarray*} \int_{-\infty}^M f(x) \; dx = \frac{1}{2} & & \\ \text{i.e., } \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^M e^{-\frac{1}{2\sigma^2}(x-\mu)^2} \; dx & = & \frac{1}{2}. \end{eqnarray*} $$

Let $\dfrac{x-\mu}{\sigma}=z$ $\Rightarrow x=\mu+\sigma z$.

Therefore, $dx = \sigma dz$. Also, for $x = -\infty$, $z=- \infty$ and for $x=M$, $z= \frac{M-\mu}{\sigma}$. Hence,

$$ \begin{eqnarray*} \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\frac{M-\mu}{\sigma}} e^{-\frac{1}{2}z^2} \; dz & = & \frac{1}{2}\\ \int_{-\infty}^{\frac{M-\mu}{\sigma}} e^{-\frac{1}{2}z^2} \; dz & = & \sqrt{\frac{\pi}{2}} \end{eqnarray*} $$

We know that

$$ \begin{eqnarray*} \int_{-\infty}^{\infty} e^{-\frac{1}{2}z^2} \; dz & = & \sqrt{2\pi}\\\nonumber 2\int_{-\infty}^{0} e^{-\frac{1}{2}z^2} \; dz & = & \sqrt{2\pi}\\ \int_{-\infty}^{0} e^{-\frac{1}{2}z^2} \; dz & = & \sqrt{\frac{\pi}{2}}. \end{eqnarray*} $$

Comparing the two integrals, we get

$$ \begin{equation*} \frac{M-\mu}{\sigma}=0 \Rightarrow M=\mu. \end{equation*} $$

Hence, the median of the normal distribution is $M=\mu$.

Mode of Normal Distribution

The mode of $N(\mu,\sigma^2)$ distribution is $\mu$.

Proof

The p.d.f. of normal distribution is

$$ \begin{equation*} f(x;\mu, \sigma^2) = \left\{ \begin{array}{ll} \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2\sigma^2}(x-\mu)^2}, & \hbox{$-\infty< x<\infty$, $-\infty<\mu<\infty$, $\sigma^2>0$;} \\ 0, & \hbox{Otherwise.} \end{array} \right. \end{equation*} $$

Taking $\log_e$ of $f(x)$, we get

$$ \begin{equation*} \log_e f(x) = c-\frac{1}{2\sigma^2}(x-\mu)^2 \end{equation*} $$

Differentiating w.r.t. $x$ and equating to zero, we get

$$ \begin{eqnarray*} & & \frac{\partial \log_e f(x)}{\partial x}=0\\ \Rightarrow & &0-\frac{2}{2\sigma^2}(x-\mu)=0\\ \Rightarrow & &x=\mu. \end{eqnarray*} $$

And $\frac{\partial^2 \log_e f(x)}{\partial x^2}\bigg|_{x=\mu} <0$.

Hence by the principle maxima and minima, at $x=\mu$, the density function becomes maximum.

Therefore, mode of normal distribution is $\mu$.

For normal distribution, mean = median = mode = $\mu$. Hence normal distribution is symmetrical distribution and it is symmetric about
$x=\mu$.

Mean deviation about mean

The mean deviation about mean is

$$ \begin{equation*} E[|X-\mu|]=\sqrt{\frac{2}{\pi}}\sigma. \end{equation*} $$

Proof

The mean deviation about mean of $N(\mu,\sigma^2)$ distribution is

$$ \begin{eqnarray*} E(|X-\mu|) &=& \int_{-\infty}^\infty |x-\mu| f(x)\; dx\\ &=& \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty |x-\mu|e^{-\frac{1}{2\sigma^2}(x-\mu)^2} \; dx. \end{eqnarray*} $$

Let $\dfrac{x-\mu}{\sigma}=z$ $\Rightarrow x=\mu+\sigma z$.

Therefore, $dx = \sigma dz$. And if $x = \pm \infty$, $z=\pm \infty$. Hence,

$$ \begin{eqnarray*} E(|X-\mu|)&=& \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^\infty |\sigma z|e^{-\frac{1}{2}z^2}\sigma \;dz\\ &=& \frac{\sigma}{\sqrt{2\pi}}\int_{-\infty}^\infty |z|e^{-\frac{1}{2}z^2}\; dz\\ &=& \frac{2\sigma }{\sqrt{2\pi}}\int_{0}^\infty ze^{-\frac{1}{2}z^2}\; dz\\ &=& \frac{\sqrt{2}\sigma}{\sqrt{\pi}}\int_{0}^\infty ze^{-\frac{1}{2}z^2}\; dz \end{eqnarray*} $$

Let $z^2=y$, $z=\sqrt{y}$ $\Rightarrow dz = \frac{dy}{2\sqrt{y}}$ and $z=0 \Rightarrow y=0$, $z=\infty\Rightarrow y=\infty$. Hence,

$$ \begin{eqnarray*} E(|X-\mu|) &=& \frac{\sqrt{2}\sigma}{\sqrt{\pi}}\int_{0}^\infty \sqrt{y}e^{-\frac{1}{2}y}\; \frac{dy}{2\sqrt{y}}\\ &=& \frac{\sqrt{2}\sigma}{2\sqrt{\pi}}\int_{0}^\infty e^{-\frac{1}{2}y}\; dy\\ &=& \frac{\sqrt{2}\sigma}{2\sqrt{\pi}}\int_{0}^\infty y^{1-1}e^{-\frac{1}{2}y}\; dy\\ &=&\frac{\sqrt{2}\sigma}{2\sqrt{\pi}}\frac{\Gamma(1)}{(\frac{1}{2})^1}\\ &=&\sqrt{\frac{2}{\pi}}\sigma. \end{eqnarray*} $$

Hence, for normal distribution, mean deviation about mean in $\sqrt{\frac{2}{\pi}}\sigma$.

The sum of two independent normal variates is also a normal variate.

Let $X$ and $Y$ are independent normal variates with $N(\mu_1, \sigma^2_1)$ and $N(\mu_2, \sigma^2_2)$ distribution respectively. Then $X+Y\sim N(\mu_1+\mu_2,\sigma^2_1+\sigma^2_2)$.

Proof

Let $X \sim N(\mu_1, \sigma^2_1)$ and $Y\sim N(\mu_2, \sigma^2_2)$.

Moreover, $X$ and $Y$ are independently distributed. The m.g.f. of $X$ is

$$ \begin{equation*} M_X(t) = e^{t\mu_1 +\frac{1}{2} t^2\sigma^2_1} \end{equation*} $$

and

$$ \begin{equation*} M_Y(t) = e^{t\mu_2 +\frac{1}{2} t^2\sigma^2_2} \end{equation*} $$

Let, $Z = X+Y$. Then the m.g.f. of $Z$ is

$$ \begin{eqnarray*} M_Z(t) &= & E(e^{tZ})\\ &=& E(e^{t(X+Y)})\\ &=& E(e^{tX}\cdot e^{tY})\\ &=& E(e^{tX})\cdot E(e^{tY}),\;\qquad (X \text{ and } Y \text{ are independent}\\ &=& M_X(t)\cdot M_Y(t) \\ &=& e^{t\mu_1 +\frac{1}{2} t^2\sigma^2_1}\cdot e^{t\mu_2 +\frac{1}{2} t^2\sigma^2_2}\\ &=&e^{t(\mu_1+\mu_2) +\frac{1}{2} t^2(\sigma^2_1+\sigma^2_2)}, \end{eqnarray*} $$

which is the m.g.f. of normal variate with parameters $\mu_1+\mu_2$ and $\sigma^2_1+\sigma^2_2$.

Hence, $X+Y \sim N(\mu_1+\mu_2,\sigma^2_1+\sigma^2_2)$.

The difference of two independent normal variates is also a normal variate.

Let $X$ and $Y$ are independent normal variates with $N(\mu_1, \sigma^2_1)$ and $N(\mu_2, \sigma^2_2)$ distribution respectively. Then $X-Y\sim N(\mu_1-\mu_2,\sigma^2_1+\sigma^2_2)$.

Proof

Let $X \sim N(\mu_1, \sigma^2_1)$ and $Y\sim N(\mu_2, \sigma^2_2)$.

Moreover, $X$ and $Y$ are independently distributed. The m.g.f. of $X$ is

$$ \begin{equation*} M_X(t) = e^{t\mu_1 +\frac{1}{2} t^2\sigma^2_1} \end{equation*} $$

and

$$ \begin{equation*} M_Y(t) = e^{t\mu_2 +\frac{1}{2} t^2\sigma^2_2} \end{equation*} $$

Let, $Z = X-Y$. Then the m.g.f. of $Z$ is

$$ \begin{eqnarray*} M_Z(t) &= & E(e^{tZ})\\ &=& E(e^{t(X-Y)})\\ &=& E(e^{tX}\cdot e^{-tY})\\ &=& E(e^{tX})\cdot E(e^{-tY}),\;\qquad (X \text{ and } Y \text{ are independent)}\\ &=& M_X(t)\cdot M_Y(-t) \\ &=& e^{t\mu_1 +\frac{1}{2} t^2\sigma^2_1}\cdot e^{-t\mu_2 +\frac{1}{2}(-t)^2\sigma^2_2}\\ &=&e^{t(\mu_1-\mu_2) +\frac{1}{2} t^2(\sigma^2_1+\sigma^2_2)}, \end{eqnarray*} $$

which is the m.g.f. of normal variate with parameters $\mu_1-\mu_2$ and $\sigma^2_1+\sigma^2_2$. Hence, $X-Y \sim N(\mu_1-\mu_2,\sigma^2_1+\sigma^2_2)$.

Distribution of sample mean

Let $X_i, i=1,2,\cdots, n$ be independent sample observation from $N(\mu, \sigma^2)$ distribution then the sample mean $\overline{X}$ is normally distributed with mean $\mu$ and variance $\dfrac{\sigma^2}{n}$.

Proof

As $X_i\sim N(\mu, \sigma^2), i=1,2,\cdots, n$,

$$ \begin{equation*} M_{X_i}(t) =e^{t\mu +\frac{1}{2} t^2\sigma^2}, \; i=1,2,\cdots,n. \end{equation*} $$

Now, the m.g.f. of $\overline{X}$ is,

$$ \begin{eqnarray*} M_{\overline{X}}(t) &= & E(e^{t\overline{X}})\\ &=& E(e^{t\frac{1}{n}\sum_i X_i})\\ &=& E( e^{\frac{t}{n}X_1} e^{\frac{t}{n}X_2}\cdots e^{\frac{t}{n}X_n})\\ &=& E( e^{\frac{t}{n}X_1})\cdot E(e^{\frac{t}{n}X_2})\cdots E(e^{\frac{t}{n}X_n}),\qquad (X \text{ and } Y \text{ are independent})\\ &=& M_{X_1}(t/n)\cdot M_{X_2}(t/n)\cdots M_{X_n}(t/n) \\ &=& \prod_{i=1}^n M_{X_i}(t/n)\\ &=& \prod_{i=1}^n e^{\frac{t}{n}\mu +\frac{1}{2} \frac{t^2}{n^2}\sigma^2}\\ &=& e^{n\frac{t}{n}\mu +\frac{n}{2}\frac{t^2}{n^2}\sigma^2}\\ &=& e^{t\mu +\frac{1}{2}t^2\frac{\sigma^2}{n}}. \end{eqnarray*} $$

which is the m.g.f. of normal variate with parameters $\mu$ and $\frac{\sigma^2}{n}$. Hence, $\overline{X}\sim N(\mu,\frac{\sigma^2}{n})$.

Standard Normal Distribution

If $X\sim N(\mu, \sigma^2)$ distribution, then the variate $Z =\frac{X-\mu}{\sigma}$ is called a standard normal variate and $Z\sim N(0,1)$ distribution. The p.d.f. of standard normal variate $Z$ is

$$ \begin{equation*} f(z)= \left\{ \begin{array}{ll} \frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}z^2}, & \hbox{$-\infty < z<\infty$;} \\ 0, & \hbox{Otherwise.} \end{array} \right. \end{equation*} $$

Statement of the Empirical Rule

Suppose you have a normal distribution with mean $\mu$ and standard deviation $\sigma$. Then, each of the following is true:

68% of the data will occur within one standard deviation of the mean.
95% of the data will occur within two standard deviations of the mean.
99.7% of the data will occur within three standard deviations of the mean.

Conclusion

In this tutorial, you learned about theory of normal distribution like the probability density function, mean, median, median, variance, moment generating function and other properties of Normal distribution.

To learn more about other probability distributions, please refer to the following tutorial:

Probability distributions

Let me know in the comments if you have any questions on Normal Distribution and your thought on this article.

VRCBuzz co-founder and passionate about making every day the greatest day of life. Raju is nerd at heart with a background in Statistics. Raju looks after overseeing day to day operations as well as focusing on strategic planning and growth of VRCBuzz products and services. Raju has more than 25 years of experience in Teaching fields. He gain energy by helping people to reach their goal and motivate to align to their passion. Raju holds a Ph.D. degree in Statistics. Raju loves to spend his leisure time on reading and implementing AI and machine learning concepts using statistical models.

Leave a Comment