Negative Binomial Distribution

Negative Binomial Distribution

A negative binomial distribution is based on an experiment which satisfies the following three conditions:

  • An experiment consists of q sequence of independent Bernoulli's trials (i.e., each trial can result in a success ($S$) or a failure ($F$)),
  • The probability of success is constant from trial to trial, i.e., $P(S\text{ on trial } i) = p$, for $i=1,2,3,\cdots$,
  • The experiment continues until a total of $r$ successes have been observed, where $r$ is a specified positive integer.

Thus, a negative binomial experiment has the properties similar to Binomial distribution with the exception that the trials will be repeated until a fixed number of successes occur. If $X$ denote the number of failures that precede the $r^{th}$ success. then the random variable $X$ is called negative binomial random variable and the distribution of $X$ is called called negative binomial distribution.

Negative Binomial Distribution

Consider a sequence of independent repetitions of a Bernoulli's trial with constant probability $p$ of success and $q$ of failure. Let the random variable $X$ denote the total number of failures in this sequence before the $r^{th}$ success, i.e., $X+r$ is equal to the number of trials necessary to produce exactly $r$ successes, where $r$ is a fixed positive integer. Here we are interested in finding the probability $x$ failures before $r^{th}$ success.

Number of ways of getting $(r-1)$ success out of $(x+r-1)$ trials equals $\binom{x+r-1}{r-1}$. Hence, probability of $x$ failures before $r$ successes is given by

$$ \begin{eqnarray*} P(X=x) &=& P[(r-1) \text{ success in } (x+r-1) \text{ trials }]\\ & & \quad \times P[r^{th} \text{ success in } (x+r) \text{ trials}] \\ &=& \binom{x+r-1}{r-1} p^{r-1} q^{x} \times p \\ &=& \binom{x+r-1}{r-1} p^{r} q^{x},\quad x=0,1,2,\ldots; r=1,2,\ldots\\ & & \qquad\qquad \qquad 0 < p, q < 1, p+q=1. \end{eqnarray*} $$

Definition of Negative Binomial Distribution

A discrete random variable $X$ is said to have negative binomial distribution if its p.m.f. is given by

$$ \begin{eqnarray*} P(X=x)&=& \binom{x+r-1}{r-1} p^{r} q^{x},\\ & & \quad x=0,1,2,\ldots; r=1,2,\ldots\\ & & \qquad 0 < p, q < 1, p+q=1. \end{eqnarray*} $$

Key Features of Negative Binomial Distribution

  • A random experiment consists of repeated trials.
  • Each trial of an experiment has two possible outcomes, like success ($S$) and failure ($F$).
  • The probability of success ($p$) remains constants for each trial.
  • The trials are independent of each other.
  • The random variable is $X$ : the number of failures before getting $r^{th}$ success $(X = 0,1,2,\cdots )$.

Second form of Negative Binomial Distribution

Another form of negative binomial distribution can be obtained on simplifying $\binom{x+r-1}{r-1}$.

Consider
$$ \begin{eqnarray*} \binom{x+r-1}{r-1} &=& \binom{x+r-1}{x} \\ &=& \frac{(x+r-1)(x+r-2)\cdots (r-1)r}{x!} \\ &=& \frac{(-1)^x (-r)(-r-1)\cdots(-r-x+2)(-r-x+1)}{x!}\\ & = & (-1)^x \binom{-r}{x}. \end{eqnarray*} $$

Thus, another form of p.m.f. of negative binomial distribution is
$$ \begin{eqnarray*} P(X=x) &=& \binom{x+r-1}{r-1} p^{r} q^{x},\quad x=0,1,2,\ldots; r=1,2,\ldots\\ & & \qquad\qquad \qquad 0 < p, q < 1, p+q=1.\\ & = & (-1)^x \binom{-r}{x} p^r q^x \\ & = & \binom{-r}{x} p^r (-q)^x, \quad x=0,1,2,\ldots; r=1,2,\ldots\\ & & \qquad\qquad \qquad 0 < p, q < 1, p+q=1. \end{eqnarray*} $$

Third form of negative binomial distribution

Let $p = \dfrac{1}{Q}$ and $q=\dfrac{P}{Q}$. So $p+q=\dfrac{1}{Q}+\dfrac{P}{Q}= \dfrac{1+P}{Q}=1$. Hence we have $Q-P=1$.

Then the above p.m.f. can be written as
$$ \begin{eqnarray*} P(X=x) & = & \binom{-r}{x} (1/Q)^r (-P/Q)^x, \quad x=0,1,2,\ldots\\ & & \qquad\qquad \qquad Q-P=1.\\ & = & \binom{-r}{x} Q^{-r} (-P/Q)^x, \quad x=0,1,2,\ldots\\ & & \qquad\qquad \qquad Q - P=1.\\ \end{eqnarray*} $$

$P(X=x)$ is $(x+1)^{th}$ terms in the expansion of $(Q-P)^{-r}$. It is known as negative binomial distribution because of $-$ve index.

Clearly, $P(x)\geq 0$ for all $x\geq 0$, and
$$ \begin{eqnarray*} \sum_{x=0}^\infty P(X=x)&=& \sum_{x=0}^\infty \binom{-r}{x} Q^{-r} (-P/Q)^x, \\ &=& Q^{-r}\sum_{x=0}^\infty \binom{-r}{x} (-P/Q)^x, \\ &=& Q^{-r} \bigg(1-\frac{P}{Q}\bigg)^{-r}\\ & & \quad \big(\because(1-q)^{-r} = \sum_{x=0}^\infty \binom{-r}{x} (-q)^x\big)\\ &=& Q^{-r} \bigg(\frac{Q-P}{Q}\bigg)^{-r}\\ &=& (Q-P)^{-r} = 1^{-r}=1. \end{eqnarray*} $$

Hence, $P(X=x)$ defined above is a legitimate probability mass function.

Geometric as a particular case of negative binomial

In negative binomial distribution, if $r=1$ then we get
$$ \begin{eqnarray*} P(X=x) &=& q^x p, x=0,1,2,\ldots;\\ & & \quad 0 < p, q < 1; p+q=1 \end{eqnarray*} $$
which is the p.m.f. of geometric distribution. Hence Geometric distribution is the particular case of negative binomial distribution.

Graph of Negative Binomial Distribution

Following graph shows the probability mass function of negative binomial distribution with parameters $r=4$ and $p=0.56$.

Negative Binomial distribution
Negative Binomial distribution

Mean of Negative Binomial Distribution

The mean of negative binomial distribution is $E(X)=\dfrac{rq}{p}$.

Proof

The mean of random variable $X$ is given by $E(X)$.

$$ \begin{eqnarray*} \mu_1^\prime =E(X) &=& \sum_{x=0}^\infty x\cdot P(X=x) \\ &=& \sum_{x=0}^\infty x\cdot \binom{-r}{x} p^r (-q)^x\\ &=& p^r \sum_{x=0}^\infty x\frac{(-r)!}{x!(-r-x)!}\cdot (-q)^{x}\\ &=& p^r \sum_{x=1}^\infty \frac{(-r)(-r-1)!}{(x-1)!(-r-x)!}\cdot (-q)^{x}\\ &=& p^r(-r)(-q)\sum_{x=1}^\infty \binom{-r-1}{x-1}\cdot (-q)^{x-1}\\ &=& p^r rq\sum_{x=1}^\infty \binom{-r-1}{x-1}\cdot p^{r-1}(-q)^{x-1}\\ &=& p^rrq(1-q)^{-r-1}\\ &=& \frac{rq}{p}. \end{eqnarray*} $$

Variance of Negative Binomial Distribution

The variance of negative binomial distribution is $V(X)=\dfrac{rq}{p^2}$.

Proof

The variance of random variable $X$ is given by

$$ V(X) = E(X^2) - [E(X)]^2 $$

Let us find the expected value $X^2$.

$$ \begin{eqnarray*} E(X^2) &=& E[X(X-1)+X] \\ &=& E[X(X-1)]+E(X)\\ &=& \sum_{x=0}^\infty x(x-1) P(X=x) +\frac{rq}{p}\\ &=& \sum_{x=0}^\infty \binom{-r}{x} p^r (-q)^x +\frac{rq}{p} \\ &=& p^r (-r)(-r-1)(-q)^2 \sum_{x=2}^\infty \binom{-r-2}{x-2}(-q)^{x-2}+\frac{rq}{p}\\ & =& r(r+1)q^2p^r (1-q)^{-r-2}+\frac{rq}{p}\\ &=& \frac{r(r+1)q^2p^r}{p^{r+2}}+\frac{rq}{p}\\ &=& \frac{r(r+1)q^2}{p^2}+\frac{rq}{p}. \end{eqnarray*} $$

Now,

$$ \begin{eqnarray*} V(X) &=& E(X^2)-[E(X)]^2 \\ &=& \frac{r(r+1)q^2}{p^2}+\frac{rq}{p}-\frac{r^2q^2}{p^2}\\ &=& \frac{rq^2}{p^2}+\frac{rq}{p}\\ &=&\frac{rq}{p^2}. \end{eqnarray*} $$

For negative binomial distribution $E(X)< V(X)$, i.e., $\dfrac{rq}{p} < \dfrac{rq}{p^2}$.

MGF of Negative Binomial Distribution

The moment generating function of negative binomial distribution is $M_X(t)=\big(Q-Pe^{t}\big)^{-r}$.

Proof

The moment generating of negative binomial distribution is

$$ \begin{eqnarray*} M_X(t) &=& E(e^{tX}) \\ &=& \sum_{x=0}^\infty e^{tx} \binom{-r}{x} Q^{-r} (-P/Q)^x, \\ &=& \sum_{x=0}^\infty \binom{-r}{x} Q^{-r} (-Pe^{t}/Q)^x, \\ &=& Q^{-r} \bigg(1-\frac{Pe^{t}}{Q}\bigg)^{-r}\\ &=& Q^{-r} \bigg(\frac{Q-Pe^{t}}{Q}\bigg)^{-r}\\ &=& \big(Q-Pe^{t}\big)^{-r}. \end{eqnarray*} $$

Hence, the m.g.f. of negative binomial distribution is $M_X(t)=\big(Q-Pe^{t}\big)^{-r}$.

Mean and Variance Using MGF

The first raw moment of $X$ is

$$ \begin{eqnarray*} \mu_1^\prime &=& \bigg[\frac{d}{dt} M_X(t)\bigg]_{t=0}\\ &=& \bigg[\frac{d}{dt} (Q-Pe^t)^{-r}\bigg]_{t=0} \\ &=& \big[(-r)(Q-Pe^t)^{-r-1} Pe^t (-1)\big]_{t=0}\\ & = & rP\big[(Q-Pe^t)^{-r-1}e^t\big]_{t=0}\\ & = & rP\qquad (\because Q-P =1). \end{eqnarray*} $$

The second raw moment of $X$ is

$$ \begin{eqnarray*} \mu_2^\prime &=& \bigg[\frac{d^2}{dt^2} M_X(t)\bigg]_{t=0}\\ &=&\bigg[\frac{d}{dt} rPe^t(Q-Pe^t)^{-r-1}\bigg]_{t=0} \\ &=& rP\bigg[e^t(-r-1)(Q-Pe^t)^{-r-2}(-Pe^t)+(Q-Pe^t)^{-r-1}e^t\bigg]_{t=0} \\ &=& rP\bigg[ (r+1)Pe^{2t}(Q-Pe^t)^{-r-2}+(Q-Pe^t)^{-r-1}e^t\bigg]_{t=0}\\ &=& r(r+1)P^2 +rP. \end{eqnarray*} $$

Therefore, the variance of $X$ is

$$ \begin{eqnarray*} \mu_2 &=& \mu_2^\prime-(\mu_1^\prime)^2 \\ &=& r(r+1)P^2 +rP-(rP)^2\\ &=& r^2P^2 + rP^2+rP - r^2P^2\\ & = & rP(P+1)\\ &=& rPQ. \end{eqnarray*} $$

Hence, mean $= \mu_1^\prime =rP$ and variance $=\mu_2 = rPQ$.

The MGF of negative binomial distribution is $M_X(t)=\big(Q-Pe^{t}\big)^{-r}$.

Letting $p=\dfrac{1}{Q}$ and $q=\dfrac{P}{Q}$, the m.g.f. of negative binomial distribution in terms of $p$ and $q$ is $M_X(t) =p^r(1-qe^t)^{-r}$.

In terms of $p$ and $q$, the mean and variance of negative binomial distribution are respectively $\dfrac{rq}{p}$ and $\dfrac{rq}{p^2}$.

CGF of Negative Binomial Distribution

The CGF of negative binomial distribution is $K_X(t)=-r\log_e(Q-Pe^t)$.

Proof

The CGF of negative binomial distribution is

$$ \begin{eqnarray*} K_X(t) &=& \log_e M_X(t)\\ &=& -r\log_e(Q-Pe^t). \end{eqnarray*} $$

$$ \begin{eqnarray*} K_X(t)&=& -r\log_e(Q-Pe^t)\\ &=& -r\log_e\bigg[Q-P\bigg(1+t +\frac{t^2}{2!}+ \frac{t^3}{3!}+\cdots \bigg) \bigg]\\ &=& -r\log_e\bigg[Q-P-P\bigg(t +\frac{t^2}{2!}+ \frac{t^3}{3!}+\cdots \bigg) \bigg]\\ &=& -r\log_e\bigg[1-P\bigg(t +\frac{t^2}{2!}+ \frac{t^3}{3!}+\cdots \bigg) \bigg]\\ &=& r\bigg[P\bigg(t +\frac{t^2}{2!}+ \frac{t^3}{3!}+\cdots \bigg)+\frac{P^2}{2}\bigg(t +\frac{t^2}{2!}+ \frac{t^3}{3!}+\cdots \bigg)^2+\\ & &\frac{P^3}{3}\bigg(t +\frac{t^2}{2!}+ \frac{t^3}{3!}+\cdots \bigg)^3+\frac{P^4}{4}\bigg(t +\frac{t^2}{2!}+ \frac{t^3}{3!}+\cdots \bigg)^4+\cdots \bigg] \end{eqnarray*} $$

The $r^{th}$ cumulant of $X$ is given by

$$ \begin{equation*} \kappa_r = \text{coefficient of $\frac{t^r}{r!}$ in the expansion of $K_X(t)$} \end{equation*} $$

Collecting the coefficient of $\frac{t^r}{r!}$ from the expansion of $K_X(t)$, we have

$$ \begin{eqnarray*} \kappa_1 =\mu_1^\prime &=& \text{coefficient of $t$ in the expansion of $K_X(t)$} \\ &=& rP \\ \end{eqnarray*} $$

$$ \begin{eqnarray*} \kappa_2 =\mu_2 &=& \text{coefficient of $\frac{t^2}{2!}$ in the expansion of $K_X(t)$} \\ &=& r(P+P^2) \\ & = & rP(1+P) = rPQ. \end{eqnarray*} $$

$$ \begin{eqnarray*} \kappa_3 =\mu_3 &=& \text{coefficient of $\frac{t^3}{3!}$ in the expansion of $K_X(t)$} \\ &=& r(P+3P^2+2P^3) \\ &=& rP(1+3P+2P^2)\\ &=& rP(1+P)(1+2P). \end{eqnarray*} $$

$$ \begin{eqnarray*} \kappa_4 &=& \text{coefficient of $\frac{t^4}{4!}$ in the expansion of $K_X(t)$} \\ &=& r[P+(3P^2+4P^2) + (6P^3+6P^3) + 6P^4] \\ &=& rPQ(1+6PQ) \end{eqnarray*} $$

Hence,

$$ \begin{eqnarray*} \mu_4 &=& \kappa_4 + 3\kappa_2^2 \\ &=& rPQ [1+3PQ(r+2)]. \end{eqnarray*} $$

Characteristics Function of negative binomial distribution

The characteristics function of negative binomial distribution is

$\phi_X(t) = p^r (1-qe^{it})^{-r}$.

Proof

The characteristics function of negative binomial distribution is
$$ \begin{eqnarray*} \phi_X(t) &=& E(e^{itX})\\ &=& M_X(it) \\ &=& (Q-Pe^{it})^{-r}\\ &=& p^r (1-qe^{it})^{-r}. \end{eqnarray*} $$

Recurrence Relation for the probability of Negative Binomial Distribution

The recurrence relation for the probabilities of negative binomial distribution is

$$ \begin{equation*} P(X=x+1) = \frac{(x+r)}{(x+1)} q \cdot P(X=x),\quad x=0,1,\cdots \end{equation*} $$

where $P(X=0) = p^r$.

Proof

The p.m.f. of negative binomial distribution is

$$ \begin{equation*} P(X=x) = \binom{x+r-1}{r-1} p^{r} q^{x} \end{equation*} $$

Then $P(X=x+1)$ is

$$ \begin{aligned} P(X=x+1) &= \binom{x+1+r-1}{r-1} p^{r} q^{x+1}\\ &=\binom{x+r}{r-1} p^{r} q^{x+1} \end{aligned} $$

Taking the ratio of $P(X=x+1)$ and $P(X=x)$, we have

$$ \begin{eqnarray*} \frac{P(X=x+1)}{P(X=x)} &=& \frac{\binom{x+r}{r-1} p^{r} q^{x+1}}{\binom{x+r-1}{r-1} p^{r} q^{x}} \\ &=& \frac{(x+r)!}{(r-1)!(x+1)!}\frac{(r-1)!x!}{(x+r-1)!}q \\ &=& \frac{(x+r)}{(x+1)}q. \end{eqnarray*} $$

Hence, the recurrence relation for probabilities of negative binomial distribution is

$$ \begin{equation*} P(X=x+1) = \frac{(x+r)}{(x+1)} q \cdot P(X=x),\quad x=0,1,\cdots \end{equation*} $$
where $P(X=0) = p^r$.

Poisson Distribution as a limiting case of Negative Binomial Distribution

Negative binomial distribution $NB(r, p)$ tends to Poisson
distribution as $r\to \infty$ and $P\to 0$ with $rP = \lambda$
(finite).

Proof

Let $X\sim NB(r,P)$ distribution.

Then the PMF of $X$ is

$$ \begin{equation*} P(X=x) =\binom{x+r-1}{r-1} Q^{-r}(P/Q)^x. \end{equation*} $$

$$ \begin{eqnarray*} \lim_{r\to \infty \atop P\to 0} P(X=x) &=& \lim_{r\to \infty\atop P\to 0} \binom{x+r-1}{r-1} Q^{-r}(P/Q)^x \\ &=& \lim_{r\to \infty \atop P\to 0} \frac{(x+r-1)!}{x!(r-1)!} (1+P)^{-r}\bigg(\frac{P}{1+P}\bigg)^x \\ &=& \lim_{r\to \infty\atop P\to 0} \frac{(x+r-1)(x+r-2)\cdots (r+1)\cdot r}{x!} (1+P)^{-r}\bigg(\frac{P}{1+P}\bigg)^x \\ &=& \lim_{r\to \infty\atop P\to 0} \frac{r^x}{x!}\cdot\frac{(x+r-1)}{r}\cdot \frac{(x+r-2)}{r}\cdots \frac{(r+1)}{r}\cdot \frac{r}{r} (1+P)^{-r}\bigg(\frac{P}{1+P}\bigg)^x\\ &=& \lim_{r\to \infty\atop P\to 0} \frac{(rP)^x}{x!}\cdot\bigg(1+\frac{x-1}{r}\bigg)\cdot \bigg(1+\frac{x-2}{r}\bigg)\cdots \bigg(1+\frac{1}{r}\bigg) (1+P)^{-r}(1+P)^{-x}\\ &=& \frac{\lambda^x}{x!}\lim_{r\to \infty\atop P\to 0} \bigg(1+\frac{x-1}{r}\bigg)\cdot \bigg(1+\frac{x-2}{r}\bigg)\cdots \bigg(1+\frac{1}{r}\bigg) (1+P)^{-r}(1+P)^{-x}\\ &=& \frac{\lambda^x}{x!}\lim_{r\to \infty} \bigg(1+\frac{\lambda}{r}\bigg)^{-r}\lim_{r\to \infty}\bigg(1+\frac{\lambda}{r}\bigg)^{-x}, \quad (\because \lambda = rP) \end{eqnarray*} $$

But

$$ \begin{equation*} \lim_{r\to \infty}\bigg(1+\frac{\lambda}{r}\bigg)^{-x}=1, \text{ and } \lim_{r\to \infty}\bigg(1+\frac{\lambda}{r}\bigg)^{-r}=e^{-\lambda}. \end{equation*} $$

Hence,

$$ \begin{eqnarray*} \lim_{r\to \infty, P\to 0} P(X=x) &=& \frac{e^{-\lambda}\lambda^x}{x!}, \\ & & \quad x=0,1,2,\cdots,\; \; \lambda > 0 \end{eqnarray*} $$

which is the probability mass function of Poisson distribution.

Conclusion

In this tutorial, you learned about theory of Negative Binomial distribution like the probability mass function, mean, variance, moment generating function and other properties of Negative Binomial distribution.

To read more about the step by step examples and calculator for Negative Binomial distribution refer the link Negative Binomial Distribution Calculator with Examples. This tutorial will help you to understand how to calculate mean, variance of Negative Binomial distribution and you will learn how to calculate probabilities and cumulative probabilities for Negative Binomial distribution with the help of step by step examples.

To learn more about other discrete probability distributions, please refer to the following tutorial:

Probability distributions

Let me know in the comments if you have any questions on Negative Binomial Distribution and your thought on this article.

VRCBuzz co-founder and passionate about making every day the greatest day of life. Raju is nerd at heart with a background in Statistics. Raju looks after overseeing day to day operations as well as focusing on strategic planning and growth of VRCBuzz products and services. Raju has more than 25 years of experience in Teaching fields. He gain energy by helping people to reach their goal and motivate to align to their passion. Raju holds a Ph.D. degree in Statistics. Raju loves to spend his leisure time on reading and implementing AI and machine learning concepts using statistical models.

Leave a Comment