Common distributions

Introduction

In the world of probability there exists a huge amount of different distributions. Some of these are almost never used, while others are used in our daily lives everywhere.

A distribution can be characterized by its probability density function (pdf) for time-continuous variables, and by its probability mass function (pmf). Many distributions also have very neat properties which make them easy to analyze and transform.

Gaussian Random Variable

Gaussian random variables are probably the most well used distribution of all. There are two main reasons for this, firstly, it mimics natural behavior pretty well in many cases, such as noise in electrical circuits, or how tall people are in a given place. The second reason is that it has properties that makes it very easy to work with.

By definition, we have the pdf given by

$latex p(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} e^{\frac{(x-\mu)^2}{2 {\sigma}^2 }}&s=2$

Where the random variable has mean $latex \mu$ and variance $latex \sigma$.The mean describes the value that the random variable, on average, takes on. Or it can be described as the “expected value”. The variance describes how much the variables moves around its mean.

The neat properties with Gaussians are the following:

If we assume that two Gaussians are independent, that they are not correlated or related, which means that they come from different sources. If we assume we have two variables $latex X_1, X_2$ we denote that they are Gaussian with the big N and following with the mean and variance in the parenthesis.

$latex X_1  \sim N( \mu_1 ,{\sigma_1}^2) $

$latex X_2  \sim N( \mu_2 ,{\sigma_2}^2) $

If we consider a third variable which is the sum of the two of these

$latex X_3 = X_1 + X_2 $

We then, from the properties of the Gaussian distribution, obtain

$latex X_3  \sim N( \mu_1 + \mu_2 ,{\sigma_1}^2 + {\sigma_2}^2) $

What a wonderful result! With simple logic we can see that this extends to more than two variables as well. However one important assumption that we made was that the random variables were independent. However in reality, we can often assume this, even if the variables are not exactly independent. An example of two independent things are the temperature in Australia and the number of people born in a small village in Russia. An example of two dependent things are the temperature in two nearby cities, we do not expect them to be exactly the same, but we expect to see a correlation, a dependence, between the temperatures.

Another important property is what happens when we multiply a Gaussian by a constant value, a scalar.

$latex X_2 = K X_1$

Where K is a constant and $latex X_1 $ is distributed as

$latex X_1  \sim N( \mu_1 ,{\sigma_1}^2) $

We then have that

$latex X_2  \sim N( K \mu_1 , K^2 { \sigma_1}^2) $

Hence the mean is simply multiplied by the constant, but the variance is multiplied by the square of the constant. This is an important difference that a lot of people sometimes forget about. The two mentioned properties are the two most fundamental and useful for the Gaussian distribution.

Poisson distribution

The Poisson distribution is another important and well used distribution. It is often used to model the number of events occurring during a certain amount of time. The Poisson random variable is a discrete variable, meaning that it only takes certain discrete values, such as 0, 1, 2 and 3.

To characterize the Poisson random variable we use only use the parameter $latex \lambda $.  This is often called the “event rate”. This describes how often event occurs.

The probability (denoted by Pr) that k number of events occur for a Poisson random variable with rate $latex \lambda $ is given by

$latex Pr(k) = \frac { \lambda^k  e^{- \lambda}}{k !}&s=2$

Where k! denotes “k factorial”, which is given by

$latex k! = (k) (k-1) (k-2) .. (2) (1) $

For example

$latex 3! = 3 * 2 * 1 = 6 $

Hence if we model something like a Poisson random variable, we can easily find the probability that we observe k number of events in a given period.