Continuous Possibility Distribution:PDF

Contents

1. Probability Density Function(PDF)

Continuous Possibility Distribution

If the probability of randomly selecting a number in a certain interval [a, b] is the same, the number becomes a random variable, and since the number of the interval is infinite, a single point cannot be specified. In other words, the probability at a particular point in a continuous variable cannot be defined. Instead, if intervals with equal probabilities are grouped, the probability of selecting a group can be defined as the length of that selected portion over the length of the entire interval. This relationship can be expressed as Equation 1.

$$\begin{align}\tag{1} P(X \in [a, b])&=1\\ P(X \in [x_1, x_2])& \varpropto (x_2-x_1)\\&= \frac{x_2-x_1}{b-a}\\ a\le x_1 \le x_2\le b \end{align}$$

Based on the above expression, the cumulative distribution function (CDF) for the random variable X is written as Equation 2.

$$\begin{align}\tag{2} F(X)=\begin{cases} 0 & \quad \text{for}\; x\le a\\ \frac{x-a}{b-a} & \quad \text{for}\; a \le x\le b\\ 1 & \quad \text{for}\; x\ge b \end{cases} \end{align}$$

In fact, in the case of a continuous variable, the probability at a point cannot be defined, so the difference between the symbols '≤' and '<' cannot be defined either.

If F(x) is a continuous function for all x, then the random variable X with the cumulative distribution function F(x) can be said to be continuous.

$$F(x)=P(X \le x)$$

In other words, the cumulative distribution function, F(x), is a function that is differentiable over all ranges.

Probability Density Function(PDF)

The probability of each case in the discrete probability function can be calculated with the probability mass function (PMF) and the cumulative distribution function (CDF). Similarly, a CDF can be defined for a continuous probability function. However, for a continuous variable whose point cannot be defined, a probability mass function (PMF) cannot be specified. In other words, it is difficult to specify any part of a continuous variable due to the infinity of its range. Therefore, in the case of a continuous variable, it is possible to replace the probability mass function of a discrete variable by calculating the probability of a range including that point, not a specific point. Such a function is called a probability density function and can be mathematically calculated using the concept of differentiation as follows.

$$\begin{align}f(x)&=\lim_{\Delta \to 0}\frac{P(x \lt X \le x+\Delta)}{\Delta}\\&=\lim_{\Delta \to 0}\frac{F(x+\Delta)-F(x)}{\Delta}\\&=\frac{dF(x)}{dx}=F^\prime(x) \\\because \; & P(x \lt X \le x+\Delta)=P(X \le x+\Delta)-P(X \lt x)\\&=F(x+\Delta)-F(x) \end{align}$$

The probability density function (PDF) of a continuous random variable X with the cumulative distribution function CDF is defined as the derivative of the CDF as shown in Equation 3. $$\begin{equation}\tag{3} f(x)=\frac{dF(x)}{dx} \end{equation}$$

The PDF above can be obtained by differentiating the CDF. Conversely, it means that CDF can be computed as the integral of PDF. If $\Delta$ is a very small value, the cumulative probability in a certain interval can be expressed as Equation 4.

$$\begin{align}\tag{4} &\begin{aligned} F(x)&=P(x \le X \le x+\delta) \varpropto f(x)\delta\\ &=\int^{ x+\delta}_{x}f(x)\delta \end{aligned}\\ &\begin{aligned}P(a \le x \le b)&=F(b)-F(a)\\&=\int^b_a f(x)dx\end{aligned} \\ & \int ^\infty_{-\infty} f(x)dx =1\end{align}$$

Example 1)
Assume a continuous random variable with the following PDF.

$$ f(x) =\begin{cases} ce^x& \quad x \ge 0\\ 0 & \quad \text{otherwise} \end{cases}$$

1) Determine c
It can be easily determined by integrating the above PMF. This problem requires integration and solving the equation. This process uses sympy's integrate(), Eq() and solve() functions. coded.

import numpy as np
import pandas as pd 
from sympy import *
import matplotlib.pyplot as plt
from scipy import stats 

c, x=symbols("c x")
f=c*exp(-x)
F=f.integrate((x, 0, oo ))
F

c

solve(Eq(F, 1), c)

[1]

2) $\displaystyle F(x)=\int^x_0e^{-x}\,dx$?

F=integrate(exp(-x),(x, 0, x))
F

$\quad \color{blue}{\displaystyle 1 - e^{- x}}$

3) $\displaystyle P(1 \lt X \lt 3)=F(3)-F(1)=\int^3_1 e^{-x}\,dx$

F=integrate(exp(-x),(x, 1, 3))
F

$\quad \color{blue}{\displaystyle - \frac{1}{e^{3}} + e^{-1}}$

N(F, 4)

0.3181

Example 2)
In a continuous random variable X having the following probability density function in all intervals

$$\displaystyle f(x)=\frac{e^{-|x|}}{2}$$

P(X ≤ 2)?

$$\begin{aligned} P(X \le 2) &=F(2)\\ &= \int^2_{-\infty} \frac{e^{-|x|}}{2} \, dx \end{aligned}$$

The above integration can be executed using sympy's integrate() function. However, this module function returns an expression instead of a number if the upper or lower bound of the integral is infinity. Therefore, the lower bound is computed with a very large value instead of infinity.

x = symbols("x")
f=exp(-abs(x))/2
integrate(f, (x, -oo, 2))

$\quad \color{blue}{\displaystyle \frac{\int\limits_{-\infty}^{2} e^{- \left|{x}\right|}\, dx}{2}}$

Fless2=integrate(f, (x, -1000000, 2))
Fless2.evalf(4)

0.9323

The integrate() function of the scipy module can be applied more simply than the sympy function used in the code above. This function returns the result and margin of error.

import scipy as sp 
sp.integrate.quad(lambda x: 1/2*np.exp(-abs(x)), -np.inf, 2)

(0.9323323583816946, 2.2674850885806563e-10)

Example 3)
The CDF for a continuous random variable X where f(x)=x between $0 \le x \le 1$ and 0 elsewhere can be expressed as

$$F(X)=\begin{cases} 0 & \quad x<0\\x & \quad 0 \le x \le 1\\1 & \quad x \ge 1 \end{cases}$$

1) If a random variable is expressed as a function $y=g(x)=e^x$, the CDF of that variable can be expressed as

$$\begin{align} F(y)&=F(e^x)\\&=P(Y \le y)\\&=P(e^X \le y)\\&P(X \le \text{ln}(y))\\&=\text{ln}(y) \end{align}$$

According to this function, the range of the random variable Y can be expressed as [1, e] based on the range of X. Within the scope, the CDF can be organized as:

$$F(y)=\begin{cases} 0 & \quad y \lt 1\\ \ln(y) & \quad 1 \le y \le e\\1 & \quad y \ge e \end{cases}$$

2) The PDF of the random variable Y can be derived from the derivative of the CDF above.

$$\begin{align} f(y)&=F^\prime(y)\\&=\frac{d(\ln(y))}{dy}\\&=\frac{1}{y}, \quad 1 \le y \le e \end{align}$$

3) E(Y)?

Expected values can be calculated using random variables Y or X.

$$\begin{align} E(Y)&=\int^e_1 yf(y)\, dy\\&=\int^e_1y\frac{1}{y} \, dy \\&=e-1\\ \text{or}\\E(Y)&=E(e^x)\\&=\int^1_0 e^xF^\prime(x)\, dx\\&=e-1 \end{align}$$

The relationship between random variables Y, f(y), and E(Y) is visualized as follows.

rng=np.arange(1, np.exp(1)+1E-6, 0.1)
plt.figure(figsize=(8, 4))
p=[1/i for i in rng]
plt.plot(rng, p)
plt.axvline(x=np.exp(1)-1, color="red", label="mean")
plt.xlabel("Y", fontsize="13", fontweight="bold")
plt.ylabel("f(y)", fontsize="13", fontweight="bold")
plt.legend(loc="best")
plt.show()

Considering again the derived part of f(y) in the above example, it can be expressed as

$$\begin{align}&y=e^x=g(x) \rightarrow x=g^{-1}(y)\\ &\begin{aligned} F(y)&=P(Y \le y)\\&=P(g(X) \le y)\\&=P(X < g^{-1}(y))\\&=F(g^{-1}(y))\\ f(y)&=\frac{d}{dy}F(x)\\&=\frac{dx}{dy}\frac{d(F(x))}{dx}\\&=\frac{dx}{dy}f^\prime(x)\\&=\frac{f^\prime(x)}{\frac{dy}{dx}}\\&=\frac{f^\prime(x)}{g^\prime(x)} \end{aligned} \end{align}$$

Under the assumption that X is a continuous random variable and g is a differentiable function, if Y=g(X), then the probability density function of Y is Equation 4.

$$\begin{equation}\tag{4}f(y)=\begin{cases} \frac{f(x)}{|g^\prime(x)|}=f(x)\big{|}\frac{dx}{dy}\big{|} & \quad \text{in}\, g(x)=y \\ 0 & \quad \text{in}\, g(x) \neq y \end{cases}\end{equation}$$

Example 4)

The probability density function of the random variable x is:

$$f(x) = \begin{cases} 4x^3 \quad & 0\le x \le 1\\ 0 \quad & \text{otherwise} \end{cases}$$

PDF of another random variable $\displaystyle Y=\frac{1}{X}$ based on random variable X?

$$\begin{align} &\begin{aligned}Y&=g(x)\\&=\frac{1}{X}\\ f(y)&=\frac{f(x)}{|g^\prime(x)|}\\&=\frac{4x^3}{\big{|}-\frac{1}{x^2}\big{|}}\\&=4x^5\\&=\frac{4}{y^5}\end{aligned}\\ &\therefore f(y)=\begin{cases} \frac{4}{y^5}& \quad y \ge 1\\0 &\quad \text{otherwise} \end{cases} \end{align}$$