Introduction to Random Processes

Random processes are fundamental to many fields, including signal processing, telecommunications, and finance. A random process is essentially a collection of random variables indexed by time or space. These processes are used to model systems that evolve over time in uncertain ways. In this blog, we’ll introduce key concepts in random processes, such as probability density functions (PDFs), mean and correlation functions, and stationary processes.

---

1.1 Basic Concepts of Random Processes

A random process (or stochastic process) is a collection of random variables indexed by time or some other parameter. Formally, we represent a random process as:

$X(t),t\in T$

where $t$ is the index, which could represent time, and $X(t)$ is the random variable at time $t$. If $T$ is a continuous set, the process is called a continuous-time process; if $T$ is discrete, the process is a discrete-time process.

---

1.1.1 PDFs and CDFs

The Probability Density Function (PDF) and Cumulative Distribution Function (CDF) are crucial in describing random variables in a random process.

• The PDF $f_X(x)$ describes the likelihood of a random variable $X$ taking on a specific value $x$. It satisfies:

${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{f}_X(x)dx=1$

• The CDF $F_X(x)$ is the probability that the random variable $X$ is less than or equal to $x$:

$F_X(x)=P(X\le x)={\int }_{-\mathrm{\infty }}^x{f}_X(t)dt$

In random processes, PDFs and CDFs describe the distribution of random variables at each point in time.

---

1.1.2 Mean and Correlation Functions

For a random process $X(t)$, the mean function ${\mu }_X(t)$ is the expected value of $X(t)$ at time $t$:

${\mu }_X(t)=E[X(t)]$

The autocorrelation function $R_X(t_1,t_2)$ describes how random variables at different times are related:

$R_X(t_1,t_2)=E[X(t_1)X(t_2)]$

For stationary processes (discussed later), this function depends only on the time difference $\tau =t_2–t_1$.

---

1.1.3 Multiple Random Processes

In practice, we often deal with multiple random processes. For two random processes $X(t)$ and $Y(t)$, the cross-correlation function describes the relationship between the two processes at different times:

$R_{XY}(t_1,t_2)=E[X(t_1)Y(t_2)]$

The joint behavior of two or more random processes is vital in applications such as multi-channel communications and sensor networks.

---

1.1.4 Stationary Processes

A random process is said to be stationary if its statistical properties do not change over time. More formally, a process $X(t)$ is stationary if:

1. The mean ${\mu }_X(t)$ is constant: ${\mu }_X(t)={\mu }_X$
2. The autocorrelation function $R_X(t_1,t_2)$ depends only on the time difference $\tau =t_2–t_1$, not on the actual times $t_1$ and $t_2$.

For stationary processes, we can simplify the autocorrelation function as:

$R_X(\tau )=E[X(t)X(t+\tau )]$

Stationary processes are easier to analyze and are widely used in fields like signal processing.

---

1.1.5 Gaussian Random Processes

A Gaussian random process is one in which every finite collection of random variables has a joint Gaussian distribution. If $X(t)$ is a Gaussian process, then for any set of times $t_1,t_2,\dots ,t_n$, the random vector $[X(t_1),X(t_2),\dots ,X(t_n)]$ follows a multivariate normal distribution.

Gaussian processes are particularly important because of their simplicity and the fact that they are completely characterized by their mean and autocorrelation functions.

---

1.1.6 Solved Problems

Problem: Consider a stationary random process with autocorrelation function $R_X(\tau )=5e^{-|\tau |}$. Find the mean square value of the process.

Solution: The mean square value is the autocorrelation function evaluated at $\tau =0$:

$E[X^2(t)]=R_X(0)=5$

Thus, the mean square value of the process is 5.

---

1.2 Processing of Random Signals

In real-world systems, signals often contain random noise or variability. Random signals are processed to extract useful information, reduce noise, or enhance desired features. We will discuss key concepts in the processing of random signals, such as power spectral density and white noise.

---

1.2.0 Processing of Random Signals

The analysis and processing of random signals are critical in areas such as communications, radar systems, and control systems. The goal is to model, filter, and enhance random signals for better performance in these systems.

---

1.2.1 Power Spectral Density

The Power Spectral Density (PSD) of a random process describes how the power of the signal is distributed across different frequency components. For a stationary random process, the PSD $S_X(f)$ is the Fourier transform of the autocorrelation function $R_X(\tau )$:

$S_X(f)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{R}_X(\tau )e^{-j2\pi f\tau }d\tau$

The PSD provides valuable insights into the frequency characteristics of a random signal and is used in fields such as telecommunications and signal processing.

---

1.2.2 Linear Time-Invariant (LTI) Systems with Random Inputs

An LTI system with a random input $X(t)$ produces an output random process $Y(t)$. The relationship between the input and output is given by the convolution of the input with the system’s impulse response $h(t)$:

$Y(t)=h(t)\ast X(t)$

In the frequency domain, the PSD of the output is related to the PSD of the input and the frequency response of the system $H(f)$:

$S_Y(f)=|H(f){|}^2{S}_X(f)$

---

1.2.3 Power in a Frequency Band

The total power of a random process in a frequency band $[f_1,f_2]$ can be computed by integrating the PSD over that band:

$P_{[f_1,f_2]}={\int }_{f_1}^{f_2}{S}_X(f)df$

This is a key calculation in applications such as filtering and noise reduction.

---

1.2.4 White Noise

White noise is a special type of random process where all frequency components have equal power. The PSD of white noise is constant over all frequencies:

$S_W(f)=N_0/2$

White noise is often used to model random disturbances or background noise in systems.

---

1.2.5 Solved Problems

Problem: A random process has an autocorrelation function $R_X(\tau )=10e^{-2|\tau |}$. Find the power spectral density (PSD) of the process.

Solution: The PSD is the Fourier transform of the autocorrelation function:

$S_X(f)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}10e^{-2|\tau |}{e}^{-j2\pi f\tau }d\tau =\frac{10}{4+(2\pi f{)}^2}$

Thus, the PSD of the process is $S_X(f)=\frac{10}{4+(2\pi f{)}^2}$.

---

1.3 Problems and Exercises

In this section, we will present several end-of-chapter problems to further solidify the understanding of random processes, power spectral density, and LTI systems.

Problem: Compute the output PSD of an LTI system with impulse response $h(t)=e^{-t}u(t)$ and input white noise with PSD $S_W(f)=N_0/2$.

Solution: The frequency response of the system is:

$H(f)=\frac{1}{1+j2\pi f}$

The output PSD is:

$S_Y(f)=|H(f){|}^2{S}_W(f)=\frac{N_0}{2(1+4{\pi }^2{f}^2)}$

---

Conclusion

In this blog, we explored key topics in random processes, including PDFs and CDFs, correlation functions, stationary and Gaussian processes, power spectral density, and white noise. These concepts form the foundation for analyzing and processing random signals in real-world applications.