Linear Systems & Random Inputs

Cheatsheet Content

### Introduction to Random Processes & Linear Systems This cheatsheet covers the fundamental concepts, formulas, and derivations for analyzing the behavior of linear systems when subjected to random inputs, a core topic in probability theory and random processes. Understanding this interaction is crucial in various fields, including signal processing, control systems, and communication. ### Random Processes Basics A random process $X(t)$ is a collection of random variables indexed by time $t$. #### Mean and Autocorrelation - **Mean Function:** $μ_X(t) = E[X(t)]$ - **Autocorrelation Function (ACF):** $R_X(t_1, t_2) = E[X(t_1)X(t_2)]$ - Measures the statistical dependence between two samples of the process at different times. #### Stationarity - **Strict-Sense Stationary (SSS):** The joint probability distribution of any set of samples does not depend on a shift in time. - **Wide-Sense Stationary (WSS):** 1. Mean is constant: $μ_X(t) = μ_X$ 2. Autocorrelation depends only on the time difference $\tau = t_2 - t_1$: $R_X(t_1, t_2) = R_X(\tau)$ - For a WSS process, $R_X(\tau) = E[X(t)X(t+\tau)]$ #### Power Spectral Density (PSD) - For a WSS process, the PSD $S_X(\omega)$ is the Fourier Transform of the ACF $R_X(\tau)$. $$S_X(\omega) = \mathcal{F}\{R_X(\tau)\} = \int_{-\infty}^{\infty} R_X(\tau) e^{-j\omega\tau} d\tau$$ - By inverse Fourier Transform: $$R_X(\tau) = \mathcal{F}^{-1}\{S_X(\omega)\} = \frac{1}{2\pi} \int_{-\infty}^{\infty} S_X(\omega) e^{j\omega\tau} d\omega$$ - **Relation to Average Power:** $E[X^2(t)] = R_X(0) = \frac{1}{2\pi} \int_{-\infty}^{\infty} S_X(\omega) d\omega$ - This is the total average power of the random process. ### Linear Systems A linear time-invariant (LTI) system is characterized by its impulse response $h(t)$ or its frequency response $H(\omega)$. #### Input-Output Relationship - **Time Domain:** If $X(t)$ is the input and $Y(t)$ is the output, then: $$Y(t) = X(t) * h(t) = \int_{-\infty}^{\infty} X(\tau) h(t-\tau) d\tau$$ - **Frequency Domain:** $$S_Y(\omega) = H(\omega) S_X(\omega)$$ (This is INCORRECT, see next section for correct PSD relationship) $$Y(\omega) = H(\omega) X(\omega)$$ ### Input-Output Relations for Random Signals in LTI Systems #### Derivation of Mean of Output Given an LTI system with impulse response $h(t)$ and input $X(t)$: $Y(t) = \int_{-\infty}^{\infty} h(\tau) X(t-\tau) d\tau$ Taking the expected value: $E[Y(t)] = E\left[\int_{-\infty}^{\infty} h(\tau) X(t-\tau) d\tau\right]$ Assuming linearity of expectation and integral (often valid): $E[Y(t)] = \int_{-\infty}^{\infty} h(\tau) E[X(t-\tau)] d\tau$ If $X(t)$ is WSS, then $E[X(t-\tau)] = μ_X$ (constant mean). $μ_Y(t) = \int_{-\infty}^{\infty} h(\tau) μ_X d\tau = μ_X \int_{-\infty}^{\infty} h(\tau) d\tau$ We know that $H(\omega) = \int_{-\infty}^{\infty} h(\tau) e^{-j\omega\tau} d\tau$. Therefore, $\int_{-\infty}^{\infty} h(\tau) d\tau = H(0)$ (the DC gain of the system). So, for a WSS input: $$μ_Y = μ_X H(0)$$ #### Derivation of Autocorrelation of Output $R_Y(t_1, t_2) = E[Y(t_1)Y(t_2)]$ $Y(t_1) = \int_{-\infty}^{\infty} h(\alpha) X(t_1-\alpha) d\alpha$ $Y(t_2) = \int_{-\infty}^{\infty} h(\beta) X(t_2-\beta) d\beta$ $R_Y(t_1, t_2) = E\left[\int_{-\infty}^{\infty} h(\alpha) X(t_1-\alpha) d\alpha \int_{-\infty}^{\infty} h(\beta) X(t_2-\beta) d\beta\right]$ $R_Y(t_1, t_2) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} h(\alpha) h(\beta) E[X(t_1-\alpha)X(t_2-\beta)] d\alpha d\beta$ Since $X(t)$ is WSS, $E[X(t_1-\alpha)X(t_2-\beta)] = R_X(t_2-\beta - (t_1-\alpha)) = R_X(t_2-t_1 - (\beta-\alpha))$. Let $\tau = t_2-t_1$. $R_Y(\tau) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} h(\alpha) h(\beta) R_X(\tau - (\beta-\alpha)) d\alpha d\beta$ This is a double convolution. It can be simplified in the frequency domain. #### Power Spectral Density of Output Taking the Fourier Transform of $R_Y(\tau)$: $S_Y(\omega) = \mathcal{F}\{R_Y(\tau)\}$ From the double convolution integral, using the convolution theorem property $\mathcal{F}\{f*g\} = F(\omega)G(\omega)$: The Fourier Transform of $h(\alpha)h(\beta)$ term is related to $H(\omega)H^*(\omega)$. The key result for a WSS input $X(t)$ to an LTI system with frequency response $H(\omega)$ is: $$S_Y(\omega) = |H(\omega)|^2 S_X(\omega)$$ **Derivation Sketch:** 1. Start with the autocorrelation $R_Y(\tau)$ integral. 2. Apply the Fourier Transform to $R_Y(\tau)$. 3. Use the property that $\mathcal{F}\{R_X(\tau - (\beta-\alpha))\} = S_X(\omega) e^{-j\omega(\beta-\alpha)}$. 4. Manipulate the integral to separate terms involving $H(\omega)$ and $H^*(\omega)$. Recall that $H^*(\omega) = \mathcal{F}\{h(-\tau)\}$. The double integral becomes: $S_Y(\omega) = \int_{-\infty}^{\infty} h(\alpha) e^{j\omega\alpha} d\alpha \int_{-\infty}^{\infty} h(\beta) S_X(\omega) e^{-j\omega\beta} d\beta$ $S_Y(\omega) = \left(\int_{-\infty}^{\infty} h(\alpha) e^{j\omega\alpha} d\alpha\right) S_X(\omega) \left(\int_{-\infty}^{\infty} h(\beta) e^{-j\omega\beta} d\beta\right)$ $S_Y(\omega) = H^*(\omega) S_X(\omega) H(\omega)$ Therefore, $S_Y(\omega) = |H(\omega)|^2 S_X(\omega)$. #### Cross-Correlation and Cross-Spectral Density - **Cross-correlation between input and output:** $R_{XY}(\tau) = E[X(t)Y(t+\tau)]$ $$R_{XY}(\tau) = R_X(\tau) * h(\tau)$$ **Derivation:** $R_{XY}(\tau) = E[X(t) \int_{-\infty}^{\infty} h(\alpha) X(t+\tau-\alpha) d\alpha]$ $R_{XY}(\tau) = \int_{-\infty}^{\infty} h(\alpha) E[X(t)X(t+\tau-\alpha)] d\alpha$ $R_{XY}(\tau) = \int_{-\infty}^{\infty} h(\alpha) R_X(\tau-\alpha) d\alpha = R_X(\tau) * h(\tau)$ - **Cross-spectral density between input and output:** $$S_{XY}(\omega) = S_X(\omega) H(\omega)$$ **Derivation:** Take the Fourier Transform of $R_{XY}(\tau)$. - **Cross-correlation between output and input:** $R_{YX}(\tau) = E[Y(t)X(t+\tau)]$ $$R_{YX}(\tau) = R_X(\tau) * h(-\tau)$$ **Derivation:** Similar to $R_{XY}(\tau)$, but with $Y(t)$ first. $R_{YX}(\tau) = E[\int_{-\infty}^{\infty} h(\alpha) X(t-\alpha) d\alpha \cdot X(t+\tau)]$ $R_{YX}(\tau) = \int_{-\infty}^{\infty} h(\alpha) E[X(t-\alpha)X(t+\tau)] d\alpha$ $R_{YX}(\tau) = \int_{-\infty}^{\infty} h(\alpha) R_X(\tau+\alpha) d\alpha$ Let $\beta = -\alpha$, then $d\alpha = -d\beta$. $R_{YX}(\tau) = \int_{\infty}^{-\infty} h(-\beta) R_X(\tau-\beta) (-d\beta) = \int_{-\infty}^{\infty} h(-\beta) R_X(\tau-\beta) d\beta = R_X(\tau) * h(-\tau)$ - **Cross-spectral density between output and input:** $$S_{YX}(\omega) = S_X(\omega) H^*(\omega)$$ **Derivation:** Take the Fourier Transform of $R_{YX}(\tau)$. Since $\mathcal{F}\{h(-\tau)\} = H^*(\omega)$. #### Summary of Input-Output Relationships for WSS Input to LTI System - **Mean:** $μ_Y = μ_X H(0)$ - **Autocorrelation (Time Domain):** $R_Y(\tau) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} h(\alpha) h(\beta) R_X(\tau - (\beta-\alpha)) d\alpha d\beta$ - **PSD (Frequency Domain):** $S_Y(\omega) = |H(\omega)|^2 S_X(\omega)$ - **Cross-PSD (Input to Output):** $S_{XY}(\omega) = S_X(\omega) H(\omega)$ - **Cross-PSD (Output to Input):** $S_{YX}(\omega) = S_X(\omega) H^*(\omega)$ ### White Noise Input White noise is a theoretical random process with a flat power spectral density. #### Properties of White Noise - **Autocorrelation:** $R_W(\tau) = N_0 \delta(\tau)$, where $N_0$ is the power spectral density level. - **Power Spectral Density:** $S_W(\omega) = N_0$ (constant for all $\omega$). - This implies infinite power, hence it's an idealization. #### Output of LTI System with White Noise Input If $X(t)$ is white noise with $S_X(\omega) = N_0$: - **Output PSD:** $S_Y(\omega) = |H(\omega)|^2 N_0$ - **Output Autocorrelation:** $R_Y(\tau) = \mathcal{F}^{-1}\{N_0 |H(\omega)|^2\}$ Since $|H(\omega)|^2 = H(\omega)H^*(\omega) = \mathcal{F}\{h(t)\} \mathcal{F}\{h(-t)\} = \mathcal{F}\{h(t) * h(-t)\}$, then $R_Y(\tau) = N_0 (h(\tau) * h(-\tau))$ The function $h(\tau) * h(-\tau)$ is also known as the autocorrelation of the impulse response $R_h(\tau)$. So, $R_Y(\tau) = N_0 R_h(\tau)$. - **Average Power of Output:** $E[Y^2(t)] = R_Y(0) = \frac{1}{2\pi} \int_{-\infty}^{\infty} N_0 |H(\omega)|^2 d\omega = \frac{N_0}{2\pi} \int_{-\infty}^{\infty} |H(\omega)|^2 d\omega$ This is a crucial result for calculating signal power at the output of a system. ### Examples and Applications #### RC Low-Pass Filter - **Impulse Response:** $h(t) = \frac{1}{RC} e^{-t/RC} u(t)$ - **Frequency Response:** $H(\omega) = \frac{1}{1+j\omega RC}$ - **Magnitude Squared:** $|H(\omega)|^2 = \frac{1}{1+(\omega RC)^2}$ If the input is white noise with $S_X(\omega) = N_0$: - **Output PSD:** $S_Y(\omega) = \frac{N_0}{1+(\omega RC)^2}$ - **Output Average Power:** $E[Y^2(t)] = \frac{N_0}{2\pi} \int_{-\infty}^{\infty} \frac{1}{1+(\omega RC)^2} d\omega$ Let $u = \omega RC$, $du = RC d\omega$. $E[Y^2(t)] = \frac{N_0}{2\pi RC} \int_{-\infty}^{\infty} \frac{1}{1+u^2} du = \frac{N_0}{2\pi RC} [\arctan(u)]_{-\infty}^{\infty}$ $E[Y^2(t)] = \frac{N_0}{2\pi RC} (\frac{\pi}{2} - (-\frac{\pi}{2})) = \frac{N_0}{2\pi RC} (\pi) = \frac{N_0}{2RC}$ This formula gives the output noise power for an RC filter. #### Band-Limited White Noise Often, noise is approximated as white noise over a certain bandwidth $B$. If $S_X(\omega) = N_0$ for $|\omega| \le B$ and $0$ otherwise, then the output PSD and power can be calculated by integrating over the relevant bandwidth.