Matrices & Linear Systems

Cheatsheet Content

### Matrix Review - **Definition:** A system of $mn$ numbers (real or complex) arranged in $m$ rows and $n$ columns. $$A = [a_{ij}]_{m \times n} \text{ where } 1 \le i \le m, 1 \le j \le n$$ - **Elements:** Each of the $mn$ numbers is an element of the matrix. - **Scalars:** Numbers in relation to a matrix are called scalars. ### Types of Matrices #### Square Matrix - If $A = [a_{ij}]_{m \times n}$ and $m = n$, $A$ is a square matrix of order $n$. Example: $\begin{pmatrix} 1 & 1 \\ 2 & 2 \end{pmatrix}$ is a 2nd order matrix. #### Rectangular Matrix - A matrix that is not a square matrix ($m \ne n$). Example: $\begin{pmatrix} 1 & -1 & 2 \\ 2 & 3 & 4 \end{pmatrix}$ is a $2 \times 3$ matrix. #### Row Matrix - A matrix of order $1 \times m$. Also called a Row vector. Example: $[1 \ 2 \ 3]_{1 \times 3}$ #### Column Matrix - A matrix of order $n \times 1$. Also called a Column vector. Example: $\begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}_{3 \times 1}$ #### Unit Matrix (Identity Matrix) - If $A = [a_{ij}]_{n \times n}$ such that $a_{ij} = 1$ for $i = j$ and $a_{ij} = 0$ for $i \ne j$, then $A$ is called a Unit Matrix, denoted by $I_n$. Example: $I_2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$ #### Zero Matrix (Null Matrix) - If $A = [a_{ij}]_{m \times n}$ such that $a_{ij} = 0$ for all $i$ and $j$, then $A$ is called a Zero matrix or Null matrix, denoted by $O$. Example: $O_{2 \times 3} = \begin{pmatrix} 0 & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix}$ #### Diagonal Elements & Principal Diagonal - For a square matrix $A = [a_{ij}]_{n \times n}$, elements $a_{ij}$ where $i = j$ (i.e., $a_{11}, a_{22}, ..., a_{nn}$) are called the diagonal elements. The line containing these elements is the principal diagonal. #### Diagonal Matrix - A square matrix where all elements except those on the leading diagonal are zero. If $d_1, d_2, ..., d_n$ are diagonal elements, $A = \text{diag}(d_1, d_2, ..., d_n)$. Example: $A = \text{diag}(3, 1, -2) = \begin{pmatrix} 3 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & -2 \end{pmatrix}$ #### Scalar Matrix - A diagonal matrix whose leading diagonal elements are equal. Example: $B = \begin{pmatrix} 3 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 3 \end{pmatrix}$ #### Equal Matrices - Two matrices $A = [a_{ij}]$ and $B = [b_{ij}]$ are equal if they are of the same type/order and $a_{ij} = b_{ij}$ for every $i$ and $j$. ### Algebra of Matrices #### Addition of Two Matrices - If $A = [a_{ij}]_{m \times n}$ and $B = [b_{ij}]_{m \times n}$ are two matrices, their sum $C = A + B = [c_{ij}]_{m \times n}$ where $c_{ij} = a_{ij} + b_{ij}$. - Thus $[a_{ij}]_{m \times n} + [b_{ij}]_{m \times n} = [a_{ij} + b_{ij}]_{m \times n}$. #### Difference of Two Matrices - If $A, B$ are two matrices of the same type, $A - B$ is defined as $A + (-B)$. #### Multiplication by a Scalar - The product of a matrix $A$ by a scalar $K$ is $KA = [Ka_{ij}]_{m \times n}$. - **Properties:** - $OA = O$ (null matrix) - $(-1)A = -A$ (negative of A) - $K_1(K_2 A) = (K_1 K_2) A = K_2(K_1 A)$ #### Matrix Multiplication - If $A = [a_{ik}]_{m \times n}$ and $B = [b_{kj}]_{n \times p}$, their product $C = AB = [c_{ij}]_{m \times p}$ where $c_{ij} = \sum_{k=1}^{n} a_{ik}b_{kj}$. - $A$ is the pre-factor, $B$ is the post-factor. - Matrices are conformable for multiplication if the number of columns of $A$ equals the number of rows of $B$. - **Associativity:** $(AB)C = A(BC)$. #### Positive Integral Powers of Square Matrices - For a square matrix $A$: - $A^2 = A \cdot A$ - $A^m = A \cdot A^{m-1}$ - $A^m A^n = A^{m+n}$ - $(A^m)^n = A^{mn}$ - Note: $I^m = I$, $O^m = O$. ### Matrix Theorems #### Distributive Property - Matrix multiplication is distributive w.r.t. addition: - $A(B + C) = AB + AC$ - $(B + C)A = BA + CA$ - Note: $A(B - C) = AB - AC$ and $(B - C)A = BA - CA$. #### Identity Matrix Property - If $A$ is a matrix of order $m \times n$, then $AI_n = I_m A = A$. ### Trace of a Square Matrix - For a square matrix $A = [a_{ij}]_{n \times n}$, the trace is defined as $\text{tr}(A) = \sum_{i=1}^{n} a_{ii} = a_{11} + a_{22} + ... + a_{nn}$. - **Properties:** - $\text{tr}(\lambda A) = \lambda \text{tr}(A)$ - $\text{tr}(A + B) = \text{tr}(A) + \text{tr}(B)$ - $\text{tr}(AB) = \text{tr}(BA)$ ### Triangular Matrix - **Upper Triangular:** A square matrix where all elements below the leading diagonal are zero. Example: $\begin{pmatrix} 1 & 2 & -3 & 0 \\ 0 & 4 & 2 & 1 \\ 0 & 0 & -6 & 2 \\ 0 & 0 & 0 & 8 \end{pmatrix}$ - **Lower Triangular:** A square matrix where all elements above the leading diagonal are zero. Example: $\begin{pmatrix} 7 & 0 & 0 & 0 & 0 \\ 5 & 3 & 0 & 0 & 0 \\ -4 & 6 & 0 & 0 & 0 \\ 2 & 1 & -8 & 5 & 0 \\ 2 & 0 & 4 & 1 & 6 \end{pmatrix}$ ### Special Square Matrices #### Idempotent Matrix - A square matrix $A$ such that $A^2 = A$. #### Nilpotent Matrix - A square matrix $A$ such that $A^m = O$ for some positive integer $m$. The least positive integer $m$ is the index of nilpotency. #### Involutory Matrix - A square matrix $A$ such that $A^2 = I$. ### Transpose of a Matrix - The matrix obtained by interchanging rows and columns of $A$, denoted by $A'$ or $A^T$. - If $A = [a_{ij}]_{m \times n}$, then $A^T = [b_{ji}]_{n \times m}$ where $b_{ji} = a_{ij}$. - **Properties:** - $(A^T)^T = A$ - $(A + B)^T = A^T + B^T$ - $(KA)^T = KA^T$ - $(AB)^T = B^T A^T$ ### Determinants #### Minors and Cofactors - For a square matrix $A = [a_{ij}]_{n \times n}$: - **Minor:** The determinant of the $(n-1)$ rowed matrix obtained by deleting the $i$-th row and $j$-th column is the minor of $a_{ij}$, denoted by $|M_{ij}|$. - **Cofactor:** The signed minor $(-1)^{i+j}|M_{ij}|$ is the cofactor of $a_{ij}$, denoted by $A_{ij}$. - The determinant of $A$ can be defined as $|A| = \sum_{j=1}^{n} a_{ij}A_{ij}$ (expansion along $i$-th row) or $|A| = \sum_{i=1}^{n} a_{ij}A_{ij}$ (expansion along $j$-th column). - **Properties:** - $|KA| = k^n|A|$ for a scalar $k$ and $n$-order matrix $A$. - $|A| = |A^T|$. - $|AB| = |A||B|$. #### Adjoint of a Square Matrix - The transpose of the matrix formed by replacing each element of $A$ with its cofactor is called the adjoint of $A$, denoted by $\text{adj} A$. - Note: $\text{adj}(kA) = k^{n-1} \text{adj} A$. #### Singular and Non-Singular Matrices - **Singular:** A square matrix $A$ is singular if $|A| = 0$. - **Non-Singular:** A square matrix $A$ is non-singular if $|A| \ne 0$. - Only non-singular matrices possess inverses. - The product of non-singular matrices is also non-singular. ### Inverse of a Matrix #### Definition - For a square matrix $A$, if a matrix $B$ exists such that $AB = BA = I$, then $B$ is the inverse of $A$, denoted by $A^{-1}$. - For $AB$ and $BA$ to be defined and equal, $A$ and $B$ must be square matrices of the same order. Non-square matrices do not have inverses. #### Invertible Matrix - A matrix that possesses an inverse. #### Properties of Inverse - Every invertible matrix has a unique inverse. - The inverse of $I_n$ is $I_n$. - If $A$ is an invertible matrix and $A = B$, then $A^{-1} = B^{-1}$. - A square matrix possesses an inverse if and only if $|A| \ne 0$. - If $|A| \ne 0$, then $A^{-1} = \frac{1}{|A|} (\text{adj} A)$. - If $A$ is an $m \times n$ matrix and $B$ is an $n \times p$ matrix, then $(AB)^T = B^T A^T$. - If $A, B$ are invertible matrices of the same order, then: - $(AB)^{-1} = B^{-1}A^{-1}$ - $(A^T)^{-1} = (A^{-1})^T$ - $(B^{-1}A^{-1})^{-1} = AB$. - $(A \cdot B \cdot ... \cdot Z)^{-1} = Z^{-1} \cdot ... \cdot B^{-1} \cdot A^{-1}$. ### Cramer's Rule (Determinant Method) - For a system of linear equations: $a_1 x + b_1 y + c_1 z = d_1$ $a_2 x + b_2 y + c_2 z = d_2$ $a_3 x + b_3 y + c_3 z = d_3$ - The solution is given by: $$x = \frac{\Delta_1}{\Delta}, \quad y = \frac{\Delta_2}{\Delta}, \quad z = \frac{\Delta_3}{\Delta} \quad (\text{if } \Delta \ne 0)$$ - Where $\Delta = \begin{vmatrix} a_1 & b_1 & c_1 \\ a_2 & b_2 & c_2 \\ a_3 & b_3 & c_3 \end{vmatrix}$ and $\Delta_1, \Delta_2, \Delta_3$ are obtained by replacing the first, second, and third columns of $\Delta$ with $d_1, d_2, d_3$ respectively. ### Symmetric Matrix - A square matrix $A = [a_{ij}]$ is symmetric if $a_{ij} = a_{ji}$ for every $i$ and $j$. - Thus $A = A^T$ or $A^T = A$. ### Skew-Symmetric Matrix - A square matrix $A = [a_{ij}]$ is skew-symmetric if $a_{ij} = -a_{ji}$ for every $i$ and $j$. - Thus $A = -A^T$ or $A^T = -A$. - Note: Every diagonal element of a skew-symmetric matrix is necessarily zero since $a_{ii} = -a_{ii} \implies 2a_{ii} = 0 \implies a_{ii} = 0$. - **Properties:** - If $A$ is symmetric, $KA$ is symmetric. - If $A$ is skew-symmetric, $KA$ is skew-symmetric. ### Orthogonal Matrix - A square matrix $A$ is orthogonal if $AA^T = A^T A = I$. That is $A^T = A^{-1}$. - **Theorem:** Any square matrix $A$ can be expressed as the sum of a symmetric and a skew-symmetric matrix uniquely. $$A = \frac{1}{2}(A + A^T) + \frac{1}{2}(A - A^T)$$ Where $P = \frac{1}{2}(A + A^T)$ is symmetric and $Q = \frac{1}{2}(A - A^T)$ is skew-symmetric. - **Theorem:** The inverse of a non-singular symmetric matrix $A$ is symmetric. - **Theorem:** If $A$ is a symmetric matrix, then $\text{adj} A$ is also symmetric. - **Theorem:** If $A, B$ are orthogonal matrices of order $n$, then $AB$ and $BA$ are orthogonal matrices. - **Theorem:** The inverse of an orthogonal matrix is orthogonal, and its transpose is also orthogonal. - If $A$ is orthogonal, then $|A| = \pm 1$. ### Complex Matrices #### Conjugate of a Matrix - The matrix obtained by replacing each element of $A$ with its corresponding conjugate complex number, denoted by $\bar{A}$. - If $A = [a_{ij}]_{m \times n}$, then $\bar{A} = [\bar{a}_{ij}]_{m \times n}$. - **Properties:** - $(\bar{\bar{A}}) = A$ - $(\overline{A+B}) = \bar{A} + \bar{B}$ - $(\overline{KA}) = \bar{K}\bar{A}$ for a complex scalar $K$. - $(\overline{AB}) = \bar{A}\bar{B}$ #### Transpose of the Conjugate (Conjugate Transpose) - If $A$ is a square matrix, its conjugate transpose is $(\bar{A})^T$, also denoted as $A^*$, $A^H$, or $A^\dagger$. - $(\bar{A})^T = (\overline{A^T})$. - This is denoted by $A^\theta$. So $A^\theta = (\bar{A})^T = (\overline{A^T})$. - **Properties:** - $(A^\theta)^\theta = A$ - $(A + B)^\theta = A^\theta + B^\theta$ - $(KA)^\theta = \bar{K}A^\theta$ - $(AB)^\theta = B^\theta A^\theta$ #### Hermitian Matrix - A square matrix $A$ is Hermitian if $A^\theta = A$. - Elements on the principal diagonal of a Hermitian matrix must be real. - A Hermitian matrix over real numbers is a real symmetric matrix. #### Skew-Hermitian Matrix - A square matrix $A$ is Skew-Hermitian if $A^\theta = -A$. - Elements on the leading diagonal must be zero or purely imaginary. - A Skew-Hermitian matrix over real numbers is a real skew-symmetric matrix. #### Unitary Matrix - A square matrix $A$ is unitary if $A^\theta A = A A^\theta = I$, meaning $A^\theta = A^{-1}$. - Every real symmetric matrix is Hermitian. - Every real skew-symmetric matrix is Skew-Hermitian. - Every real orthogonal matrix is Unitary. ### Elementary Transformations (Operations) on a Matrix - **Interchange of rows:** $R_i \leftrightarrow R_j$. - **Scalar multiplication of a row:** $R_i \rightarrow kR_i$ (where $k \ne 0$). - **Adding a scalar multiple of one row to another:** $R_j \rightarrow R_j + kR_i$. - Similar operations apply to columns ($C_i \leftrightarrow C_j$, $C_i \rightarrow kC_i$, $C_j \rightarrow C_j + kC_i$). - An elementary transformation is either a row transformation or a column transformation. - Elementary operations do not change the rank of a matrix. ### Equivalence of Matrices - Matrix $B$ is equivalent to $A$ (denoted $B \sim A$) if $B$ is obtained from $A$ by a finite chain of elementary transformations. - **Results:** - If $A \sim B$, then $\text{rank}(A) = \text{rank}(B)$. - If $A$ and $B$ have the same size and rank, then $A$ and $B$ are equivalent. ### Sub-Matrix - A matrix obtained by deleting some rows or columns (or both) from a given matrix. ### Minor of a Matrix - The determinant of a square sub-matrix of $A$. If the order of the square sub-matrix is $t$, it is a minor of order $t$. ### Rank of a Matrix - For an $m \times n$ matrix $A$: - If $A$ is a null matrix, $\text{rank}(A) = 0$. - If $A$ is non-zero, its rank $r$ is the largest order of any non-zero minor. - Formally, rank $r$ means: - Every $(r+1)$-th order minor of $A$ is $0$. - There exists at least one $r$-th order minor of $A$ which is not $0$. - Rank is denoted by $\rho(A)$. - **Notes:** - The rank of a non-zero matrix is the order of its highest order non-zero minor. - Every matrix has a rank. - Rank of a matrix is unique. - $\rho(A) \ge 1$ if $A$ is non-zero. - $\rho(A) \le \min(m, n)$. - If $\rho(A) = r$, then every minor of $A$ of order $r+1$ or more is zero. - Rank of the identity matrix $I_n$ is $n$. - If $A$ is an $n$-order non-singular matrix (i.e., $\det A \ne 0$), then $\rho(A) = n$. - **Important Results for determining rank:** - The rank of a matrix is $\le r$ if all minors of order $(r+1)$ vanish. - The rank of a matrix is $\ge r$ if there is at least one minor of order $r$ which is not zero. ### Augmented Matrix and Linear Systems - A system of linear equations can be written in matrix form $AX=B$. - The augmented matrix is $[A | B]$, formed by appending the column vector $B$ to matrix $A$. - Solving linear systems often involves elementary row operations on the augmented matrix. ### Zero Row and Non-Zero Row - **Zero Row:** A row where all elements are zeros. - **Non-Zero Row:** A row with at least one non-zero element. ### Echelon Form of a Matrix - A matrix is in Echelon form if it satisfies: 1. Zero rows (if any) are below all non-zero rows. 2. The first non-zero entry in each non-zero row is $1$. (Optional, but often included for Reduced Row Echelon Form). 3. The number of zeros before the first non-zero element in a row is less than the number of such zeros in the next row. - **Important Result:** The number of non-zero rows in the row echelon form of $A$ is the rank of $A$. - The rank of the transpose of a matrix is the same as that of the original matrix. ### Reduction to Normal Form - Every $m \times n$ matrix of rank $r$ can be reduced to one of the following normal forms by elementary row or column operations: $$I_r, \quad \begin{pmatrix} I_r & O \end{pmatrix}, \quad \begin{pmatrix} I_r \\ O \end{pmatrix}, \quad \begin{pmatrix} I_r & O \\ O & O \end{pmatrix}$$ where $I_r$ is the $r$-rowed unit matrix. This is also called the "first canonical form". - **Corollary 1:** The rank of an $m \times n$ matrix $A$ is $r$ if and only if it can be reduced to one of the normal forms. - **Corollary 2:** If $A$ is an $m \times n$ matrix of rank $r$, there exist non-singular matrices $P$ and $Q$ such that $PAQ = \begin{pmatrix} I_r & O \\ O & O \end{pmatrix}$. ### Elementary Matrix - A matrix obtained from a unit matrix by a single elementary transformation. - **Theorem:** Every elementary row (column) transformation of a matrix can be obtained by pre-multiplication (post-multiplication) with a corresponding elementary matrix. ### Inverse by Elementary Transformations (Gauss-Jordan Method) - To find the inverse of a non-singular square matrix $A$ of order $n$: 1. Write $A = I_n A$. 2. Apply a sequence of elementary row operations to $A$ (on the left side) and $I_n$ (on the right side) until $A$ is transformed into $I_n$. 3. The matrix that $I_n$ (on the right side) transforms into will be $A^{-1}$, i.e., $I_n = BA$, so $B = A^{-1}$. - Alternatively, apply elementary row operations to the augmented matrix $[A | I]$ until it becomes $[I | B]$, then $B = A^{-1}$.