# Linear Algebra: Results Summary

1. Definitions

1.1. Vector Spaces

• Let ${V \neq \phi}$ be a set, and ${F}$ be a field. We say ${V}$ is a vector space if there exists two binary operations ${+: V \times V \rightarrow V}$ and ${\cdot: F \times V \rightarrow V}$, under which ${V}$ is closed and satisfies:
1. ${(V,+)}$ is a commutative group
2. If ${\lambda, \mu \in F, x \in V}$, then ${(\lambda \mu) \cdot x = \lambda \cdot (\mu x)}$
3. ${\lambda \cdot (x+y) = \lambda \cdot x + \lambda \cdot y}$
${(\lambda + \mu)\cdot x = \lambda \cdot x + \mu \cdot x}$

4. ${1 \cdot x = x}$, where ${1}$ is the identity of ${F}$
• Subspaces are subsets of ${V}$, which are closed under vector addition and scalar multiplication
• If ${S'}$ be the smallest subspace of ${V}$ containing ${S}$, ${S'}$ is called the subspace generated by ${S}$
• It can be shown that, ${S' = [S]}$
• finite dimensional: ${V}$ is finite dimensional over ${F}$ if ${\exists}$ a finite set ${S \subset V}$, s.t. ${V = [S]}$. Otherwise, ${V}$ is infinite dimensional.
• basis: ${S}$ is a basis of ${V}$ over ${F}$ if ${V = [S]}$, and no proper subset of ${S}$ spans ${V}$
• LI: ${\{x_1, \ldots, x_n\}}$ is LI iff

$\displaystyle \sum_{j=1}^n \lambda_j x_j = 0 \Rightarrow \lambda_j = 0 \quad \forall 1 \leq j \leq n$

${S}$ is LI if every finite subset of ${S}$ is LI

• Complementary subspace: Let ${W}$ be a subspace of ${V}$. We say a subspace ${\overline{W}}$ of ${V}$ complementary of ${W}$ if
1. ${W+\overline{W}=V}$
2. ${W \cap \overline{W} = \{0\}}$

1.2. Quotient Space

• ${V}$ is a vector space over ${F}$ and ${W}$ is a subspace of ${V}$. Quotient space

$\displaystyle V/W = \{x + W : x \in V\}$

$\displaystyle (x + W) +_{V/W} (y+W) = (x+y) + W$

multiplication:

$\displaystyle \lambda \cdot_{V/W} (x+W) = \lambda x + W$

${V/W}$ can be shown to be a vector space

1.3. Linear Mappings

• Let ${V_1, V_2}$ be two vector spaces over same field ${F}$. A mapping ${T:V_1 \rightarrow V_2}$ is linear if

$\displaystyle T(\lambda x + \mu y) = \lambda T(x) + \mu T(y) \qquad \lambda, \mu \in F, x,y \in V_1$

• ${T}$ is isomorphism if it is one-one, onto and linear
• Let ${V}$ be a vector space over ${F}$. ${{\mathcal L}(V,F) \triangleq V^{*}}$ is called the dual space of ${V}$
• Given a basis ${\{v_i\}_{i=1}^n}$ of ${V}$, we can derive a basis of the dual space as ${\Lambda_{v_j}(v_k)= \delta_{jk}}$
• For any ${v\in V}$, define, ${f_v : V^* \rightarrow F, f_v(\Lambda)=\Lambda (v), \Lambda \in V^*}$. Set of all such ${f_v}$‘s form the dual of ${V^*}$ and is denoted by ${V^{**}}$
• Define ${\psi : V \rightarrow V^{**}, \psi(v)=f_v}$
• ${\psi}$ is linear, one-one, in addition onto for finite dimensional vector spaces.
• Annihilator: Let ${V}$ be a vector space and ${S \subset V, S \neq \phi}$, annihilator of ${S}$ is defined as,

$\displaystyle S^a = \{\Lambda \in V^* : \Lambda(x)=0_F, \forall x \in S\}$

• Let ${T: V \rightarrow W}$, linear. Let ${\{v_i\}_{i=1}^n}$ and ${\{w_i\}_{i=1}^m}$ be two bases respectively.

$\displaystyle \begin{array}{rcl} T(v_j) &=& \sum_{k=1}^m \alpha_{kj}w_k \qquad 1 \leq j \leq n \\ T(v_1)&=& \alpha_{11}w_1+\cdots+\alpha_{m1}w_m \\ \vdots &=& \vdots\\ T(v_n) &=& \alpha_{n1}w_1+\cdots+\alpha_{mn}w_m \end{array}$

The matrix associated with this transformation is,

$\displaystyle \begin{array}{rcl} \begin{bmatrix} \alpha_{11} & \cdots & \alpha_{1n} \\ \alpha_{21} & \cdots & \alpha_{2n} \\ \vdots & & \vdots \\ \alpha_{m1} & \cdots & \alpha_{mn} \end{bmatrix} \end{array}$

• Given ${T \in {\mathcal L}(V,W)}$, we associate, ${T^t : W^* \rightarrow V^*}$ given by

$\displaystyle (T^t\Lambda)(v)=\Lambda(Tv) \qquad \forall v \in V$

1.4. Inner Product Spaces

• An orthonormal set ${\{v_i\}_{i=1}^n}$ in an inner product space ${V}$ is said to be complete, if this can’t be properly included in any other orthonormal set.
• For a given linear transformation ${T: V \rightarrow V}$, the transformation ${T^*: V \rightarrow V}$ is called the adjoint of ${T}$ if ${\langle Tx,y\rangle =\langle x,T^*y\rangle , \quad \forall x,y \in V}$
• We say ${T \in {\cal L}(V)}$ is a positive operator if ${T=T^*}$ and ${\langle Tx,x\rangle \geq 0 \quad \forall x \in V}$ (non-negative semidefinite). Further, ${T}$ is strictly positive if ${T}$ is positive and ${\langle Tx,x\rangle > 0 \quad \forall x \neq 0}$
• Let ${V}$ be a unitary space (complex inner product), then ${T \in {\cal L}(V)}$ which preserves inner product is called unitary transformation. If ${V}$ is Eucliedean (real inner product), ${T}$ is orthogonal transformation
• Let ${V}$ be an Eucliedean space and ${h:V \rightarrow V}$. We say ${h}$ is a rigid motion if ${||hx-hy||=||x-y|| \quad \forall x,y \in V}$. A special kind of rigid motion is translation, ${T_v(u) = v+u}$ is called translation by ${v}$. e.g., any unitary transformation ${S}$ is a rigid motion.

2. Theorems

2.1. Vector Spaces

• Every field is a vector space over itself
• Let ${V}$ be a finite dimensional vector space over ${F}$, then ${V}$ has a basis
• If ${\{v_i\}_{i=1}^n}$ span ${V}$ and are LI, then it is a basis
• If ${\{v_i\}_{i=1}^n}$ span ${V}$ and ${dim(V)=n}$, then it is a basis
• If ${V}$ is a finite dimensional vector space having a basis consisting of ${n}$ elements, any LI set of ${n}$ vectors in ${V}$ is a basis of ${V}$
• Let ${dim(V)< \infty}$. Then any two bases of ${V}$ have same number of elements
• If ${W}$ is a subspace of a finite dimensional vector space ${V}$, ${dim(W) \leq dim(V)}$, and equality iff ${W=V}$
• For a finite dimensional vector space ${V}$, the following statements are equivalent
1. ${\{v_i\}_{i=1}^n}$ is a basis of ${V}$
2. ${\{v_i\}_{i=1}^n}$ is maximal linear independent set
3. ${\{v_i\}_{i=1}^n}$ is minimal generating set
• Subspaces ${W}$ and ${\overline{W}}$ are complementary iff each ${x \in V}$ can be uniquely written as ${x=a+b, a \in W, b \in \overline{W}}$. We say ${V=W \oplus \overline{W}}$, direct sum
• If ${W}$ is a subspace of a finite dimensional vector space ${V}$, then ${W}$ has a complementary subspace
• Theorem 1 Let ${W_1}$ and ${W_2}$ be two subspaces of a finite dimensional vector space ${V}$, then

$\displaystyle dim(W_1+W_2)=dim(W_1)+dim(W_2)-dim(W_1 \cap W_2)$

2.2. Quotient Space

• Theorem 2 Let ${V}$ be a finite dimensional vector space over ${F}$ and ${W}$ is a subspace of ${V}$. Then

$\displaystyle dim(V/W) = dim(V)-dim(W)$

2.3. Linear Mappings

• For any linear transformation ${T: V \rightarrow W }$, the vector spaces ${V}$ and ${W}$ are on same scalar field ${F}$
• Let ${T: V \rightarrow W }$ be linear and ${S \subset V}$ s.t. ${V=[S]}$, then ${{\Re}(T)=[T(S)]}$
• Let ${dim(V)=n}$, ${T: V \rightarrow W }$ be linear, then ${dim({\Re}(T)) \leq n}$
• If ${\{x_1,\cdots,x_k\}}$ are LI in ${V}$ and ${T}$ one-one, then ${\{Tx_1,\cdots,Tx_k\}}$ are LI in ${W}$
• Let ${dim(V)=n}$, then ${dim({\Re}(T))=n \Leftrightarrow T}$ is one-one
• Theorem 3 : Let ${V}$ and ${W}$ be vector spaces. Let ${\{v_i\}_{i=1}^n}$ is a basis of ${V}$. Let ${w_i, 1 \leq i \leq n}$ be any set of (not necessarily distinct) vectors in ${W}$. Then ${\exists}$ a unique linear map ${T: V \rightarrow W }$ such that ${T(v_i)=w_i}$

• Theorem 4 (Rank-Nullity Theorem) : Let ${V}$ be a finite dimensional vector space and ${T: V \rightarrow W }$, linear, then

$\displaystyle \begin{array}{rcl} dim(V) &=& dim(Ker(T))+ dim({\Re}(T))\end{array}$

• If ${T: V \rightarrow W }$ be an isomorphism, so is ${T^{-1}: W \rightarrow V }$
• ${T: V \rightarrow W }$, ${V}$ and ${W}$ finite dimensional and ${dim V = dim W}$, then ${T}$ is 1-1 ${\Leftrightarrow T}$ is onto
• Theorem 5 (Fundamental Theorem of Homomorphism) : Let ${V}$ and ${W}$ be vector spaces over ${F}$. ${T: V \rightarrow W }$ be a surjective linear map. Then, ${V/Ker(T) \cong W}$

• Let ${V}$ and ${W}$ be vector spaces. Then the dimension of ${{\mathcal L}(V,W)}$ is ${dim V \times dim W}$
• If ${S_1 \subset S_2}$ then ${S_2^a \subset S_1^a}$
• If ${dim(V)< \infty}$ and ${S}$ be a subspace of ${V}$, then

$\displaystyle dim(V)=dim(S)+dim(S^a)$

(The proof is useful for many similar proofs)

• Solution of homogeneous linear equations: A homogeneous linear system of equations ${Ax=0, A \in M_{m \times n}(F), x \in F^n}$ has solution iff ${\Lambda_x}$ is in the annihilator of the row space of ${A}$. ${\Lambda_x(y)=x^T y}$
• Solution of non-homogeneous linear equations: Let ${M=\{\alpha_1,\cdots,\alpha_m\}}$ be a set of ${m}$ vectors in ${F^n}$ and ${b_1,\cdots, b_m}$ be ${m}$ scalars. Let

$\displaystyle \begin{array}{rcl} \alpha_j &=& (a_{j1},\cdots,a_{jn}) \in F^n \qquad 1 \leq j \leq m \\ \alpha_j' &=& (a_{j1},\cdots,a_{jn},b_j) \in F^{n+1} \qquad 1 \leq j \leq m \\ M &=& \{\alpha_1,\cdots,\alpha_m\} \\ M' &=& \{\alpha_1',\cdots,\alpha_m'\} \end{array}$

then ${\exists \Lambda \in (F^n)^*}$ such that ${\Lambda(\alpha_j)=b_j, 1 \leq j \leq m}$ iff ${dim[M]=dim[M']}$

In other words, non-homogeneous system of equation has a solution iff the row rank of the coefficient matrix is equal to the row rank of its augmented matrix.

• If ${A}$ is the matrix associated with ${T}$, that with ${T^t}$ will be ${A^T}$

2.4. Inner Product Spaces

• Useful inequalities
Cauchy-Schwartz: ${|\langle x,y\rangle| \leq \sqrt{\langle x,x\rangle}.\sqrt{\langle y,y\rangle}}$
Hölder’s: Let ${p>1}$ real and ${q}$ be s.t. ${\frac{1}{p}+\frac{1}{q}=1}$

$\displaystyle \left|\sum_{j=1}^n x_j y_j\right| \leq \left( \sum_{j=1}^n |x_j|^p\right)^{\frac{1}{p}}.\left( \sum_{j=1}^n |y_j|^q\right)^{\frac{1}{q}}$

This is a generalization for Cauchy-Schwartz in ordinary real inner products
Minkowski’s: Let ${p>1}$ real

$\displaystyle ||x+y||_p \leq ||x||_p + ||y||_p$

${||.||_p : L_p}$ norm on ${\mathbb{R}^n}$

• Theorem 6 Let ${S}$ be an orthogonal set not containing the zero element, then ${S}$ is LI

• Theorem 7 Let ${V}$ be an inner product space and ${\{v_i\}_{i=1}^k}$ be an LI set. Then ${\exists}$ a set ${\{w_i\}_{i=1}^k}$ in ${V}$ s.t.

1. ${\{w_i\}_{i=1}^k}$ is orthonormal
2. for each ${l, 1 \leq l \leq k}$, ${[v_1 \dots v_l]=[w_1 \dots w_l]}$

The proof of this uses the Gram-Schmidt orthogonalization procedure, which is useful for extracting orthogonal basis from LI. The orthogonal set of vectors can be written as,

$\displaystyle u_n = v_n - \sum_{j=1}^{n-1} \frac{\langle v_n, u_j \rangle}{\langle u_j, u_j\rangle} u_j \quad n = 1, \dots, k$

Further we normalize it to get the orthonormal set ${\{w_i\}_{i=1}^k}$.

• Theorem 8 Let ${\{v_i\}_{i=1}^n}$ be an orthonormal basis for an inner product space ${V}$, then for any ${v \in V}$

$\displaystyle v = \sum_{j=1}^n \langle v,v_j \rangle v_j$

for any ${v,w \in V}$

$\displaystyle \langle v,w \rangle =\sum_{j=1}^n \langle v,v_j\rangle \langle v_j,w\rangle$

• Theorem 9 Let ${\{v_i\}_{i=1}^n}$ be a set in ${V}$, s.t. ${\langle v_j,v_j\rangle = 1, 1 \leq j \leq n}$. Suppose ${\forall v \in V}$

$\displaystyle \langle v,v\rangle = \sum_{j=1}^n |\langle v,v_j\rangle |^2$

then ${\{v_i\}_{i=1}^n}$ is an orthonormal basis

• Theorem 10 (Bessel’s inequality) Let ${\{v_i\}_{i=1}^n}$ be an orthonormal set in an inner product space ${V}$, then for any ${w \in V}$

$\displaystyle \sum_{j=1}^n |\langle w,v_j\rangle |^2 \leq \|w\|^2$

• Let ${\{v_i\}_{i=1}^n}$ be an orthonormal set in an inner product space ${V}$, then the following are equivalent:
1. ${\{v_i\}_{i=1}^n}$ is complete
2. ${\langle v,v_j\rangle = 0}$ for ${1 \leq j \leq n \Rightarrow v = 0}$
3. ${V = [v_1 \dots v_n]}$
4. For each ${v \in V}$, ${v = \sum_{j=1}^n \langle v,v_j\rangle v_j}$
5. For any ${v,w \in V}$

$\displaystyle \langle v,w\rangle =\sum_{j=1}^n \langle v,v_j\rangle \langle v_j,w\rangle \qquad \mbox{ (Parseval's identity)}$

6. For any ${v \in V}$

$\displaystyle \|v\|^2 = \sum_{j=1}^n |\langle v,v_j\rangle |^2$

• Theorem 11 Let ${S}$ be a finite dimensional subspace of an inner product space ${V}$, then ${V = S \oplus S^{\bot}}$. ${S^{\bot}}$ is the orthogonal complement of ${S}$

• Theorem 12 (Pythagoras) Let ${S}$ be a finite dimensional subspace of an inner product space ${V}$, then for each ${v \in V}$,

$\displaystyle v = v_S + v_{S^{\bot}} \quad v_S \in S, v_{S^{\bot}} \in S^{\bot}$

1. ${\|v\|^2 = \|v_S\|^2 + \|v_{S^{\bot}}\|^2}$
2. ${\|v-v_S\| \leq \|v-w\|, \forall w \in S}$; Best approximation

• Theorem 13 (Riesz Representation Theorem) If ${V}$ be a finite dimensional inner product space and ${\Lambda \in V^*}$, then ${\exists}$ a unique ${y \in V}$ such that ${\Lambda(x)=\langle x,y\rangle , \quad \forall x \in V}$. So, we write ${\Lambda \equiv \Lambda_y}$. ${\Lambda_y(.) = \langle .,y\rangle }$. This way every ${\Lambda \in V^*}$ can be uniquely identified by a ${y \in V}$

• The mapping ${T: V \rightarrow V^*}$ given by ${T(y)=\Lambda_y}$ can be shown to be one-one, onto and conjugate linear.
• For polynomial inner product space, ${V = \mathbb{C}[x]}$, with inner product ${\langle p,q\rangle = \int_0^1 p(x)\overline{q(x)}dx}$, if the functional is defined as a evaluation of the polynomial at some point ${z_0}$, i.e., ${\Lambda(p)=p(z_0)}$, then the only possible functional is the zero functional, i.e., ${\Lambda \equiv \Lambda_0}$
• Theorem 14 Let ${T \in {\cal L}(V)}$ and suppose ${T^*}$ exists, then

1. ${Ker T^* = \Re (T)^{\bot}}$
2. If ${dim V < \infty}$, ${T}$ and ${T^*}$ has same rank

• Some properties of the adjoint transformation:

$\displaystyle \begin{array}{rcl} (T_1+T_2)^* &=& T_1^* + T_2^* \\ (\alpha T)^* &=& \overline{\alpha} T^* \\ (T^*)^* &=& T \qquad \mbox{ (involution)} \\ (T_1T_2)^* &=& T_2^* T_1^* \end{array}$

• Proposition 15 ${T \in {\cal L}(V)}$ is self-adjoint iff the matrix of ${T}$ wrt some orthonormal basis of ${V}$ is self-adjoint (Hermitian)

• Proposition 16 If ${V}$ is a complex inner product space and ${T \in {\cal L}(V)}$, (${T}$ not necessarily self-adjoint), then

$\displaystyle T \equiv 0 \Leftrightarrow \langle Tx,x\rangle= 0 \quad \forall x \in V$

This is a special property for complex, if ${T=T^*}$, this is true for any inner product

• Proposition 17 Let ${V}$ be a complex inner product space and ${T \in {\cal L}(V)}$, then

$\displaystyle T=T^* \Leftrightarrow \langle Tx,x\rangle \mbox{ is real } \forall x \in V$

• Proposition 18 Let ${V}$ be an inner product space and ${T \in {\cal L}(V)}$, strictly positive, ${dim(V) < \infty}$ then ${T}$ is invertible

• Proposition 19 Let ${V}$ be an inner product space with ${dim(V) < \infty}$, ${P \in {\cal L}(V)}$ is an orthogonal projection iff ${P=P^2=P^*}$

• Every orthogonal projection is a positive linear transformation
• Theorem 20 Let ${T \in {\cal L}(V)}$ preserves inner product, then

1. ${T}$ preserves norm (isometric)
2. ${T}$ is linear
3. ${T}$ is one-one
4. If ${dim(V) < \infty}$, then ${T}$ is invertible

• Proposition 21 ${T}$ is a unitary transformation on a finite dimensional inner product space if and only if ${T}$ maps orthonormal basis to orthonormal basis

• Proposition 22 Let ${V}$ be unitary and ${dim(V) < infty}$, ${T \in {\cal L}(V)}$. Then,

$\displaystyle T\mbox{ is unitary } \Leftrightarrow T^* \mbox{ exists and } TT^*=T^*T=\mathbb{I}$

• Theorem 23 Let ${h:V \rightarrow V}$ be a rigid motion (preserves distance in Euclidean space, real inner product), then ${h = T_aS}$, where ${T_a}$ is a translation by ${a}$ and ${S}$ is unitary

Basically, ${h(v)=T_{h(0)}(\underbrace{h(v)-h(0)}_{\mbox{claim: this is unitary}})}$