Linear Algebra: Results Summary

1. Definitions

1.1. Vector Spaces

  • Let {V \neq \phi} be a set, and {F} be a field. We say {V} is a vector space if there exists two binary operations {+: V \times V \rightarrow V} and {\cdot: F \times V \rightarrow V}, under which {V} is closed and satisfies:
    1. {(V,+)} is a commutative group
    2. If {\lambda, \mu \in F, x \in V}, then {(\lambda \mu) \cdot x = \lambda \cdot (\mu x)}
    3. {\lambda \cdot (x+y) = \lambda \cdot x + \lambda \cdot y}
      {(\lambda + \mu)\cdot x = \lambda \cdot x + \mu \cdot x}

    4. {1 \cdot x = x}, where {1} is the identity of {F}
  • Subspaces are subsets of {V}, which are closed under vector addition and scalar multiplication
  • If {S'} be the smallest subspace of {V} containing {S}, {S'} is called the subspace generated by {S}
  • It can be shown that, {S' = [S]}
  • finite dimensional: {V} is finite dimensional over {F} if {\exists} a finite set {S \subset V}, s.t. {V = [S]}. Otherwise, {V} is infinite dimensional.
  • basis: {S} is a basis of {V} over {F} if {V = [S]}, and no proper subset of {S} spans {V}
  • LI: {\{x_1, \ldots, x_n\}} is LI iff

    \displaystyle \sum_{j=1}^n \lambda_j x_j = 0 \Rightarrow \lambda_j = 0 \quad \forall 1 \leq j \leq n

    {S} is LI if every finite subset of {S} is LI

  • Complementary subspace: Let {W} be a subspace of {V}. We say a subspace {\overline{W}} of {V} complementary of {W} if
    1. {W+\overline{W}=V}
    2. {W \cap \overline{W} = \{0\}}

1.2. Quotient Space

  • {V} is a vector space over {F} and {W} is a subspace of {V}. Quotient space

    \displaystyle V/W = \{x + W : x \in V\}

    addition:

    \displaystyle (x + W) +_{V/W} (y+W) = (x+y) + W

    multiplication:

    \displaystyle \lambda \cdot_{V/W} (x+W) = \lambda x + W

    {V/W} can be shown to be a vector space

1.3. Linear Mappings

  • Let {V_1, V_2} be two vector spaces over same field {F}. A mapping {T:V_1 \rightarrow V_2} is linear if

    \displaystyle T(\lambda x + \mu y) = \lambda T(x) + \mu T(y) \qquad \lambda, \mu \in F, x,y \in V_1

  • {T} is isomorphism if it is one-one, onto and linear
  • Let {V} be a vector space over {F}. {{\mathcal L}(V,F) \triangleq V^{*}} is called the dual space of {V}
  • Given a basis {\{v_i\}_{i=1}^n} of {V}, we can derive a basis of the dual space as {\Lambda_{v_j}(v_k)= \delta_{jk}}
  • For any {v\in V}, define, {f_v : V^* \rightarrow F, f_v(\Lambda)=\Lambda (v), \Lambda \in V^*}. Set of all such {f_v}‘s form the dual of {V^*} and is denoted by {V^{**}}
  • Define {\psi : V \rightarrow V^{**}, \psi(v)=f_v}
  • {\psi} is linear, one-one, in addition onto for finite dimensional vector spaces.
  • Annihilator: Let {V} be a vector space and {S \subset V, S \neq \phi}, annihilator of {S} is defined as,

    \displaystyle S^a = \{\Lambda \in V^* : \Lambda(x)=0_F, \forall x \in S\}

  • Let {T: V \rightarrow W}, linear. Let {\{v_i\}_{i=1}^n} and {\{w_i\}_{i=1}^m} be two bases respectively.

    \displaystyle  \begin{array}{rcl}  T(v_j) &=& \sum_{k=1}^m \alpha_{kj}w_k \qquad 1 \leq j \leq n \\ T(v_1)&=& \alpha_{11}w_1+\cdots+\alpha_{m1}w_m \\ \vdots &=& \vdots\\ T(v_n) &=& \alpha_{n1}w_1+\cdots+\alpha_{mn}w_m \end{array}

    The matrix associated with this transformation is,

    \displaystyle  \begin{array}{rcl}  \begin{bmatrix} \alpha_{11} & \cdots & \alpha_{1n} \\ \alpha_{21} & \cdots & \alpha_{2n} \\ \vdots & & \vdots \\ \alpha_{m1} & \cdots & \alpha_{mn} \end{bmatrix} \end{array}

  • Given {T \in {\mathcal L}(V,W)}, we associate, {T^t : W^* \rightarrow V^*} given by

    \displaystyle (T^t\Lambda)(v)=\Lambda(Tv) \qquad \forall v \in V

1.4. Inner Product Spaces

  • An orthonormal set {\{v_i\}_{i=1}^n} in an inner product space {V} is said to be complete, if this can’t be properly included in any other orthonormal set.
  • For a given linear transformation {T: V \rightarrow V}, the transformation {T^*: V \rightarrow V} is called the adjoint of {T} if {\langle Tx,y\rangle =\langle x,T^*y\rangle , \quad \forall x,y \in V}
  • We say {T \in {\cal L}(V)} is a positive operator if {T=T^*} and {\langle Tx,x\rangle \geq 0 \quad \forall x \in V} (non-negative semidefinite). Further, {T} is strictly positive if {T} is positive and {\langle Tx,x\rangle > 0 \quad \forall x \neq 0}
  • Let {V} be a unitary space (complex inner product), then {T \in {\cal L}(V)} which preserves inner product is called unitary transformation. If {V} is Eucliedean (real inner product), {T} is orthogonal transformation
  • Let {V} be an Eucliedean space and {h:V \rightarrow V}. We say {h} is a rigid motion if {||hx-hy||=||x-y|| \quad \forall x,y \in V}. A special kind of rigid motion is translation, {T_v(u) = v+u} is called translation by {v}. e.g., any unitary transformation {S} is a rigid motion.

2. Theorems

2.1. Vector Spaces

  • Every field is a vector space over itself
  • Let {V} be a finite dimensional vector space over {F}, then {V} has a basis
  • If {\{v_i\}_{i=1}^n} span {V} and are LI, then it is a basis
  • If {\{v_i\}_{i=1}^n} span {V} and {dim(V)=n}, then it is a basis
  • If {V} is a finite dimensional vector space having a basis consisting of {n} elements, any LI set of {n} vectors in {V} is a basis of {V}
  • Let {dim(V)< \infty}. Then any two bases of {V} have same number of elements
  • If {W} is a subspace of a finite dimensional vector space {V}, {dim(W) \leq dim(V)}, and equality iff {W=V}
  • For a finite dimensional vector space {V}, the following statements are equivalent
    1. {\{v_i\}_{i=1}^n} is a basis of {V}
    2. {\{v_i\}_{i=1}^n} is maximal linear independent set
    3. {\{v_i\}_{i=1}^n} is minimal generating set
  • Subspaces {W} and {\overline{W}} are complementary iff each {x \in V} can be uniquely written as {x=a+b, a \in W, b \in \overline{W}}. We say {V=W \oplus \overline{W}}, direct sum
  • If {W} is a subspace of a finite dimensional vector space {V}, then {W} has a complementary subspace
  • Theorem 1 Let {W_1} and {W_2} be two subspaces of a finite dimensional vector space {V}, then

    \displaystyle dim(W_1+W_2)=dim(W_1)+dim(W_2)-dim(W_1 \cap W_2)

2.2. Quotient Space

  • Theorem 2 Let {V} be a finite dimensional vector space over {F} and {W} is a subspace of {V}. Then

    \displaystyle dim(V/W) = dim(V)-dim(W)

2.3. Linear Mappings

  • For any linear transformation {T: V \rightarrow W }, the vector spaces {V} and {W} are on same scalar field {F}
  • Let {T: V \rightarrow W } be linear and {S \subset V} s.t. {V=[S]}, then {{\Re}(T)=[T(S)]}
  • Let {dim(V)=n}, {T: V \rightarrow W } be linear, then {dim({\Re}(T)) \leq n}
  • If {\{x_1,\cdots,x_k\}} are LI in {V} and {T} one-one, then {\{Tx_1,\cdots,Tx_k\}} are LI in {W}
  • Let {dim(V)=n}, then {dim({\Re}(T))=n \Leftrightarrow T} is one-one
  • Theorem 3 : Let {V} and {W} be vector spaces. Let {\{v_i\}_{i=1}^n} is a basis of {V}. Let {w_i, 1 \leq i \leq n} be any set of (not necessarily distinct) vectors in {W}. Then {\exists} a unique linear map {T: V \rightarrow W } such that {T(v_i)=w_i}

  • Theorem 4 (Rank-Nullity Theorem) : Let {V} be a finite dimensional vector space and {T: V \rightarrow W }, linear, then

    \displaystyle  \begin{array}{rcl}  dim(V) &=& dim(Ker(T))+ dim({\Re}(T))\end{array}

  • If {T: V \rightarrow W } be an isomorphism, so is {T^{-1}: W \rightarrow V }
  • {T: V \rightarrow W }, {V} and {W} finite dimensional and {dim V = dim W}, then {T} is 1-1 {\Leftrightarrow T} is onto
  • Theorem 5 (Fundamental Theorem of Homomorphism) : Let {V} and {W} be vector spaces over {F}. {T: V \rightarrow W } be a surjective linear map. Then, {V/Ker(T) \cong W}

  • Let {V} and {W} be vector spaces. Then the dimension of {{\mathcal L}(V,W)} is {dim V \times dim W}
  • If {S_1 \subset S_2} then {S_2^a \subset S_1^a}
  • If {dim(V)< \infty} and {S} be a subspace of {V}, then

    \displaystyle dim(V)=dim(S)+dim(S^a)

    (The proof is useful for many similar proofs)

  • Solution of homogeneous linear equations: A homogeneous linear system of equations {Ax=0, A \in M_{m \times n}(F), x \in F^n} has solution iff {\Lambda_x} is in the annihilator of the row space of {A}. {\Lambda_x(y)=x^T y}
  • Solution of non-homogeneous linear equations: Let {M=\{\alpha_1,\cdots,\alpha_m\}} be a set of {m} vectors in {F^n} and {b_1,\cdots, b_m} be {m} scalars. Let

    \displaystyle  \begin{array}{rcl}  \alpha_j &=& (a_{j1},\cdots,a_{jn}) \in F^n \qquad 1 \leq j \leq m \\ \alpha_j' &=& (a_{j1},\cdots,a_{jn},b_j) \in F^{n+1} \qquad 1 \leq j \leq m \\ M &=& \{\alpha_1,\cdots,\alpha_m\} \\ M' &=& \{\alpha_1',\cdots,\alpha_m'\} \end{array}

    then {\exists \Lambda \in (F^n)^*} such that {\Lambda(\alpha_j)=b_j, 1 \leq j \leq m} iff {dim[M]=dim[M']}

    In other words, non-homogeneous system of equation has a solution iff the row rank of the coefficient matrix is equal to the row rank of its augmented matrix.

  • If {A} is the matrix associated with {T}, that with {T^t} will be {A^T}

2.4. Inner Product Spaces

  • Useful inequalities
    Cauchy-Schwartz: {|\langle x,y\rangle| \leq \sqrt{\langle x,x\rangle}.\sqrt{\langle y,y\rangle}}
    Hölder’s: Let {p>1} real and {q} be s.t. {\frac{1}{p}+\frac{1}{q}=1}

    \displaystyle \left|\sum_{j=1}^n x_j y_j\right| \leq \left( \sum_{j=1}^n |x_j|^p\right)^{\frac{1}{p}}.\left( \sum_{j=1}^n |y_j|^q\right)^{\frac{1}{q}}

    This is a generalization for Cauchy-Schwartz in ordinary real inner products
    Minkowski’s: Let {p>1} real

    \displaystyle ||x+y||_p \leq ||x||_p + ||y||_p

    {||.||_p : L_p} norm on {\mathbb{R}^n}

  • Theorem 6 Let {S} be an orthogonal set not containing the zero element, then {S} is LI

  • Theorem 7 Let {V} be an inner product space and {\{v_i\}_{i=1}^k} be an LI set. Then {\exists} a set {\{w_i\}_{i=1}^k} in {V} s.t.

    1. {\{w_i\}_{i=1}^k} is orthonormal
    2. for each {l, 1 \leq l \leq k}, {[v_1 \dots v_l]=[w_1 \dots w_l]}

    The proof of this uses the Gram-Schmidt orthogonalization procedure, which is useful for extracting orthogonal basis from LI. The orthogonal set of vectors can be written as,

    \displaystyle u_n = v_n - \sum_{j=1}^{n-1} \frac{\langle v_n, u_j \rangle}{\langle u_j, u_j\rangle} u_j \quad n = 1, \dots, k

    Further we normalize it to get the orthonormal set {\{w_i\}_{i=1}^k}.

  • Theorem 8 Let {\{v_i\}_{i=1}^n} be an orthonormal basis for an inner product space {V}, then for any {v \in V}

    \displaystyle v = \sum_{j=1}^n \langle v,v_j \rangle v_j

    for any {v,w \in V}

    \displaystyle \langle v,w \rangle =\sum_{j=1}^n \langle v,v_j\rangle \langle v_j,w\rangle

  • Theorem 9 Let {\{v_i\}_{i=1}^n} be a set in {V}, s.t. {\langle v_j,v_j\rangle = 1, 1 \leq j \leq n}. Suppose {\forall v \in V}

    \displaystyle \langle v,v\rangle = \sum_{j=1}^n |\langle v,v_j\rangle |^2

    then {\{v_i\}_{i=1}^n} is an orthonormal basis

  • Theorem 10 (Bessel’s inequality) Let {\{v_i\}_{i=1}^n} be an orthonormal set in an inner product space {V}, then for any {w \in V}

    \displaystyle \sum_{j=1}^n |\langle w,v_j\rangle |^2 \leq \|w\|^2

  • Let {\{v_i\}_{i=1}^n} be an orthonormal set in an inner product space {V}, then the following are equivalent:
    1. {\{v_i\}_{i=1}^n} is complete
    2. {\langle v,v_j\rangle = 0} for {1 \leq j \leq n \Rightarrow v = 0}
    3. {V = [v_1 \dots v_n]}
    4. For each {v \in V}, {v = \sum_{j=1}^n \langle v,v_j\rangle v_j}
    5. For any {v,w \in V}

      \displaystyle \langle v,w\rangle =\sum_{j=1}^n \langle v,v_j\rangle \langle v_j,w\rangle \qquad \mbox{ (Parseval's identity)}

    6. For any {v \in V}

      \displaystyle \|v\|^2 = \sum_{j=1}^n |\langle v,v_j\rangle |^2

  • Theorem 11 Let {S} be a finite dimensional subspace of an inner product space {V}, then {V = S \oplus S^{\bot}}. {S^{\bot}} is the orthogonal complement of {S}

  • Theorem 12 (Pythagoras) Let {S} be a finite dimensional subspace of an inner product space {V}, then for each {v \in V},

    \displaystyle v = v_S + v_{S^{\bot}} \quad v_S \in S, v_{S^{\bot}} \in S^{\bot}

    1. {\|v\|^2 = \|v_S\|^2 + \|v_{S^{\bot}}\|^2}
    2. {\|v-v_S\| \leq \|v-w\|, \forall w \in S}; Best approximation

  • Theorem 13 (Riesz Representation Theorem) If {V} be a finite dimensional inner product space and {\Lambda \in V^*}, then {\exists} a unique {y \in V} such that {\Lambda(x)=\langle x,y\rangle , \quad \forall x \in V}. So, we write {\Lambda \equiv \Lambda_y}. {\Lambda_y(.) = \langle .,y\rangle }. This way every {\Lambda \in V^*} can be uniquely identified by a {y \in V}

  • The mapping {T: V \rightarrow V^*} given by {T(y)=\Lambda_y} can be shown to be one-one, onto and conjugate linear.
  • For polynomial inner product space, {V = \mathbb{C}[x]}, with inner product {\langle p,q\rangle = \int_0^1 p(x)\overline{q(x)}dx}, if the functional is defined as a evaluation of the polynomial at some point {z_0}, i.e., {\Lambda(p)=p(z_0)}, then the only possible functional is the zero functional, i.e., {\Lambda \equiv \Lambda_0}
  • Theorem 14 Let {T \in {\cal L}(V)} and suppose {T^*} exists, then

    1. {Ker T^* = \Re (T)^{\bot}}
    2. If {dim V < \infty}, {T} and {T^*} has same rank

  • Some properties of the adjoint transformation:

    \displaystyle  \begin{array}{rcl}  (T_1+T_2)^* &=& T_1^* + T_2^* \\ (\alpha T)^* &=& \overline{\alpha} T^* \\ (T^*)^* &=& T \qquad \mbox{ (involution)} \\ (T_1T_2)^* &=& T_2^* T_1^* \end{array}

  • Proposition 15 {T \in {\cal L}(V)} is self-adjoint iff the matrix of {T} wrt some orthonormal basis of {V} is self-adjoint (Hermitian)

  • Proposition 16 If {V} is a complex inner product space and {T \in {\cal L}(V)}, ({T} not necessarily self-adjoint), then

    \displaystyle T \equiv 0 \Leftrightarrow \langle Tx,x\rangle= 0 \quad \forall x \in V

    This is a special property for complex, if {T=T^*}, this is true for any inner product

  • Proposition 17 Let {V} be a complex inner product space and {T \in {\cal L}(V)}, then

    \displaystyle T=T^* \Leftrightarrow \langle Tx,x\rangle \mbox{ is real } \forall x \in V

  • Proposition 18 Let {V} be an inner product space and {T \in {\cal L}(V)}, strictly positive, {dim(V) < \infty} then {T} is invertible

  • Proposition 19 Let {V} be an inner product space with {dim(V) < \infty}, {P \in {\cal L}(V)} is an orthogonal projection iff {P=P^2=P^*}

  • Every orthogonal projection is a positive linear transformation
  • Theorem 20 Let {T \in {\cal L}(V)} preserves inner product, then

    1. {T} preserves norm (isometric)
    2. {T} is linear
    3. {T} is one-one
    4. If {dim(V) < \infty}, then {T} is invertible

  • Proposition 21 {T} is a unitary transformation on a finite dimensional inner product space if and only if {T} maps orthonormal basis to orthonormal basis

  • Proposition 22 Let {V} be unitary and {dim(V) < infty}, {T \in {\cal L}(V)}. Then,

    \displaystyle T\mbox{ is unitary } \Leftrightarrow T^* \mbox{ exists and } TT^*=T^*T=\mathbb{I}

  • Theorem 23 Let {h:V \rightarrow V} be a rigid motion (preserves distance in Euclidean space, real inner product), then {h = T_aS}, where {T_a} is a translation by {a} and {S} is unitary

    Basically, {h(v)=T_{h(0)}(\underbrace{h(v)-h(0)}_{\mbox{claim: this is unitary}})}

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s