https://theintuition.blog/linear-transformations-and-its-sub-spaces/ LINEAR TRANSFORMATION AND FOUR SUB-SPACES

Linear Transformations / mapping are the most simple and easy to comprehend functions. Many problems are not linear in nature but we impose upon them a linearity by certain conditions. From calculus talking about differentiation on which we impose that the quantity $x\to 0$ Or In Double integral the length of the strip is taken infinitesimally small so that the variation of the function along that strip can be considered as a constant. Even in computational mathematics the non-linear equations are discretized using appropriate techniques to obtain the set of linear equations which are efficient and fast to solve.

Now let us understand some basic terminology of linear algebra.

The Row and Column Picture of a matrix :

Starting with the simple linear equation $x-2y = 1$ and $3x-6y = 11$ which can be written in the form $\begin{bmatrix} 1 & -2 \\ 3 & -6 \end{bmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{bmatrix} 1 \\ 11 \end{bmatrix}$ as it can be readily noticed that this set of equations has no solution. Lets see that graphically !

Row Picture

Column Picture

We can notice in row picture the two lines are parallel so no meeting point similarly in column picture it is impossible to obtain the output vector b from the linear combination of the two column vectors ( $c_1,c_2$ ) and hence (No Solution !)

Now on we can refer the above two pictures as Row Space and Column Space. In most of the texts the column picture is used because of its simplicity.

What is an n-dimensional space $\mathbb{R}^n$ ?

The standard n-dimensional space $\mathbb{R}^n$ comtains all real column vectors with n-components. A vector in this space follows the rules of vector addition and scalar multiplication such that the result of this operation produces the resultant vector in the same space.

E.g. $\mathbb{R}^5$ contains all the column vectors with its five components “5D-Space” and for components which are complex numbers lies in complex plane $\mathbb{C}^n$

Understanding Columns/Row space and Null space:

Suppose we are trying to solve the linear equation $Ax = b$ and assume that all of them lies in the same space then for the given matrix A we can get output b using every possible vector x which is similar to taking all linear combinations of A and this is what called a column space of A.

Column space of A consist of all linear combinations of the columns of A and they fill the column space $C(A)$

E.g. $Ax$ is $\begin{bmatrix} 1 & 0 \\ 4 & 3 \\ 2 & 3 \end{bmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} \implies x_1\begin{bmatrix} 1 \\ 4 \\ 2 \end{bmatrix} + x_2\begin{bmatrix} 0 \\ 3 \\ 3 \end{bmatrix}$

All the combination of this two columns fills up the plane in $\mathbb{R}^3$ . If the vector b lies in same $C(A)$ then it is solvable but for most of the problems they do not. Now continuing with the above example the column space is plotted in the image below with two vectors $B1 = [5,15,0]$ and $B2 = [0,3,3]$ as it can be noticed that the vector B1 lies in the column space and it is solvable (you can verify also !) while B2 lies outside the column space and it is not.

Null Space $N(A)$ :

The Null Space of A Consists of all solution of $x$ to $Ax = 0$ . If $x, y$ are in null space then the rule of scalar multiplication and vector addition gives the net vector in the same null space.

e.g. The linear equation $x+2y+3z = 0$ can be written as $\begin{bmatrix} 1 & 2 & 3 \end{bmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix} = 0$ , Here the two free variable $y, z$ can be chosen as 0 or 1 this gives us the two vectors in null space $S_1 = \begin{bmatrix} -2 \\ 1 \\ 0 \end{bmatrix} ; S_2 = \begin{bmatrix} -3 \\ 0 \\ 1 \end{bmatrix}$

So all the vectors on the $Ax = 0$ plane are the linear combinations of this $S_1$ and $S_2$ vectors which are also called as “Special Solution” and of course $\begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}$ is also the solution to this null space also called as “Trivial Solution“. The dimension of the null space is equal to number of special solution in above example the dimension $N(A)$ is $\mathbb{R}^2$

But for many matrices the only solution to $Ax = 0$ is $x = 0$ their null space contains only the zero vector and no special solution i.e., no combination of the column vectors can produce ${b = 0}$ apart from the zero combination i.e., ${x = 0}$ this also signifies the other important thing that the columns of the given matrix are “Independent“

Till now we know three different spaces : column space, row space and null space let us have a look in some special case of $A_{m \times n}$

$m = n = rank(r)$

This is the most ideal case (Square matrix) in which the columns/rows of a matrix are independent and has full rank and the solution is obtained by simply $x = A^{-1}\cdot{b}$ . There are no free variables/special solution and the only vector in the null space is Zero vector.

2. $m \geq n = r$ Full Column Rank Matrix

Here all the columns of A are independent and also number of equations greater than number of free variables, therefore contains no free variables or special solution to A, so $N(A)$ contains only the zero vector and the solution to $Ax = b$ has unique solution or No solution depending on whether the b lies in the column space or outside that.

3. $n \geq m = r$ Full Row Rank Matrix

Here all the rows of A are independent but the number of variables greater than the number of equations so there exists some free variables and hence $Ax = b$ has unique or infinitely many solutions. The number of special solution in $N(A) = n - r$

4. $rank \leq (m,n)$ More Common Case

I this case the solution to $Ax = b$ has no solution or infinitely many solutions.

Linear Independence revisited !

The columns of the given matrix A are linearly independent if the null space only contains the zero vector i.e., the only solution to $Ax = 0$ is $x = 0$ . The given vectors $v_1, v_2, ..., v_n$ are independent if they do not share the same plane.

e.g. For a matrix $A_{5 \times 7}$ here 7 vectors lies in $\mathbb{R}^5$ plane and the vector will be dependent based on the above listed cases.

Vector Spanning a Vector Space :

A set of vectors spans a space if their linear combinations fills the space.

e.g. $v_1 = [1, 0]; v_2 = [0, 1]; v_3 = [4, 7]$ this set of vectors spans a $\mathbb{R}^2$ space because the third vector is dependent vector. and $w_1 = [1, 0]; w_2 = [0, 1]$ spans the whole $\mathbb{R}^2$ space. So, the set of three vectors can span a line or a whole $\mathbb{R}^3$ space and also 10 vectors can span only a plane.

“So it all depends upon the independence of the vectors in the space”

The most basic conclusion that can be drawn from the above definition is

Two Vectors of any dimension cannot span all of the $\mathbb{R}^3$ space even if they all are linearly independent and similarly four vectors cannot span all of the $\mathbb{R}^3$

So we does not want less number of independent vectors and not more also we just want enough number and basis is just right !

What Are Basis ?

A basis for a vector space is a sequence of vectors which are linearly independent and they span the space, and The dimension of a space is the number of vectors in every basis.

e.g.-(1) For an Invertible matrix det(A) exists and which implies that the columns/rows are independent and the columns of matrix A spans the column space $C(A)$ of dimension $\mathbb{R}^n$ .

e.g.-(2) Consider the line through point $v = (1,7,3)$ has the dimension of one because it contains only one vector in its basis. Now perpendicular to this line is the plane $x + 7y+3z = 0$ in this plane there are two basis or two special solution $v_1 = (-7, 1, 0); v_2 = (-3, 0, 1)$ , so its basis contains two vector and hence dimension of two.

Note: The Dimension of column space and row space is equal to rank of a matrix $(r)$ and of a null space is $(n - r)$

The Four Fundamental Spaces :

Row Space $C(A^T)$

The dimension of the row space is the number of independent rows ( non-zero rows in Echelon form ) and the non-zero rows in Echelon form forms a basis.

Column Space $C(A)$

Similar to row space the number of independent columns (non-zero columns in Echelon form) and the non-zero columns in Echelon form forms the basis.

Null Space $N(A)$ Solution to $A \cdot x = 0$

For the matrix $A_{m \times n}$ and the rank $r$ the dimension of $N(A)$ is $n - r$ which is equal to the number of special solution/free variables.

Left Null Space $N(A)$ Solution to $A^T \cdot x = 0$

For the matrix $A_{m \times n}$ and the rank $r$ the dimension of $N(A)$ is $m - r$ which is equal to the number of special solution/free variables. The solution to this looks for the combination of rows of matrix A that produces Zero.

Dimension of $\mathbb{R}^n$ = Dimension of $C(A) = r$ + Dimension of $N(A) = n - r$

LINEAR TRANSFORMATION AND FOUR SUB-SPACES

Like this:

Related

Leave a ReplyCancel reply

Share this:

Like this:

Related

Related Posts

Leave a ReplyCancel reply

Discover more from The Intuition.Blog