[Linear Algebra] Vector space, Subspace and Column space

 

 

Vector spaces

A vector space is a space that satisfies two requirements :

 

1. $\vec{v}+\vec{w}$ and $c\vec{v}$ are in the space.

 

2. all combinations $c\vec{v} + d\vec{w}$ are in the space.

 

So, a real vector space is a set of vectors together with rules for vector addition and multiplication by real numbers.

 

Examples of three spaces

1. The inifinite-dimensional space $\mathbb{R}^\infty$ is a space. 

 

2. The space of $3\times 2$ matrices. It also satisfy all requirements of vector space, so in this case, this is a space in which vectors are matrices.

 

3. The space of functions $f(x)$. if we admit all functions $f$ that are defined on a fixed interval, then the vectors are functions and the dimension is larger infinity than $\mathbb{R}^\infty$ 

 

Subspace

 Let's say, if we choose any 2-dimensional plane through the origin from $\mathbb{R}^3$, then the plane is a vector space in its own right. Any of two vector stay in the plane after multiplying or making a linear combination of them. We call such vector space as subspace of the original space $\mathbb{R}^3$. All subspaces have to through the origin.

 

 Then what is the difference between a subset and a subspace? It is made clear by following example. If we think about $\mathbb{R}^2$ plane, and let a subset is the first quadrant of the $x$-$y$ plane; the coordinates on $x \geq 0$ and $y \geq 0$. It is a subset of the original vector space, but it's not a subspace. It's because if we multiply a negative scalar to a vector that towards to, say $(2,2)$, is in third quadrance instead of being in the subspace. 

 

Union of Subspaces 

 Suppose we have two subspaces and take the union of two subspace : $P$ and $L$. Does the union of subspaces form a space? ($P \cup L$ is all vectors in $P$ or $L$ or both)

 

NO. This is not a subspace. 

 

Intersection of Subspaces

 Then does an intersection of two subspaces form a subspace? ($P \cap L$ is all vectors in both)

 

YES. An intersecation of two vector space is a subspace. It probably smaller than each one.

 

---

Column Space

The column space contains all linear combinations of the columns of A. It is a subspace of $\mathbb{R}^m$. 

 

Suppose we have a matrix $A$ like below. 

Then the column space of $A$ is a subspace of $\mathbb{R}^4$.

 

And if we have an linear system like below, 

does $A\vec{x}=\vec{b}$ have a solution for every $\vec{b}$?

 

NO. The combinations of the columns do not span the whole $\mathbb{R}^4$ space.

 

Which right-hand sides are okay? In other words, which b's allow this system to be solved?

 

The linear system $A\vec{x}=\vec{b}$ can be solved if and only if when $\vec{b}$ is in the column space $C(A)$.

 

What the column space looks like? It can be easily seen that an arbitrary combination of any of two column can comprise the third vector. So these columns are dependent and a linear combination of any of two vectors makes nothing new. So, what the column space forms are two vectors of $\mathbb{R}^4$. It is going to be very thin 2d plane inside the original space $\mathbb{R}^4$.

 

As a result, a column space can be between zero space and the whole space $\mathbb{R}^m$.

---

Nullspace

The nullspace of a matrix consists of all vectors $\vec{x}$ such that $A\vec{x}=0$. It is denoted by $N(A)$. It is a subspace of $\mathbb{R}^n$.

 

In above case, all solutions $\vec{x}$ to $A\vec{x}=0$ form a nullspace of $A$. But, does the all solution vectors form a subspace?

 

YES. It can be proved like this : If $A\vec{v}=0$ and $A\vec{w}=0$, then $A(\vec{v}+\vec{w})=0$. Because of the distributive law of matrix calculation, the terms inside the parenthesis can be separated into $A\vec{v}+A\vec{w}$ and each terms are equal to zero. Also, $A(12\vec{v})=0$ because scalar multiple can be moved outside parenthesis. The nullspace of $A$ is the line of all points $x=c, y=c, z=-c$.(The line goes through the origin, as any subspace must.)

 

Here is a question. If right hand size is not $\vec{0}$, do the solutions form a subspace?

 

NO. Absolutely don't. Only the solutions to a homogeneous equation ($b=0$) form a subspace. There could be bunch of solutions, but not space. 

 

 

---

References

• Gilbert Strang. 18.06 Linear Algebra. Spring 2010. Massachusetts Institute of Technology: MIT OpenCourseWare, https://ocw.mit.edu. License: Creative Commons BY-NC-SA.
• Strang, G. (2012). Linear algebra and its applications. Thomson Brooks/Cole.
  
  

Footnotes