Spaces, Systems, Matrices

Section 2.3 Spaces, Systems, Matrices

Subsection 2.3.1 A Rose by Any Other Name

Consider the system of equations

\begin{align*} x+3y-z\amp =0\\ y-z\amp =0. \end{align*}

Using the techniques we have studied this simplifies to

\begin{align*} x+2z\amp =0\\ y-z\amp =0 \end{align*}

so that the general solution is

\begin{align*} x\amp =-2z\\ y\amp =z \end{align*}

If we write the solutions as vectors we get

\begin{equation*} \left[ \begin{array}{r} -2 \\ 1 \\ 1 \end{array} \right]z \end{equation*}

so that both

\begin{equation*} \left[ \begin{array}{r} -4 \\ 2 \\ 2 \end{array} \right]\mbox{ and } \left[ \begin{array}{r} 6 \\ -3 \\ -3 \end{array} \right] \end{equation*}

will be solutions with \(z=2\) and \(z=-3\) respectively. However, if we add or subtract these from each other

\begin{equation*} \left[ \begin{array}{r} -4 \\ 2 \\ 2 \end{array} \right]+ \left[ \begin{array}{r} 6 \\ -3 \\ -3 \end{array} \right]= \left[ \begin{array}{r} 2 \\ -1 \\ -1 \end{array} \right]\mbox{ or } \left[ \begin{array}{r} -4 \\ 2 \\ 2 \end{array} \right]- \left[ \begin{array}{r} 6 \\ -3 \\ -3 \end{array} \right]= \left[ \begin{array}{r} -10 \\ 5 \\ 5 \end{array} \right] \end{equation*}

we get two more solutions. What this tells us is that any multiple of a solution is a solution and any sum of two solutions is another solution; solutions therefore form a vector space of their own. However since not all vectors in \(\mathbb{R}^3\) are solutions to the system, the solutions are a subspace of something larger.

What we see here is another way to think of solutions to a system of equations. The solutions of a homogeneous system of equations form a subspace. Therefore we can talk about a basis for the solutions and the span of the solutions. This is another example of how one idea can be looked at from different perspectives.

Subsection 2.3.2 A Universe in a Matrix

If we let

\begin{equation*} A= \left[ \begin{array}{rrr} 1 \amp 3 \amp -1\\ 0 \amp 1 \amp -1\\ \end{array} \right] \end{equation*}

then there are three spacial subspaces associated with the matrix. The column space which is the set of all \(\vec{b}\) such that there is a solution for

\begin{equation*} A\vec{x}=\vec{b}. \end{equation*}

The row space which is the set of all \(\vec{c}\) such that there is a solution for

\begin{equation*} \vec{y}A=\vec{c}. \end{equation*}

And, the null space which is the set of all solutions to the homogeneous equation

\begin{equation*} A\vec{x}=\vec{0}. \end{equation*}

Subsubsection 2.3.2.1 The Null Space

The null space is the simplest to explain. In this case it is precisely the set of solutions we found above

\begin{equation*} Null(A)= \left\{ \left[ \begin{array}{r} -2 \\ 1 \\ 1 \end{array} \right]t: t\in\mathbb{R} \right\} \end{equation*}

which is a subspace of \(\mathbb{R}^3\text{.}\) Because of how matrix multiplication works the dimension the null space is in must always be the same as the number of columns in the matrix.

Subsubsection 2.3.2.2 The Column Space

The column space is a little harder. Consider the equation

\begin{equation*} A\vec{x}=\vec{b}\equiv \left[ \begin{array}{rrr} 1 \amp 3 \amp -1\\ 0 \amp 1 \amp -1\\ \end{array} \right] \left[ \begin{array}{r} x_1\\ x_2\\ x_3 \end{array} \right]= \left[ \begin{array}{r} b_1\\ b_2 \end{array} \right] \end{equation*}

We can write this as a linear combination of vectors from \(\mathbb{R}^2\)

\begin{equation*} \left[ \begin{array}{r} 1\\ 0 \end{array} \right]x_1+ \left[ \begin{array}{r} 3\\ 1 \end{array} \right]x_2+ \left[ \begin{array}{r} -1\\ -1 \end{array} \right]x_3= \left[ \begin{array}{r} b_1\\ b_2 \end{array} \right]. \end{equation*}

Thus the column space, unlike the null space, is in this case in \(\mathbb{R}^2\text{.}\) In general the dimension the column space is in is the same as the number of rows in the matrix.

If we let \(x_1=2\) and \(x_2=-1\) then

\begin{equation*} \left[ \begin{array}{r} 1\\ 0 \end{array} \right]x_1+ \left[ \begin{array}{r} 3\\ 1 \end{array} \right]x_2= 2 \left[ \begin{array}{r} 1\\ 0 \end{array} \right]- \left[ \begin{array}{r} 3\\ 1 \end{array} \right]= \left[ \begin{array}{r} -1\\ -1 \end{array} \right]. \end{equation*}

That is we don't really need the third vector, the column space of \(A\) is therefore

\begin{equation*} Col(A)= \left\{ \left[ \begin{array}{r} 1\\ 0 \end{array} \right]s+ \left[ \begin{array}{r} 3\\ 1 \end{array} \right]t: s,t\in\mathbb{R} \right\}. \end{equation*}

With a little work you should be able to, at this point, convince yourself that that includes every point in \(\mathbb{R}^2\text{.}\)

Subsubsection 2.3.2.3 The Row Space

The row space is similar to the column space, except you use rows, instead of columns. Consider the equation

\begin{equation*} \vec{y}A\vec{x}=\vec{c}\equiv \left[ \begin{array}{rr} y_1 \amp y_2 \end{array} \right] \left[ \begin{array}{rrr} 1 \amp 3 \amp -1\\ 0 \amp 1 \amp -1\\ \end{array} \right]= \left[ \begin{array}{rrr} c_1\amp c_2\amp c_3 \end{array} \right] \end{equation*}

which we can also write as a linear combination of vectors

\begin{equation*} \left[ \begin{array}{r} 1 \\ 3 \\ -1 \end{array} \right]y_1+ \left[ \begin{array}{r} 0 \\ 1 \\ -1 \end{array} \right]y_2= \left[ \begin{array}{r} c_1 \\ c_2 \\ c_3 \end{array} \right] \end{equation*}

or, finally, as a set

\begin{equation*} Row(A)= \left\{ \left[ \begin{array}{r} 1 \\ 3 \\ -1 \end{array} \right]t+ \left[ \begin{array}{r} 0 \\ 1 \\ -1 \end{array} \right]s : s,t\in\mathbb{R} \right\}. \end{equation*}

Notice that the row space, like the null space, is in this case a subset of \(\mathbb{R}^3\text{.}\) In general the dimension the row space is in will be equal to the number of columns in the matrix.

Pulling Things Together.

Consider the matrix which consists of the two vectors that define the row space and the one vector that defines the null space,

\begin{equation*} \mathcal{B}= \left[ \begin{array}{rrr} 1 \amp 0 \amp -2\\ 3 \amp 1 \amp 1\\ -1 \amp -1 \amp 1\\ \end{array} \right]. \end{equation*}

If we row reduce this to its row reduced echelon form we get identity matrix (you should pause for a moment and make sure you can get that.) This means that we could solve any equation of the form \(\mathcal{B}\vec{x}=\vec{b}\) and that it would have a unique solution for any \(\vec{b}\in\mathbb{R}^3\text{.}\) That is, the three vectors that make up the columns of the matrix are linearly independent and form a basis for \(\mathbb{R}^3\text{.}\) This is a result that will actually be true for any matrix.

If you evaluate the code below you can see how the row space and null space complement each other to give us all of \(\mathbb{R}\text{.}\) You can also see that we need three dimensions to draw them which is the number of columns we originally started with. However, the null space here is a line, which is one dimensional. The row space is a plane which is two dimensional. So, the dimension of a space is not the same as the number of dimensions you need to draw or understand it.

Theorem 2.3.1.

Given an \(m\times n\) matrix \(A\) the row space is the set of all linear combinations of the rows and the null space is the set of all solutions to \(A\vec{x}=\vec{0}\text{.}\) Bases for each taken together will form a basis for \(\mathbb{R}^n\text{.}\)

Section Vocabulary.

Column Space, Row Space, Null Space