Professor Dave again, let’s get independent. With some basic definitions out of the way, we now want to start thinking about ways we can express vector spaces in the simplest, most efficient way. To do this, we need the fewest number of elements than can be used to write any other elements within the vector space.
To put it simply, we don’t need any unnecessary information. Let’s learn how to do this now. Let’s start by considering three vectors: a, b, and c.
Let’s also say that c is equal to (a plus 2b). In this case, for any linear combination of a, b, and c, such as 2a + b - c, c can be replaced by (a plus 2b), making it a combination of only a and b. Because c can be written in terms of a and b, it is unnecessary information, and the vectors a, b, and c are said to be linearly dependent.
That the vector c can be written in terms of a and b, makes it “dependent” on them. On the other hand, if none of the vectors can be written in terms of the others, they are linearly independent. The way this is expressed is as follows.
If you have vectors v1, v2, all the way to vn, in a vector space V, then to be linearly independent, the equation c1v1 + c2v2 all the way to CNVN = 0 can only be satisfied if all the scalars from c1 to cn equal zero. Let’s simply take this first element over to the other side of the equation. By doing this we can see that the element could be expressed in terms of all the others if any of the coefficients were nonzero.
But if they are all zero, then the left side simplifies to zero, and for the equation to be satisfied, our last scalar must then also be zero. This is the requirement for linear independence. So to see if a set of vectors or elements are linearly independent, we must simply see if there is a set of nonzero scalars that exists that makes the linear combination of the elements equal to zero.
If those nonzero solutions exist, then the elements are linearly dependent. If there is no solution aside from all zeros, the elements are linearly independent. This ends up being the same as trying to solve a set of equations, which is a task we are familiar with.
Let’s start with a simple example, two vectors, (1, 1) and (1, -1). As we just discussed, we’ll make a linear combination, multiplying these vectors by scalars and setting the sum equal to zero. We will immediately see that we can rewrite this as a set of two equations with two unknowns, which we could solve by substitution or elimination.
Opting for the latter, let’s just add these two equations together so that the c2 terms cancel, and we will get two c1 equals zero, so c1 equals zero. Plugging this into either equation we find that c2 must also equal zero. Our only solution for this system is all zeros, and so our two vectors are linearly independent.
This makes sense, because there’s nothing we can multiply one vector by in order to get the other. Solving a simple system like this is a piece of cake, but often we’ll be dealing with much more complicated systems. However, we can still use the methods we’ve already learned by treating the scalars as our unknown variables.
If we solve the system and find only zeros, the vectors are linearly independent, but if we find that any number of scalars are nonzero, or that any scalar can be set to any nonzero value, the system will be linearly dependent. When a variable can take on any value for the system, this is called a “free variable”. The method we can use in such a situation involves setting up a matrix and using row reduction to get it into row echelon form, so that the nonzero elements of the matrix are only on the diagonal and upper right side, as we have learned previously.
To demonstrate, let’s consider the vectors (1, 1, 3, 3), (-1, 1, -1, -3), and (2, 5, 9, 6). We can take these vectors and multiply each by a scalar, and then set their sum to zero. We can then turn this into a matrix equation and get to solving.
In past examples like this, we’ve included the right hand side as an additional column in our matrix, making an augmented matrix. But since the right side is all zeros, we can actually just ignore it in this case. Now recall that row echelon form has leading ones down the diagonal, and all the elements below the diagonal are zero.
We can use simple row operations to achieve this. Our first element of the first row is already 1, which will make it easy to get the elements below it equal to zero. Going row by row, first we will subtract the first row from the second row to make the second row’s first element zero.
Next we will subtract 3 times the first row from the third to get a new third row, and we will also subtract 3 times the first row from the fourth to get a new fourth row. Now the first column looks great. Next, we want to make the second element of the second row equal to one, while also making all elements below it zero.
We can go ahead and subtract the second row from the third to get a new third row, since all of the elements easily cancel to give zeros. Now we can just divide the second row by two to make the leading element equal to one. Looking at our solution, we have only two rows with nonzero entries, which is one less than our number of unknown scalars.
This implies that we have a free variable that could be set to anything, meaning it’s not required to be zero. Because of this, the set of vectors is linearly dependent. We can see this by writing out our solution as a set of equations, like this.
We end up with two equations with three unknowns. This is not enough to constrain our unknowns, and we must have one free variable which can relate one of our vectors to the others. So in this particular case, we conclude linear dependence for these three vectors.
If after our row operations we were left with a surviving 1 in the third row, third column, when we wrote our set of equations, we would have had to include a third equation that sets c3 equal to zero. Then, substituting that into the second equation, c2 would have to be zero, and then plugging in zero for both c2 and c3 in the first equation, we would find that c1 is equal to zero as well. All three scalars would end up being zero, and this is how we would identify this as a linearly independent system.
We have one more method of determining linear independence, and we can use this if we have a set of vectors that makes a square matrix. This method involves finding the determinant, which is something we can only do with square matrices. If the determinant ends up being 0, the vectors are linearly dependent, and if the determinant is not zero, the vectors are linearly independent.
Let’s go back to our simple pair of vectors (1, 1) and (1, -1). We have already seen they are linearly independent, so the determinant of the matrix made of these vectors should be nonzero. Let’s set up the matrix and test it.
To take the determinant we find one times negative one, minus the quantity one times one, giving us a determinant of -2. Since this is not zero, the rule tells us these vectors are linearly independent, which we previously verified. Finally, given that vectors spaces can be made of more than just vectors, we can also check for the linear independence of things other than vectors.
For one last example, let’s take a set of polynomials: x^2 + x - 2, x^2 - 3x + 5, and 2x^2 + 6x - 11. Just like checking for independence of any elements, we want to multiply each by a scalar and set the sum equal to zero. The most efficient way to proceed will involve grouping like terms in powers of x, so we take the x squared terms and make this, then we do the same for the x terms, and then for the constants.
If this seems confusing, you can also just distribute the scalars, group up like terms, and factor out the x terms if you prefer. Now, for this equation to be equal to zero for all x, each coefficient must also be zero, so this first coefficient must be set equal to zero, and then the second, and then the third. This gives us three new equations involving our unknown scalars.
While it did take a few extra steps, we once again have a system of equations to solve just like we have been doing. In this case, once we set up the coefficient matrix, we see that it is a square matrix, so if we want to use our new trick, all we have to do is find out if the determinant is zero, which would make the polynomials linearly dependent, or if the determinant is nonzero, making the polynomials linearly independent. Going through and solving for the determinant, a calculation which we will show all at once for our purposes here, we find that it is equal to zero, and the polynomials are linearly dependent.
So vector spaces and related concepts certainly may seem abstract, and they are, but they are useful because they allow us to unify things like functions, vectors, and matrices, so that we can use the same set of tools to answer certain questions. Questions like: Can this object be written in terms of other objects? This type of thinking, abstraction, and unification, is prevalent in advanced mathematics, so we should probably just get used to it now.
With linear independence understood, we can move on and combine this with the concept of span to form another key concept. But first, let’s check comprehension.
