Hello everyone.

My maths class has begun a topic on general vector spaces and to my astonishment, we have leapt into an area of linear algebra, which is quite important, far too quickly for my liking.

So I am in need of some more great explanations!! (please).

1. Subspaces
Firstly, subspaces seem fairly trivial to me. (obviously I am not getting it!). They are defined by being closed under scalar multiplication and vector addition. So my question is...

Is it enough to show that a subspace exists for a particular set of vectors by performing these two operations? What do they mean geometrically?

2. Span
Secondly, when we talk about spanning vectors, does this just mean to say a set of vectors which span a subspace? And just say you have three vectors...
u = <1, 1, 2>
v = <1, 0, 1>
w = <2, 1, 3>
...which we have find out whether they span R^3.
Is it enough to just show that they can be expressed as a linear combination? Does this show they span R^3 or is there more to it?

3. Linear Independence
What is this??? It seems to me that a set of vectors IS linearly dependent iff they can be expressed as a linear combination of the other vectors in that set, and obviously independant if it cannot. To me this seems trivial. Can someone give me a practical use for linear independence and what important concepts can be made clearer by knowing what this means.

4. Basis
Lastly: Now I was lost in today's lecture about basis. All I know is that a set of vectors are bases of a vector space if they are linearly independent AND spans the vector space accordingly. Now my question is, why would anyone use this? Sure you would be able to represent higher dimensional vector spaces in terms of their bases but what real use does this have? How would you go about determining a basis for a given set of vectors?

As far as I am concerned this is just about writing down long vectors and adding and multiplying them together and not really getting anywhere. (If you haven't guessed by now, I am really not getting this! However, I really would love to know :) :)

Thankyou very much in advance for any help or a better explanation that anyone can provide.

Originally posted by oxymoron
1. Subspaces
Firstly, subspaces seem fairly trivial to me. (obviously I am not getting it!). They are defined by being closed under scalar multiplication and vector addition. So my question is...

Is it enough to show that a subspace exists for a particular set of vectors by performing these two operations? What do they mean geometrically?

yes, given a particular set of vectors, if they are closed under addition and scalar multiplication, that is sufficient to show they form a subspace.

here is the picture you should have: think of your vector space as the set of all arrows in R³. subspaces of this vector space include any plane through the origin in R³, any line through the origin, and the origin all by itself.

you will soon have to deal with much more general vector spaces than R³, but you will get all the intuition about how vector spaces work from this simple picture. (unless you start dealing with infinite dimensional vector spaces)

it is easy to show that if you add two vectors in a particular line or plane, you get a new vector still in that line or plane. likewise with scalar multiplication. therefore, by definition, they are subspaces.

vector spaces are sometimes called linear spaces, because they always look flat, like the line or the plane.

2. Span
Secondly, when we talk about spanning vectors, does this just mean to say a set of vectors which span a subspace? And just say you have three vectors...
u = <1, 1, 2>
v = <1, 0, 1>
w = <2, 1, 3>
...which we have find out whether they span R^3.
Is it enough to just show that they can be expressed as a linear combination? Does this show they span R^3 or is there more to it?
no, this is not enough. any vector can be expressed as a linear combination of the standard basis vectors (or any other basis for that matter).

for example, if your vectors were <1,1,1>, <2,2,2> and <-1,-1,-1> these vectors do not span R³. they are all in a line, so you can add them and multiply them as much as you want, and your result will always be in that same line.

the vectors span a space if, when you total up all the vectors you get by adding and multiplying by scalar your starting vectors, you get the whole space in question.

even if your vectors do not span the whole space, the set of all vectors that you get by adding and multiplying is still a vector subspace, and you call this subspace the span of those vectors.


3. Linear Independence
What is this??? It seems to me that a set of vectors IS linearly dependent iff they can be expressed as a linear combination of the other vectors in that set, and obviously independant if it cannot. To me this seems trivial. Can someone give me a practical use for linear independence and what important concepts can be made clearer by knowing what this means.
hmm... a set of vectors is linearly independent iff you cannot add them up in any nontrivial way and get zero.

in R³, three vectors are linearly independent if and only if they are not all in the same plane. you need to specify exactly two vectors to determine a plane. if i give you a third vector in that plane, it is redundant. it is linearly dependent on the other two.

likewise, if i give you two vectors that are colinear, they must be linear dependent. there is only one degree of freedom in a line (through the origin), it only takes one point to determine that line. therefore i cannot fit two linear independent vectors in a line.

you can think of linear independence as a sort of generalization of perpendicularity. not that independent vectors are always perpendicular to each other, but they do at least have some component that is perpendicular to any other vector. like, if i give you one vector, it determines a line. a linearly independent vector doesn t have to be perpendicular to this line, but it must at least have some component that is perpendicular. it can t point in the same direction.

now you have 2 linearly independent vectors, they determine a plane. if i want to give you a third linearly independent vector, it doesn t have to be perpendicular to the plane, but it must have some component which is. it cannot be coplanar with the other two. if it is coplanar, even if it doesn t point in the same direction as either of the first two vectors, it cannot be linearly independent.

a set of vectors that truly is perpendicular to each other must necessarily be independent.

i m not sure exactly what you mean with this sentence: "iff they can be expressed as a linear combination of the other vectors in that set,". any vector can be expressed as a linear combination of other vectors, always. so this statement contains no information about the linear independence (or lack thereof) of these vectors.

4. Basis
Lastly: Now I was lost in today's lecture about basis. All I know is that a set of vectors are bases of a vector space if they are linearly independent AND spans the vector space accordingly. Now my question is, why would anyone use this? Sure you would be able to represent higher dimensional vector spaces in terms of their bases but what real use does this have? How would you go about determining a basis for a given set of vectors?

As far as I am concerned this is just about writing down long vectors and adding and multiplying them together and not really getting anywhere. (If you haven't guessed by now, I am really not getting this! However, I really would love to know :) :)

Thankyou very much in advance for any help or a better explanation that anyone can provide. [/B]
you don t really determine a basis. you just pick one out of a hat. it is a lot like choosing coordinates in a physics problem. in a physics problem, you have to choose which way you want to call the x-, y-, and z-axis. choosing a basis is the same thing. only the basis vectors don t have to be perpendicular. just linearly independent, which, as i said, is a sort of generalization.

linear independence is that property of a set of vectors which guarantees you the ability to express any vector in the space uniquely as a combination of the starting vectors. this is very important. you wouldn t want to do a physics problem where there were an infinite number of ways to write each point as an (x,y,z) triple.

you also want the basis vectors to span the space, otherwise you would have a whole bunch of points in the space that can t be expressed in terms of the basis vectors at all. now that s just no good. if your basis vectors were in the x-y plane, then you can t write anything along the z-axis in terms of the basis vectors.

so, that the basis spans the space guarantees you that you can write any vector in the space as a combination of the basis vectors. that the basis vectors are linearly independent guarantees you that you can do so uniquely.

hope that helps.

To say that a set of vectors "spans" a space means that every vector can be written as a linear combination of vectors in that set.

To say that a set of vectors is "linearly independent" means that each vector can be written in at most ONE way as a linear combination of vectors in that set.

A basis for a space is a set that both spans it AND is linearly independent.

The whole point of a basis is that any vector space contains an infinite number of vectors. A basis for a (finite-dimensional) vector space contains only a finite number of vectors- we need only look at those instead of all.

It is much like being able to differentiate something like
3x^4- 3x^3+ 7x^2+ 4x+ 7 by just learning one power law rather than having to learn a different law for each polynomial.

:eek: Wow, Lethe, thanks a lot for your efforts!!! It helps a great deal when you hear it explained another way. Thankyou.

However, still some questions. :)

2. for example, if your vectors were <1,1,1>, <2,2,2> and <-1,-1,-1> these vectors do not span R3. they are all in a line, so you can add them and multiply them as much as you want, and your result will always be in that same line.

Are you saying that since the only other vectors you can get by adding and multiplying these three vectors together all line up in the same line and therefore do not span R3. Do they span R2 consequently??

In this case would the minimal number of vectors needed to span R3 by two? Since if ideally you had two vectors which spanned a plane in R3 be enough to show that the plane is a subspace of R3?? (I hope so, otherwise I am still lost! :confused: )

3. in R3, three vectors are linearly independent if and only if they are not all in the same plane. you need to specify exactly two vectors to determine a plane. if i give you a third vector in that plane, it is redundant. it is linearly dependent on the other two.

Okay, I get that linearly independent means 'geometrically' that for R3 the set of two vectors do not lie on the same line or in the same plane. Is this because if you scalar multiply them you simply get an elongated version of your original line? and hence linearly dependent. If you scalar multiply them and get a line that does not lie in the same plane (for R3 of course) then it sticks up or down out of the plane and then is linearly independent. I think you explained this well. Much appreciated!

4. you don t really determine a basis. you just pick one out of a hat. it is a lot like choosing coordinates in a physics problem. in a physics problem, you have to choose which way you want to call the x-, y-, and z-axis. choosing a basis is the same thing. only the basis vectors don t have to be perpendicular. just linearly independent, which, as i said, is a sort of generalization.

Okay say our regular coordinate system for physics is the unit vectors i, j and k with unit length so...
i = <1, 0, 0>
j = <0, 1, 0>
k = <0, 0, 1>
..Is this a basis for R3 (as you said you can pick any one out of a hat) this one seems fairly basic to me!. This looks like they are also all linearly independent of each other because they all lie in separate planes!!! Does this mean anything??

i m not sure exactly what you mean with this sentence: "iff they can be expressed as a linear combination of the other vectors in that set,". any vector can be expressed as a linear combination of other vectors, always. so this statement contains no information about the linear independence (or lack thereof) of these vectors.

I mean (take the example just above) that because i, j and k can be written as:

a<1, 0, 0> + b<0, 1, 0> + c<0, 0, 1> = ai + bj + ck

...this is a linear combination.

Say you had...
u = <1, 2, 1>
v = <2, 9, 0>
w = <3, 3, 4>

b = c<1, 2, 1> + d<2, 9, 0> + e<3, 3, 4>
b = (c + 2d + 3e, 2c + 9d + 3e, c + 4e)
Show this system has a solution...
.....| 1 2 3 |
A =| 2 9 3 |
.....| 1 0 4 |


detA = -1

Since the three vectors can be expressed as a linear combination AND has a nonzero determinant this shows that they are linearly dependent AND span R3.

Does this seem correct to you?

so, that the basis spans the space guarantees you that you can write any vector in the space as a combination of the basis vectors. that the basis vectors are linearly independent guarantees you that you can do so uniquely.

Exactly!

Cheers. Ben.

Originally posted by oxymoron
:eek: Wow, Lethe, thanks a lot for your efforts!!! It helps a great deal when you hear it explained another way. Thankyou.

However, still some questions. :)

2.

Are you saying that since the only other vectors you can get by adding and multiplying these three vectors together all line up in the same line and therefore do not span R3.
correct.
Do they span R2 consequently??No! R2 is a plane, these vectors are all in a line, so they only span a 1-dimensional subspace!

In this case would the minimal number of vectors needed to span R3 by two? Since if ideally you had two vectors which spanned a plane in R3 be enough to show that the plane is a subspace of R3?? (I hope so, otherwise I am still lost! :confused: )
3 is the number of vectors you need to span R3! that s why its called R3! by definition, the dimension of a vector space is the number of vectors needed to make a basis for that space. R3 is 3-dimensional, so you need 3 vectors.

2 vectors span a plane in R3. this is sufficient to show that the plane is a subspace of R3 (but it is not all of R3, thus 2 vectors do not span R3). in general, the span of any set of vectors is always a subspace.

3.

Okay, I get that linearly independent means 'geometrically' that for R3 the set of two vectors do not lie on the same line or in the same plane. Is this because if you scalar multiply them you simply get an elongated version of your original line? and hence linearly dependent.
yes.
If you scalar multiply them and get a line that does not lie in the same plane (for R3 of course) then it sticks up or down out of the plane and then is linearly independent.
no! you cannot scalar multiply a vector and get an independent vector. if you scalar multiply a vector in a plane, it will always remain in that plane. this is because all scalar multiplication does is stretch vectors, not rotate.
I think you explained this well. Much appreciated!
you re welcome.

4.

Okay say our regular coordinate system for physics is the unit vectors i, j and k with unit length so...
i = <1, 0, 0>
j = <0, 1, 0>
k = <0, 0, 1>
..Is this a basis for R3 (as you said you can pick any one out of a hat) this one seems fairly basic to me!
yes, this is a basis. if it seems basic to you, good. it should.

. This looks like they are also all linearly independent of each other because they all lie in separate planes!!! Does this mean anything??
you need to be a little careful here. it doesn t make sense to say that three vectors lie in three different planes. it takes two vectors to determine a plane. but essentially you ve got the idea.

to put it a little more accurately, you should say that these vectors are independent because each of the 3 vectors lies outside the plane determined by the remaining 2.



I mean (take the example just above) that because i, j and k can be written as:

a<1, 0, 0> + b<0, 1, 0> + c<0, 0, 1> = ai + bj + ck

...this is a linear combination.
yes.

Say you had...
u = <1, 2, 1>
v = <2, 9, 0>
w = <3, 3, 4>

b = c<1, 2, 1> + d<2, 9, 0> + e<3, 3, 4>
b = (c + 2d + 3e, 2c + 9d + 3e, c + 4e)
Show this system has a solution...
.....| 1 2 3 |
A =| 2 9 3 |
.....| 1 0 4 |


detA = -1

Since the three vectors can be expressed as a linear combination AND has a nonzero determinant this shows that they are linearly dependent AND span R3.

Does this seem correct to you?
yes, this seems correct to me.

Originally posted by oxymoron

Say you had...
u = <1, 2, 1>
v = <2, 9, 0>
w = <3, 3, 4>

b = c<1, 2, 1> + d<2, 9, 0> + e<3, 3, 4>
b = (c + 2d + 3e, 2c + 9d + 3e, c + 4e)
Show this system has a solution...
.....| 1 2 3 |
A =| 2 9 3 |
.....| 1 0 4 |


detA = -1

Since the three vectors can be expressed as a linear combination AND has a nonzero determinant this shows that they are linearly dependent AND span R3.

Does this seem correct to you?

to make it a little more precise what is going on, with this solution, let s recall that the definition of linear independence is that you cannot sum the vectors to zero unless c, d, and e are all zero.

so you have a matrix equation. A(c,d,e)^T=(0,0,0)^T. if A has nonzero determinant, then A is invertible, and you can multiply by A^-1 to get (c,d,e)=(0,0,0), and therefore the vectors are independent. if not, if the determinant is zero, then they are dependent.

3 is the number of vectors you need to span R3! that s why its called R3! by definition, the dimension of a vector space is the number of vectors needed to make a basis for that space. R3 is 3-dimensional, so you need 3 vectors.

Got it. Happy with that!

QUOTE]no! you cannot scalar multiply a vector and get an independent vector. if you scalar multiply a vector in a plane, it will always remain in that plane. this is because all scalar multiplication does is stretch vectors, not rotate.[/QUOTE]

Oops!

Thanks again Lethe and HallsofIvy for your concise explanations and taking the time to read my questions (and making prompt replies!!). I think that wraps it up! (until next week!)

See ya.

Ben.

Question

A system of simultaneous linear equations can be written in matrix form as Ax = b.

Prove that if x0 is a particular solution to Ax = b, then the set of solutions of Ax = b is...

S = {x = x0 + s : s is an element of S0}

...where S0 is the solution space of the corresponding homogeneous system.

Answer (correct me if I am wrong)

Ax = b is a non-homogeneous system.

Writing in matrix form (which I won't do here!)

Ax = c1x1 + c2x2 + ...

This is a linear combination of the column vectors of A. To be consistent with Ax = b, the column vector b must also span the column vectors of A. Thus Ax = b only works if b lies in the column space of A.

Is it correct to say that the system has a unique solution?

--> Answer <---
Suppose x0 and s are both solutions to Ax = b.
Then set x = x0 + s
Then x satisfies...

Ax = A(x0 + s) = Ax0 + As = b + b = 2b

Is this closed under scalar multiplication? If so, would b be a solution to Ax = b and hence x0 and s be elements of the solution space S closed under addition?

that solution looks totally wrong to me.

how about this. let x be any particular solution. let y be any other solution.

then y = x + (y - x)

A(y-x)=b-b=0 so y-x can is in the solution space of the corresponding homogeneous equation.

qed.

Originally posted by oxymoron

Is it correct to say that the system has a unique solution?
this is true if and only if A is invertible (in which case, the homogenous equation has no nontrivial solution, and this whole problem becomes trivial. just multiply both sides of the equation by A^-1 to get the solution space)
x0 and s are both solutions to Ax = b.
Then set x = x0 + s
Then x satisfies...

Ax = A(x0 + s) = Ax0 + As = b + b = 2b
[/b]
As=0, remember?

Is this closed under scalar multiplication? If so, would b be a solution to Ax = b and hence x0 and s be elements of the solution space S closed under addition?
the set of solutions to the inhomogeneous equation is not a vector space, not closed under addition, etc.

Thankyou lethe.

Say you have a matrix A. If you reduce it to row-echelon form and find that it has say three non-zero rows and three is it safe to declare that the row-space is three-dimensional?

Is this also rank = 3?

Take the same matrix but this time we want to find the nullity. Does this involve finding the dimension of the solution space? If yes, could this mean simply finding a solution to Ax = 0?

Okay, so you solve the matrix and find that you only need four vectors a, b, c, and d to be able to make any other linear combination of vectors. Hence a basis of the solution space has been found. Does this straight away imply that the nullity is four as well?