3. Multiplication and Inverse Matrices

the topic I've been talking I've been multiplying matrices already but certainly time for me to discuss the rules for matrix multiplication and the interesting part is the many ways you can do it and they all give the same answer so it's and they're all important so matrix multiplication and then uh come inverses so we're we we mentioned the inverse of a matrix but there's that's a big deal lots to do about inverses and how to find them okay so I'll begin with how to multiply two matrices first way okay so suppose I have a matrix

a multiplying a matrix B and giving me a result well I could call it C A * B okay uh so let me just review the rule for for this entry that's the entry in row I and column J so that's the IJ entry right there is CI J we always write the row number and then the column number so I might I might maybe I take it c34 just to make it specific so instead of i j let me use numbers c34 so where does that come from the 34 entry it comes from Row

three here Row three and column four as you know column four and can I just write down or can we write down the formula for it c34 is if we look at the whole row and the whole whole column the quick way for me to say it is Row 3 of a I could use a DOT for dotproduct I won't often use that actually dot column 4 of B and but this gives us a chance to just like use a little Matrix not ation what are the entries what's this first entry in Row three the

first the first that number that's sitting right there is a so it's got two indices and what are they 31 so there's an a 31 there now what's the first guy at the top of column 4 so what what's sitting up there B4 four right so that this dot product starts with a 31 * b14 and then what's the next so this is like I'm accumulating this sum then comes the next guy a32 second column times B24 second row so it's B a32 B24 and so on just practice with indices oh let let me even

practice with the a summation formula so this is I I most of the course I use whole vectors I very seldom uh get down to the details of these particular entries but here we better do it so I'm it's some kind of a sum right of things in Row three column K shall I say times things in row K column 4 you see that that's what's that's what we're seeing here this is K is one here K is two on along so up so the sum goes all the way along the row and down the

column say one to n so that's what the a the c34 entry looks like like a sum of a 3 k b K4 just takes a little practice to do that okay and uh oh well maybe I should say when are we allowed to multiply these matrices what are the shapes of these things the shapes are uh if we allow them to be not necessarily Square matrices if they're Square they've got to be the same size if they're rectangular they're not the same size if they're rectangular this might be well I always think of a

as m byn m rows n columns so that sum goes to n now what's the point how many rows does B have to have n n the number of rows in B the number of guys that we meet coming down has to match the number of ones across so B will have to be n by something whatever P so the the number of columns here has to match the number of rows there and then what's the result what's the shape of the result what's the shape of C the output well it's got these same M

rows or it's got M rows and how many columns p m by P okay so they're m time P Little Numbers in there entries and each one looks like that okay so that's the standard rule that's the way people think of multiplying matrices um I do it too but that's that's I I want to talk about other ways to look at that same calculation uh looking at whole columns and whole rows okay so can I do ab C again a equaling C again but now tell me tell me about uh yeah let me I'll put

it up here so here goes a again time B producing C and again this is is M byn this is n by P and this is M by P okay now I want to look at whole columns I I want to look at The Columns of in fact here's the second way to multiply matrices cuz I'm going to build on what I know already how do I multiply a matrix by a column how do I I I know how to multiply this matrix by that column shall I call that column one that tells me column

one of the answer the Matrix times the First Column is that First Column cuz none of this stuff entered that part of the answer the Matrix times the second column is the second column of the answer do you see what I'm saying that I could think of multiplying a matrix by a vector which I already knew how to do and I can think of the V I can think of just P columns sitting side by side just like resting next to each other and I multiply a times each one of those and I get the

P Columns of the answer do you see this is this is quite nice to be able to think okay matrix multiplication works so that I could just think of having several columns multiplying by a and getting The Columns of the answer so like here's column one I call that here's a shall I call that column one and what's going in there is a * column 1 okay so that's the picture a column at a time so what does that tell me what does that tell me about these columns these Columns of c are combinations as

we've seen that before of columns of a every one of these comes from a * this and a * a vector is a combination of The Columns of a and right and it makes sense because The Columns of a have length M and The Columns of C have length M and every column of C is a is some combination of The Columns of a and it's these numbers in here that tell me what combination it is do you see that that that out that in that answer C I'm seeing stuff that's col that's combinations of

these columns now suppose I look at it that's two ways now the Third Way is look at it by rows so now let me change to rows okay so now I can think of a row of a a a row of a multiplying all these rows here and producing a row of the the product so this row takes a combination of these rows and that's the answer so these rows of c are combinations of what of tell me about to finish that the rows of C when I when I I have a matrix B it's

got its rows and I multiply by a and what does that do it mixes the rows up it makes it creates combinations of the rows of B thanks rows of B that's what I wanted to see that this that this answer I can see where the pieces are coming from the rows in the answer are coming as combinations of these rows the columns in the answer are com coming as combinations of of those columns and now that so that's three ways now you can say okay what's the fourth way the fourth way so that's now

we've got like the regular way the column way the row way and what's left uh the the one that I can I I want to tell you about well one way is columns times rows what happens if I multiply so this was row time column it gave a number okay now I want to ask you about column times row what does If I multiply a column of a times a row of B what shape am I ending up with so if I take a column times a row that's definitely different from taking a row times

a column so a column of a was what what's the shape of a column of a m by one a column of a is is a column it's got M entries and one coln and what's a row of B it's got one row and P column so what's the shap what do I get if I multiply a column by a row I get a big Matrix I get a fullsize matrix If I multiply a column by a row I get should we just do one let me take the the column 2 3 4times the Row

1 6 that is a that product there I I'm just following the rules of matrix multiplication those rules are just looking like kind of petique kind of small because the the rows here are so short and the columns there are so short but they're the same length one in so what's the answer what's the answer if I do 234 * 1 16 just for practice well what's the first row of the answer 212 and the second row of the answer is 318 and the third row of the answer is 424 actually what am I I

mean that's a very special Matrix there very special Matrix what can you tell me about its columns The Columns of that Matrix they're multiples of this guy right they're multiples of that one which follows our rule we said that the columns of the answer were combinations but there's only to take a combination of one guy it's just a multiple the rows of the answer what can you tell me about those three rows they're all multip multiples of this row they're all multiples of six as we expect but I'm getting a full siiz Matrix and now

just to complete this thought if I have uh now let me write down the Fourth Way AB is a sum of columns of a times rows of B so that for example if my if my Matrix was 2 3 4 and then had another column say 7 8 9 and my Matrix here has say started with six and then had another column like 0 0 then here's the fourth way okay I've got two columns there I've got two rows there so the beautiful rule is seeing the whole thing by columns and rows is that I

can take the First Column times the first row and add the second column times the second row so that's the fourth way that that I can take columns times rows First Column time first row second column time second row and add actually what will I get what will the answer be for that matrix multiplication well this one is just going to give us zero so in fact I'm back to this that's the answer for that matrix multiplication I'm I'm sort of like happy to put up here these facts about matrix multiplication because it gives me

a chance to write down special matrices like this this is a special Matrix uh all those rows lie on the same line all those rows lie on the line through one six if I draw a picture of all these row vectors they're all the same direction if I draw a picture of these two column vectors they're in the same direction later I would use this language not too much later either I would say the row space which is like all the combinations of the rows is just a line for this Matrix the row space is

the line through the vector six all the rows lie on that line and the column space is also a line all the columns l y on the line through the vector 2 3 4 so this is like a really minimal Matrix and it's because of these ones okay so that's a third way now even yeah I can I when you will you take this is I I want to say one more thing about matrix multiplication while we're on this subject and it's this you could also multiply you can also cut the Matrix into blocks and

do the multiplication by blocks yeah that that's actually so uh useful that I that I want to uh mention it block multiplication so I could take my Matrix a and I could chop it up like maybe just for Simplicity let me chop it into two into Four Square block suppose it's Square let's just take a nice case and B suppose it's Square also same size though these sizes don't have to be the same what they have to do is match properly here they certainly will match so here's the rule for Block multiplication that if this

has blocks like uh uh a so maybe A1 A2 A3 A4 are the blocks here and these blocks are B1 B2 B3 and B4 then the answer I can find block I can find that block and if you tell me what's in that block then I'm going to be quiet about matrix multiplication for the rest of the day what goes into that block you see these might be this Matrix might be these matrices might be like 20 by 20 with blocks that are 10 by 10 to take the easy case where all the blocks are

the same shape and the point is that I could multiply those by blocks and what goes in here what's that what's that what's that block in the answer A1 B1 that's a matrix times a matrix it's the right size 10 by 10 anymore Plus what's the what what what else goes in there A2 B3 right it's just like block rows times block columns I don't nobody I think not even gaus could see instantly that it works but somehow if we check it through all five ways were're doing the same multiplications so this this familiar multiplication

is what we're really doing when we do it by columns by rows by columns times rows and by blocks okay I just have to like get the rules straight for matrix multiplication okay all right I'm ready for the second topic which is inverses okay ready for inverse and let me do it for square matrices first okay so I've got a square Matrix a and and it may or may not have an inverse right not all matrices have inverses in fact like that's the most important question you can ask about the Matrix is if it's square

if you know it's square is it invertible or not if it is invertible then then there is some other Matrix shall I call it a inverse and what's the what's if a inverse exists so this is if there's a big if here if this Matrix exists and and it'll be really Central to figure out when does it exist and then if it does exist how would you find it but what's the what's the the equation here that I haven't uh that I have to finish now this Matrix if it exists multiplies a and produces I

thanks the identity and actually there's a little more to it because normally I mean that right now that's like a left inverse it's sitting on the left of X but a real an inverse for a square Matrix could be on the right as well so so uh this is true too that it if I have a yeah in fact this is not this is probably the this is something that's not easy to prove but it works that a left that for square matrices a left inverse is also a right inverse if I can find a

matrix on the left that gets the identity then also that Matrix on the right will produce that identity for rectangular matrices we'll see a left inverse that isn't a right inverse in fact the shapes wouldn't allow it but for square matrices the shapes allow it and it happens if a has an inverse okay so give me some cases let's see I hate to be negative here but let's talk about the case with no inverse so so this is this is these matrices are called invertible or non singular those are the good ones and we want

to be able to identify how if we're given a matrix has it got an inverse can I talk about the singular case no inverse all right uh best to start with an example tell me an example let's let's get an example up here let's make it 2x two of a matrix that has not got an inverse and let's see why let me let me write one up no inv let's let's see why let me write up 1326 why does that Matrix have no inut you could you could answer that various ways uh Give Me One

Reason well you could if you know about determinant which you're not supposed to uh you could take its determinant and you would get zero okay now all right let me let me ask you other reasons let me ask for other reasons that that Matrix isn't invertible here I could use use what I'm saying here suppose um suppose a times some other Matrix gave the identity why is that not possible because yeah tell I'm I'm thinking about columns Here If I multiply this Matrix a by some other Matrix then the the the result what can you

tell me about the column they're all multiples of those columns right if I multiply a by another Matrix that the product has columns that come from those columns so can I get the identity Matrix no way The Columns of the identity Matrix like one Z it's not a combination of those columns because those two columns lie on the both lie on the same line every combination is just going to be on that line and I can't get one zero so do you see do you see that that sort of column picture of the Matrix not

being invertible in fact here's another reason oh this is even a more important reason well how can I say more important all those are important this is another way to see it a matrix has no inverse yeah here now this is important a matrix has no a square Matrix won't have an inverse if there's if no inverse because I can solve I can find an x a a a vector x with a * this a * X giving zero that this is the reason Reon I like best that Matrix won't have an inverse can you

well let me change I to U so tell me a vector X that uh solves ax equals z i mean this is like the key equation in mathematics all the key equations have zero on the right hand side so what's the s tell tell me an X here so now I'm going to put slip in the x that you tell me and I'm going to get zero what what x would do that job 3 and negative 1 is that the one you picked or yeah or or another Well if you pick zero and zero I'm

not so excited right CU that would always work so it's all it's really the fact that this Vector isn't zero is important it's a nonzero vector and 31 would do it that just says three of this column minus one of that column is the zero column okay so now I know that uh a couldn't be invertible what what's the what's the reasoning if if ax is zero suppose I multiplied by a inverse yeah here's the reason here here this is why This spells disaster for an inverse a matrix can't have an inverse if some combination

of the columns gives gives nothing because I could take axal z i could multiply by a inverse and what would I discover suppose I take that equation and I multiply by I if a inverse existed which of course I'm going to come to the conclusion it can cuz if it existed if there was an a inverse to this Dopey Matrix I would multiply that equation by that inverse and I would discover X is zero If I multiply a by a inverse on the left I get X If I multiply by a inverse on the right

I get zero so I would discover X was Zero but if X is not Zer X this guy wasn't zero there it is 3 - one so conclusion only take us some time to really work with that conclusion our conclusion will be that that in that that noninvertible matrixes singular matrices some combinations of their some combination of their columns gives the zero column they they take some Vector X into zero and there's no way a inverse been recovered right that's what this equation says this equation says I take this Vector X and multiplying by a

gives zero but then when I multiply by a inverse I can never escape from zero so there couldn't be an a inverse where here okay now fix all right now let me take a all right back to the positive side let's take a Matrix that does have an inverse and why not invert it okay can I so let me take on this third board a matrix so I fix that up a little uh tell me a matrix that has got an inverse well let me say 1 3 2 what should I put there well don't

put six I guess right you want any any favorites here one or eight I don't care seven seven okay seven is a lucky all right seven okay okay so now what's our idea we believe that this Matrix is inverted those who like determinants have quickly taken its determinant and found it wasn't zero those are like columns and probably that that department is not totally popular yet but those are like columns we'll look at those two columns and say hey they point in different directions so I can get anything now let let me see what do

I mean how am I going to compute a inverse so a inverse here's a inverse and I have to find it and and what do I get when I'm when I do this multiplication the identity you know forgive me for taking 2 by two but like it's good to keep the computations manageable and let the ideas come out Okay now what's the idea I want I'm looking for this Matrix a inverse how am I going to find it right now it's uh I've got four numbers to find I'm going to look at the First Column

let me take this First Column a what's up there what equation yeah tell me this what equation does the First Column satisfy the First Column satisfies a * that column is 1 Z the First Column of the answer and the second column CD satisfies a * that second column is 01 do you see that finding the inverse is like solving two systems one system when the right hand side is one zero I'm just going to split it into two pieces that I don't I don't even need to rewrite it I can I can take a

time so let me put it here A times column J of a inverse is column J of the identity I've got in equations I've got well two in this case and they have the same Matrix a but they have different right hand sides the right hand sides are just The Columns of the identity this guy and this guy and these are the two solutions do you do you see what I'm doing I'm looking at that equation by columns I'm looking at a * this column giving that guy and a * that column giving that guy

so essentially so this is like the gaus we're back to gaus we're back to solving systems of equations but we're solving we've got two right hand sides instead of one that's where Jordan comes in so so at the very beginning of the lecture I mentioned gal Jordan let me write it up again okay here's the Gus Jordan idea g j is solves solve two equations at once okay let me show you how the how the mechanics go H how do I solve a a single equation so the two equations are uh 1327 multiplying a gives

1 Z and the other equation is the same 1327 multiplying CD gives 01 okay that'll tell me the two columns of the inverse I'll have the inverse in other words if I can solve with this Matrix a if I can solve with that right hand side and that right hand side I'm inverted I've got it okay and the and Jordan sort of said to G solve them together look at the Matrix if we just solve this one I would look at 13 27 and how do I deal with the right hand side I stick it

on as an extra column right you remember that's that's this augment Matrix that's the Matrix when I'm watching the right hand side at the same time doing the same thing to the right side that I do to the left so I just carry it along as an extra column now I'm going to carry along two extra columns and I'm going to do whatever gal wants right I'm going to do elimination I'm going to get this to be simple and this this thing will turn into the inverse this is what's coming I'm going to do elimination

steps to make this into the identity and lo and behold the inverse will show up here let's do it okay so what are the elimination steps so you see that here's my Matrix a and here's the identity like stuck on augmented on suppos to be switched did I oh no they weren't supposed to be switched sorry thanks okay thank you very much and there I've got them right okay thanks okay so let's do elimination all right it's going to be simple right so I take two of this row away from this row so this row

stays the same and two of those come away from this that leads me with a zero and a one and two of these away from this is that what that is that what you're getting after one elimination step I let me sort of separate the the left half from the right half so two of that first row got subtracted from the second row now now this is an upper triangular form gaus would quit but Jordan says keep going you use elimination upwards subtract a multiple of equation two from equation one to to get rid of

the three so let's go the whole way so now I'm going to this guy is fine but I'm going to what do I do now what's my final step that that produces the inverse I multiply this by the right number to get up to there to to to remove that three so I guess since since this is a one there's the pivot sitting there I multiply it by three and subtract from that so what do I get I'll have one zero oh yeah that was my whole point I'll multiply this by three and subtract from

that which will give me 7 and I multiply this by three and subtract from that which gives me a minus three and what's my hope belief here here I started with with a and the identity and I ended up with the identity and who that better be a inverse that's the gaus Jordan idea start start with this long Matrix double length AI eliminate eliminate until this part is down to I then this one will must be for some reason and we got to find the reason must be a inverse shall I just check that it

works let me just check that can I multiply this Matrix this this part times a I'll carry a over here here and just do that multiplication you'll see I'll do it the oldfashioned way 7 - 6 is a 1 21 - 21 is a 0 - 2 + 2 is a 0 - 6 + 7 is a 1 check so that is the inverse that's the gaus Jordan idea so you'll one of the homework problems or more than one uh for Wednesday will ask you to go through those steps I think you just got to

go through gaus Jordan a couple of times but so I like yeah just to see the mechanics but the important thing is why is like what happened why did we why did we get a inverse let me ask you that [Music] we got so we take we we do row reduction we do elimination on this long Matrix AI until the first half is up then the second half is a inverse well how do I see that let me put up here how I see that so here's my here's my Gus Jordan thing and and I'm

doing stuff to it so I'm well whole lot of ease remember those are those elimination Matrix those are the those are the things that we figured out last time yes that's what an elimination step is is it's it's in Matrix form I'm multiplying by some e and the result well so I'm multiplying by a whole bunch of e so I get a can I call the overall Matrix e that's the elimination Matrix the product of all those little pieces what do I mean by little pieces well there was an elimination Matrix that subtracted two of

that away from that then there was an elimination Matrix that subtracted three of that away from that I I guess in this case that was all so there were just two e in this case one that did this step and one that did this step and together they give me an e that does both steps and the net result was to get an I here and you can tell me what that has to be this is like the the picture of what happened if e multiplied a whatever that e is we never figured it out

in in by by by in this way but whatever that e * that e is e * a is what's E * a it's a that e whatever the heck it was multiplied a and produce I so e must be EA equally I tells us what e is namely it is it's the inverse of a great and therefore when the second half when e multiplies I it's e but that's a inverse you see the the picture looking that way e * a is the identity that tells us what e has to be it has to

be the inverse and therefore on the right hand side where e where we just smartly tucked on the identity it's turning in step by step it's turning into a invers there is the the statement of G Jordan elimination that's how you find the inverse we're we look we can look at it as elimination at solving n equations at the same time and packing on N columns solving those equations and up shows the end Columns of a okay thanks St ready