From the series: Differential Equations and Linear Algebra
Gilbert Strang, Massachusetts Institute of Technology (MIT)
A and B are “similar” if B = M-1AM for some matrix M. B then has the same eigenvalues as A.
OK, thanks. Here's a second video that involves the matrix exponential. But it has a new idea in it, a basic new idea. And that idea is two matrices being called "similar." So that word "similar" has a specific meaning, that a matrix A, is similar to another matrix B, if B comes from A this way. Notice this way. It means there's some matrix M-- could be any invertible matrix. So that I take A, multiply on the right by M and on the left by M inverse. That'd probably give me a new matrix. Call it B. That matrix is called "similar" to B. I'll show you examples of matrices that are similar. But first is to get this definition in mind.
So in general, a lot of matrices are similar to-- if I have a certain matrix A, I can take any M, and I'll get a similar matrix B. So there are lots of similar matrices. And the point is all those similar matrices have the same eigenvalues. So there's a little family of matrices there, all similar to each other and all with the same eigenvalues. Why do they have the same eigenvalues? I'll just show you, one line.
Suppose B has an eigenvalue of lambda. So B is M inverse AM. So I have this. M inverse AMx is lambda x. That's Bx. B has an eigenvalue of lambda. I want to show that A has an eigenvalue of lambda. OK.
So I look at this. I multiply both sides by M. That cancels this. So when I multiply by M, this is gone, and I have AMx. But the M shows up on the right-hand side, I have lambda Mx. Now I would just look at that, and I say, yes. A has an eigenvector-- Mx with eigenvalue lambda. A times that vector is lambda times that vector. So lambda is an eigenvalue of A. It has a different eigenvector, of course. If matrices have the same eigenvalues and the same eigenvectors, that's the same matrix. But if I do this, allow an M matrix to get in there, that changes the eigenvectors. Here they were originally x for B. And now for A, they're M times x. It does not change the eigenvalues because of this M on both sides allowed me to bring M over to the right-hand side and make that work. OK.
Here are some similar matrices. Let me take some. So these will be all similar. Say 2, 3, 0, 4. OK? That's a matrix A. I can see its eigenvalues are 2 and 4. Well, I know that it will be similar to the diagonal matrix. So there is some matrix M that connects this one with this one, connects this A with that B. Well, that B is really capital lambda. And we know what matrix connects the original A to its eigenvalue matrix. What is the M that does that? It's the eigenvector matrix. So to get this particular-- to get this guy, starting from here, I use M is V for this example to produce that. Then B is lambda.
But there are other possibilities. So let me see. I think probably a matrix is-- there is the matrix, A transpose. Is that similar to A? Is A transpose similar to A? Well, answer-- yes.
The transpose matrix has those same eigenvalues, 2 and 4, and different eigenvectors. And those eigenvectors would connect the original A and this A or that A transpose. So the transpose of a matrix is similar to the matrix.
What about if I change the order? 4, 0, 0, 2. So I've just flipped the 2 and the 4, but of course I haven't changed the eigenvalues. You could find the M that does that. You can find an M so that if I multiply on the right by M and on the left by M inverse, it flips those. So there's another matrix similar. Oh, there could be plenty more.
All I want to do is have the eigenvalues be 4 and 2. Shall I just create some more? Here is a 0, 6. I wanted to get the trace right. 4 plus 2 matches 0 plus 6. Now I have to get the determinant right. That has a determinant of 8. What about a 2 and a minus 4 there? I think I've got the trace correct-- 6. And I've got the determinant correct-- 8. And there the determinant is 8. So that would be a similar matrix. All similar matrices. A family of similar matrices with the eigenvalues 4 and 2.
So I want to do another example of similar matrices. What will be different in this example is there'll be missing eigenvectors. So let me say, 2, 2, 0, 1. So that has eigenvalues 2 and 2 but only one eigenvector.
Here is another matrix like that. Say, so the trace should be 4. The determinant should be 4. So maybe I put a 2 and a minus 2 there. I think that has the correct trace, 4, and the great determinant, also 4. So that will have eigenvalues 2 and 2 and only one eigenvector, so it's similar to this.
Now here's the point. You might say, what about 2, 2, 0, 0. That has the correct eigenvalues, but it's not similar. There's no matrix M that connects that diagonal matrix with these other matrices. That matrix has no missing eigenvectors. These matrices have one missing eigenvector.
What's called the Jordan form. The Jordan form. So that didn't belong. That's not in that family. The Jordan form is-- you could say-- well, that'll be the Jordan form. The most beautiful member of the family is the Jordan form.
So I have a whole lot of matrices that are similar. That is the most beautiful, but it's not in the family. It's related but not in the family. It's not similar to those. And the best one would be this one. So the Jordan form would be that one with the eigenvalues on the diagonal. But because there's a missing eigenvector, there has to be a reason for that. And it's in the 1 there, and I can't have a 0 there. OK. So that's the idea of similar matrices.
And now I do have one more important note, a caution about matrix exponentials. Can I just tell you this caution, this caution?
If I look at e to the A times e to the B. The exponential of A times the exponential of B. My caution is that usually that is not e to the B, e to the A. If I put B and A in the opposite order, I get something different. And it's also not e to the A plus B. Those are all different. Which, if I had 1 by 1, just numbers here, of course, that's the great rule for exponentials. But for matrix exponentials, that rule doesn't work. That is not the same as e to the A plus B. And I can show you why.
e to the A is I plus A plus 1/2 A squared and so on. e to the B is I plus B plus 1/2 B squared and so on. And I do that multiplication. And I get I. And I get an A. And I get a B times I. And now I get 1/2 B squared and an AB and 1/2 A squared. Can I put those down? 1/2 A squared, and there's an A times a B. And there's a 1/2 B squared.
OK. This makes the point. If I multiply the exponentials in this order, I get A times B. What if I multiply them in the other order, in that order? If I multiply e to the B times e to the A, then the B's will be out in front of the A's. And this would become a BA, which can be different.
So already I see that the two are different. Here is e to the A, e to the B. It has A before B. If I do it this way, it'll have B before A. If I do it this way, it'll have a mixture. So e to the A plus B will have a I and an A and a B and a 1/2 A plus B squared. So that'll be 1/2 of A squared plus AB plus BA plus B squared. Different again.
Now I have a sort of symmetric mixture of A and B. In this case, I had A before B. In this case, I had B on the left side of A.
So all three of those are different, even in this term of the series that defines those exponentials. And that means that systems of equations, if the coefficients change over time, are definitely harder. We were able to solve dy dt equals, say, cosine of t times y.
Do you remember how-- that this was solvable for a 1 by 1. We put the exponent-- the solution was y is e to the-- we integrated cosine t and got sine t times y of 0. e to the sine t--
Can I just think of putting that into the differential equation-- its derivative. The derivative of e to the sine t will be e to the sine t. I'm using the chain rule. The derivative of e to the sine t will be e to the sine t again, times the derivative of sine t, which is cos t, so it works. That's fine as a solution.
But if I have matrices here-- if I have matrices, then the whole thing goes wrong. You could say that the chain rule goes wrong. You can't put the integral up there and then take the derivative and expect it to come back down. The chain rule will not work for matrix exponentials, the simple chain rule. And the fact is that we don't have nice formulas for the solutions to linear systems with time-varying coefficients. That has become a harder problem when we went from one equation to a system of an equation.
So this is the caution slide about matrix exponentials. They're beautiful. They work perfectly if you just have one matrix A. But if somehow two matrices are in there or a bunch of different matrices, then you lose the good rules, and you lose the solution.
OK. Thank you.
1.1: Overview of Differential Equations Linear equations include dy/dt = y, dy/dt = –y, dy/dt = 2ty . The equation dy/dt = y *y is nonlinear.
1.2: The Calculus You Need The sum rule, product rule, and chain rule produce new derivatives from the derivatives of xn , sin(x ) and ex . The Fundamental Theorem of Calculus says that the integral inverts the derivative.
1.4b: Response to Exponential Input, exp(s*t) With exponential input, est , from outside and exponential growth, eat , from inside, the solution, y(t), is a combination of two exponentials.
1.4c: Response to Oscillating Input, cos(w*t) An oscillating input cos(ωt ) produces an oscillating output with the same frequency ω (and a phase shift).
1.4d: Solution for Any Input, q(t) To solve a linear first order equation, multiply each input q(s) by its growth factor and integrate those outputs.
1.4e: Step Function and Delta Function A unit step function jumps from 0 to 1. Its slope is a delta function: zero everywhere except infinite at the jump.
1.5: Response to Complex Exponential, exp(i*w*t) = cos(w*t)+i*sin(w*t) For linear equations, the solution for f = cos(ωt ) is the real part of the solution for f = eiωt . That complex solution has magnitude G (the gain).
1.6: Integrating Factor for a Constant Rate, a The integrating factor e-at multiplies the differential equation, y’=ay+q, to give the derivative of e-at y: ready for integration.
1.6b: Integrating Factor for a Varying Rate, a(t) The integral of a varying interest rate provides the exponent in the growing solution (the bank balance).
1.7: The Logistic Equation When –by2 slows down growth and makes the equation nonlinear, the solution approaches a steady state y( ∞) = a/b.
1.7c: The Stability and Instability of Steady States Steady state solutions can be stable or unstable – a simple test decides.
2.1: Second Order Equations For the oscillation equation with no damping and no forcing, all solutions share the same natural frequency.
2.1b: Forced Harmonic Motion With forcing f = cos(ωt ), the particular solution is Y *cos(ωt ). But if the forcing frequency equals the natural frequency there is resonance.
2.3: Unforced Damped Motion With constant coefficients in a differential equation, the basic solutions are exponentials est . The exponent s solves a simple equation such as As2 + Bs + C = 0 .
2.3c: Impulse Response and Step Response The impulse response g is the solution when the force is an impulse (a delta function). This also solves a null equation (no force) with a nonzero initial condition.
2.4: Exponential Response - Possible Resonance Resonance occurs when the natural frequency matches the forcing frequency — equal exponents from inside and outside.
2.4b: Second Order Equations With Damping A damped forced equation has a particular solution y = G cos(ωt – α). The damping ratio provides insight into the null solutions.
2.5: Electrical Networks: Voltages and Currents Current flowing around an RLC loop solves a linear equation with coefficients L (inductance), R (resistance), and 1/C (C = capacitance).
2.6: Methods of Undetermined Coefficients With constant coefficients and special forcing terms (powers of t , cosines/sines, exponentials), a particular solution has this same form.
2.6b: An Example of Method of Undetermined Coefficients This method is also successful for forces and solutions such as (at2 + bt +c) est : substitute into the equation to find a, b, c .
2.6c: Variations of Parameters Combine null solutions y1 and y2 with coefficients c1(t) and c2(t) to find a particular solution for any f(t).
2.7: Laplace Transform: First Order Equation Transform each term in the linear differential equation to create an algebra problem. You can then transform the algebra solution back to the ODE solution, y(t) .
2.7b: Laplace Transform: Second Order Equation The second derivative transforms to s2Y and the algebra problem involves the transfer function 1/ (As2 + Bs +C).
3.1: Pictures of the Solutions The direction field for dy/dt = f(t,y) has an arrow with slope f at each point t, y . Arrows with the same slope lie along an isocline.
3.2: Phase Plane Pictures: Source, Sink Saddle Solutions to second order equations can approach infinity or zero. Saddle points contain a positive and also a negative exponent or eigenvalue.
3.2b: Phase Plane Pictures: Spirals and Centers Imaginary exponents with pure oscillation provide a “center” in the phase plane. The point (y, dy/dt) travels forever around an ellipse.
3.2c: Two First Order Equations: Stability A second order equation gives two first order equations for y and dy/dt . The matrix becomes a companion matrix.
3.3: Linearization at Critical Points A critical point is a constant solution Y to the differential equation y’ = f(y) . Near that Y , the sign of df/dy decides stability or instability.
3.3b: Linearization of y'=f(y,z) and z'=g(y,z) With two equations, a critical point has f(Y,Z) = 0 and g(Y,Z) = 0. Near those constant solutions, the two linearized equations use the 2 by 2 matrix of partial derivatives of f and g .
3.3c: Eigenvalues and Stability: 2 by 2 Matrix, A Two equations y’ = Ay are stable (solutions approach zero) when the trace of A is negative and the determinant is positive.
5.1: The Column Space of a Matrix, A An m by n matrix A has n columns each in R m . Capturing all combinations Av of these columns gives the column space – a subspace of R m .
5.4: Independence, Basis, and Dimension Vectors v 1 to v d are a basis for a subspace if their combinations span the whole subspace and are independent: no basis vector is a combination of the others. Dimension d = number of basis vectors.
5.5: The Big Picture of Linear Algebra A matrix produces four subspaces – column space, row space (same dimension), the space of vectors perpendicular to all rows (the nullspace), and the space of vectors perpendicular to all columns.
5.6: Graphs A graph has n nodes connected by m edges (other edges can be missing). This is a useful model for the Internet, the brain, pipeline systems, and much more.
6.1: Eigenvalues and Eigenvectors The eigenvectors x remain in the same direction when multiplied by the matrix (A x = λx ). An n x n matrix has n eigenvalues.
6.2: Diagonalizing a Matrix A matrix can be diagonalized if it has n independent eigenvectors. The diagonal matrix Λis the eigenvalue matrix.
6.3: Solving Linear Systems d y /dt = A y contains solutions y = eλt x where λ and x are an eigenvalue / eigenvector pair for A .
6.4: The Matrix Exponential, exp(A*t) The shortest form of the solution uses the matrix exponential y = eAt y (0) . The matrix eAt has eigenvalues eλt and the eigenvectors of A.
6.4b: Similar Matrices, A and B=M^(-1)*A*M A and B are “similar” if B = M-1AM for some matrix M . B then has the same eigenvalues as A .
6.5: Symmetric Matrices, Real Eigenvalues, Orthogonal Eigenvectors Symmetric matrices have n perpendicular eigenvectors and n real eigenvalues.
7.2: Positive Definite Matrices, S=A'*A A positive definite matrix S has positive eigenvalues, positive pivots, positive determinants, and positive energy vT Sv for every vector v. S = AT A is always positive definite if A has independent columns.
7.2b: Singular Value Decomposition, SVD The SVD factors each matrix A into an orthogonal matrix U times a diagonal matrix Σ (the singular value) times another orthogonal matrix VT : rotation times stretch times rotation.
7.3: Boundary Conditions Replace Initial Conditions A second order equation can change its initial conditions on y(0) and dy/dt(0) to boundary conditions on y(0) and y(1) .
8.1: Fourier Series A Fourier series separates a periodic function F(x) into a combination (infinite) of all basis functions cos(nx) and sin(nx) .
8.1b: Examples of Fourier Series Even functions use only cosines (F(–x) = F(x) ) and odd functions use only sines. The coefficients an and bn come from integrals of F(x) cos(nx ) and F(x) sin(nx ).
8.1c: Fourier Series Solution of Laplace's Equation Inside a circle, the solution u (r , θ) combines rn cos(n θ) and rn sin(n θ). The boundary solution combines all entries in a Fourier series to match the boundary conditions.
8.3: Heat Equation The heat equation ∂u /∂t = ∂2u /∂x2 starts from a temperature distribution u at t = 0 and follows it for t > 0 as it quickly becomes smooth.