From the series: Differential Equations and Linear Algebra
Gilbert Strang, Massachusetts Institute of Technology (MIT)
Solutions to second order equations can approach infinity or zero. Saddle points contain a positive and also a negative exponent or eigenvalue.
OK. This is the second video for Chapter 3. And it's going to be pictures again. But it's pictures for a second order equation. And I'll make them-- these will be nice. We'll know formulas here. These will be constant coefficient, linear second order equations. And we know that the solution-- there are two special solutions, e to the s1 t and e to the s2t, two null solutions, and any combination is a null solution. So we're talking about null equations, 0 on the right-hand side. And we just want to draw that picture that goes with solutions like that.
So here is the magic word, phase plane, phase plane. We're going to draw the pictures in a plane. Because that's what a blackboard is. And the axes we'll choose will be y and y prime, not t. You'll see how t, time, comes into the picture. But we have the two axes will be y and y prime.
So I had to figure out what y prime was. It just brings down an s1 from that term, and brings down an s2 from that term. And now here's the example. Here is the first example.
So I took this particular equation. Notice that the damping term is negative. I have negative damping. This will be unstable. Solutions will go out to infinity. And I can find those solutions. Because you know that I look for e to the st. I always look for e to the st. I plug in e to the st. I get an s squared from two derivatives, minus 3s from one derivative, plus 2 equaling 0. I factor that, and I find the 2s1 is 1, and s2 is 2. And now I'm ready for the phase plane picture.
OK. Phase plane picture, so here are my solutions. s is 1 or 2. Then the derivative has a 1 or a 2. And here's my plane. Here's my plane. And I want to draw on that the solutions. These solutions, I actually have formulas. I just want to draw them. So I'm plotting.
One example would be that c1 be 1, and let c2 be 0. So that's gone. c1 is 1. I just have that picture. What kind of a picture do I have in the phase plane, in the y, y prime plane, when that's y and that's y prime? Well, those are equal.
So y equals y prime for that solution. y equals y prime along the 45-degree line. It's just like y equal x. y prime is y. And what's happening on this 45-degree line? The solution is this solution, is going straight out the line. As t increases, y and y prime both increase. I go out. This is t going to infinity.
And what about t going to minus infinity? Because we got the whole picture here. When t goes to minus infinity that goes to 0, that goes to 0. Here is the point where the universe began. The Big Bang is right there at t equal minus infinity. And as t increases, this point, y, y prime, is traveling along that 45-degree line. Because y equals y prime, and out there.
And what about the rest of the line? Well, if c1 was negative, if c1 was negative I'd have a minus there, and a minus there. I would just have minuses. And I'd be going out that line. Well, that's one line in my whole plane, but not all.
Now let me take as a second line c1 equals 0. So nothing from e to the t, and let me take y as e to the 2t, and y prime then would be 2e to the 2t. OK. What's happening in the phase plane for this solution, now looking at this one?
Well in this solution, in this case, y prime is 2 times y. y prime is 2 times y. So I'm staying on the line y prime, where y prime is 2 times y. It's a steeper line, steeper line. So that was the case, this was the line where c2 was 0. There was no e to the 2t on that first line that we drew. In the second line that we drew, c1 is 0. There's no e to the t. Everything is in e to the 2t.
So now c1 is 0 on this line. OK. And we just go out it. As t increases, y prime increases faster. Because of the factor 2. So it goes up steeply. And it goes this way. When c2 is negative, if I took a minus and a minus, I would just go down the other way on the same line. And this is still the Big Bang, t equal minus infinity, where everything starts. OK.
So that is two lines, the two special lines in the phase plane. But now I have to draw all the other curves. And they will be curves. And where will they come from? They will come from a combination. So now I'm ready for that one.
Let me take the case c1 equal 1, c2 equal 1. Yeah, why not? c1 equal 1. So I can erase c1. c2 equal 1, I can erase c2. And now I have another solution, y and y prime. And I want to put it in the phase plane. So at every value of t, at every value of t that's a point. That's a value of y. This is a value of y prime. I plot the points y and y prime, and I look at the picture.
And again as t changes, as t changes I'll travel along the solution curve in the phase plane. I'll travel along. As t changes, y will grow, y prime will grow. I'll head out here. But I won't be on that straight line or that straight line. Because those were the cases when I had only one of the two solutions. These were the special solutions. And now I have a combination.
So what happens as t goes to infinity? As t goes to infinity, this wins. As t goes to infinity, the e to the 2t is bigger than e to the t. So this is the larger term. So it approaches. This curve now will approach closer and closer to the one when the line with slope 2. The 2 will be the winner out here.
But at t equal minus infinity, near the Big Bang, at t equal minus infinity, e to the 2t is even more small. So at t equal minus infinity, or t equal minus 10, let's say, this is e to the minus 10. This would be e to the minus 20; very, very, very small. These would win.
So what happens for this solution is it starts out along the line given by the not-so-small t, the not-so-small exponent. It starts up that line. But t is increasing. When t passes some point, this 2t will be bigger than t. And it will, I guess, at t equals 0, 2t will be bigger than t. And from that point on, from the t equal 0 point-- oh, I could even plot the t equals 0.
So at t equals 0, y is 2 and y prime is 3. So at 1, 2, 1, 2, 3; somewhere in there. So you see, the curve starts up along the line where e to the t is bigger. They have the same size at t equals 0, both equal 1. This is at t equals 0. And then for large times, this one wins. So I approach that line.
I don't know if you can see that curve. And I don't swear to the slopes of that curve. But in between in there is filled with curves that start out with this slope, and end with that slope. And the same here, it'll start with this slope. But then go-- probably this is a better picture. Yeah. That's a better picture. Yeah. It will just go up with slope. At the end it will have slope 2 going upwards. Yeah. That looks good.
Well, you could say I only drew part of the phase plane. And you're completely right. If I start somewhere here, what would you think would happen? What would you think would happen if I start with that value of y that much, and that value of y prime? It would have some mixture of-- there would be a c1 and a c2. So the other curves that I haven't drawn yet come from the other c1 and c2.
I've done c1 equal 1, and c2 equal 1. And c1 equal c2 equal 1. But now I have many more possibilities. And what they do is they will-- so suppose I start there. It will approach-- this is the winner. This is the winner. Where c1 is 1, where this is happening, there is the winner for large time.
So all curves swing up toward parallel to that line. Or down here, they swing down parallel to that line. So things here will swing down this way. That's the phase plane.
May I do one more example to show that this was a source? This is called a source. Because the solution goes to infinity. Wherever you start, the solution goes to infinity. It's unstable, totally unstable.
Now if I change to a positive damping, then I would have a plus sign there. These would be plus signs. I would have s equal minus 1, or s equal minus 2. So with positive damping, I damp out naturally. And this picture would be the same, except all the lines are coming in to 0, 0. The solutions are damping to 0, 0; to nothing happening. So I just track the same lines, but in the opposite direction. So instead of this being the Big Bang, it's the end of the universe, t equal infinity.
OK. I'm up for one more picture of this possibility. And let me take the equation y double prime equal 4y. So my equation will be s squared equal 4, s equals 2 or minus 2. And when I draw the phase plane and the solutions, the solutions will be c1 e to the 2t, and c2 e to the minus 2t, from a 2 and a minus 2. That's the solution we all know.
And now I should compute its slope. y prime will be 2c1 e to the 2t minus 2c2 e to the 2t. And now you just want me to draw those pictures. You just want me to draw those pictures, and let me try to say what happens here.
This is a saddle point. It's called a saddle, when we have in one direction things are growing, but in the other, things are decreasing. So most solutions, if c1 is not 0, then the growth is going to win, and that will disappear. But there is the possibility that c1 is 0. So there will be one line coming from there. There will be one line coming from there. Maybe I can try to draw that.
Again, I'll draw that pure line, where c1 is 0. So that pure line is coming. These are minuses here. So that line is coming in to the center. So that's why we have a saddle. We approach a saddle if along this where this is minus 2 of that, so I think it would be-- so it's a slope of minus 2. So I think a slope like that, so again this is y. This is y prime. This is the slope of minus 2. And that's this curve.
So it will be very exceptional that we're right on that line. All other points, all other curves in this phase plane, are going to have a little c1 in them. And then this will take over. And that gives us, as we saw before, this slope. This is 2 times that. So that line is where everybody wants to go. And only if you start exactly on this line do you get this picture, and you come into the saddle.
Instead of the Big Bang, or the end of the universe, this is now the saddle point, where we come in on this most special of all lines, coming from this picture. But almost always this is the dominant thing. And we go out.
So if I take a typical starting point, I'll go out this like that, or like this, oh no. Yeah, no. I'll go out. It'll have to go out. So if I start anywhere here, these are probably they're hyperbolas going out in that direction. I don't swear that they're hyperbolas.
Here again we might start in. Because we have big numbers here. But then e to the t takes over. And we go out. So those go out. These go out. And these go out. So this is the big line. That's the line coming from here. And that's where everything wants to go, and everything eventually goes that way, except the one line where c1 is 0.
So this dominant term is not even here then. And then we should become inwards. So that saddle point is the special point where you could go out, if you go the right way. Or you could come in, if you go the other special, special way.
OK. So that sources, sinks and saddles. And I still have to draw the pictures, which involves spirals that come from complex s, where we have oscillation. That'll be the next video.
1.1: Overview of Differential Equations Linear equations include dy/dt = y, dy/dt = –y, dy/dt = 2ty . The equation dy/dt = y *y is nonlinear.
1.2: The Calculus You Need The sum rule, product rule, and chain rule produce new derivatives from the derivatives of xn , sin(x ) and ex . The Fundamental Theorem of Calculus says that the integral inverts the derivative.
1.4b: Response to Exponential Input, exp(s*t) With exponential input, est , from outside and exponential growth, eat , from inside, the solution, y(t), is a combination of two exponentials.
1.4c: Response to Oscillating Input, cos(w*t) An oscillating input cos(ωt ) produces an oscillating output with the same frequency ω (and a phase shift).
1.4d: Solution for Any Input, q(t) To solve a linear first order equation, multiply each input q(s) by its growth factor and integrate those outputs.
1.4e: Step Function and Delta Function A unit step function jumps from 0 to 1. Its slope is a delta function: zero everywhere except infinite at the jump.
1.5: Response to Complex Exponential, exp(i*w*t) = cos(w*t)+i*sin(w*t) For linear equations, the solution for f = cos(ωt ) is the real part of the solution for f = eiωt . That complex solution has magnitude G (the gain).
1.6: Integrating Factor for a Constant Rate, a The integrating factor e-at multiplies the differential equation, y’=ay+q, to give the derivative of e-at y: ready for integration.
1.6b: Integrating Factor for a Varying Rate, a(t) The integral of a varying interest rate provides the exponent in the growing solution (the bank balance).
1.7: The Logistic Equation When –by2 slows down growth and makes the equation nonlinear, the solution approaches a steady state y( ∞) = a/b.
1.7c: The Stability and Instability of Steady States Steady state solutions can be stable or unstable – a simple test decides.
2.1: Second Order Equations For the oscillation equation with no damping and no forcing, all solutions share the same natural frequency.
2.1b: Forced Harmonic Motion With forcing f = cos(ωt ), the particular solution is Y *cos(ωt ). But if the forcing frequency equals the natural frequency there is resonance.
2.3: Unforced Damped Motion With constant coefficients in a differential equation, the basic solutions are exponentials est . The exponent s solves a simple equation such as As2 + Bs + C = 0 .
2.3c: Impulse Response and Step Response The impulse response g is the solution when the force is an impulse (a delta function). This also solves a null equation (no force) with a nonzero initial condition.
2.4: Exponential Response - Possible Resonance Resonance occurs when the natural frequency matches the forcing frequency — equal exponents from inside and outside.
2.4b: Second Order Equations With Damping A damped forced equation has a particular solution y = G cos(ωt – α). The damping ratio provides insight into the null solutions.
2.5: Electrical Networks: Voltages and Currents Current flowing around an RLC loop solves a linear equation with coefficients L (inductance), R (resistance), and 1/C (C = capacitance).
2.6: Methods of Undetermined Coefficients With constant coefficients and special forcing terms (powers of t , cosines/sines, exponentials), a particular solution has this same form.
2.6b: An Example of Method of Undetermined Coefficients This method is also successful for forces and solutions such as (at2 + bt +c) est : substitute into the equation to find a, b, c .
2.6c: Variations of Parameters Combine null solutions y1 and y2 with coefficients c1(t) and c2(t) to find a particular solution for any f(t).
2.7: Laplace Transform: First Order Equation Transform each term in the linear differential equation to create an algebra problem. You can then transform the algebra solution back to the ODE solution, y(t) .
2.7b: Laplace Transform: Second Order Equation The second derivative transforms to s2Y and the algebra problem involves the transfer function 1/ (As2 + Bs +C).
3.1: Pictures of the Solutions The direction field for dy/dt = f(t,y) has an arrow with slope f at each point t, y . Arrows with the same slope lie along an isocline.
3.2: Phase Plane Pictures: Source, Sink Saddle Solutions to second order equations can approach infinity or zero. Saddle points contain a positive and also a negative exponent or eigenvalue.
3.2b: Phase Plane Pictures: Spirals and Centers Imaginary exponents with pure oscillation provide a “center” in the phase plane. The point (y, dy/dt) travels forever around an ellipse.
3.2c: Two First Order Equations: Stability A second order equation gives two first order equations for y and dy/dt . The matrix becomes a companion matrix.
3.3: Linearization at Critical Points A critical point is a constant solution Y to the differential equation y’ = f(y) . Near that Y , the sign of df/dy decides stability or instability.
3.3b: Linearization of y'=f(y,z) and z'=g(y,z) With two equations, a critical point has f(Y,Z) = 0 and g(Y,Z) = 0. Near those constant solutions, the two linearized equations use the 2 by 2 matrix of partial derivatives of f and g .
3.3c: Eigenvalues and Stability: 2 by 2 Matrix, A Two equations y’ = Ay are stable (solutions approach zero) when the trace of A is negative and the determinant is positive.
5.1: The Column Space of a Matrix, A An m by n matrix A has n columns each in R m . Capturing all combinations Av of these columns gives the column space – a subspace of R m .
5.4: Independence, Basis, and Dimension Vectors v 1 to v d are a basis for a subspace if their combinations span the whole subspace and are independent: no basis vector is a combination of the others. Dimension d = number of basis vectors.
5.5: The Big Picture of Linear Algebra A matrix produces four subspaces – column space, row space (same dimension), the space of vectors perpendicular to all rows (the nullspace), and the space of vectors perpendicular to all columns.
5.6: Graphs A graph has n nodes connected by m edges (other edges can be missing). This is a useful model for the Internet, the brain, pipeline systems, and much more.
6.1: Eigenvalues and Eigenvectors The eigenvectors x remain in the same direction when multiplied by the matrix (A x = λx ). An n x n matrix has n eigenvalues.
6.2: Diagonalizing a Matrix A matrix can be diagonalized if it has n independent eigenvectors. The diagonal matrix Λis the eigenvalue matrix.
6.3: Solving Linear Systems d y /dt = A y contains solutions y = eλt x where λ and x are an eigenvalue / eigenvector pair for A .
6.4: The Matrix Exponential, exp(A*t) The shortest form of the solution uses the matrix exponential y = eAt y (0) . The matrix eAt has eigenvalues eλt and the eigenvectors of A.
6.4b: Similar Matrices, A and B=M^(-1)*A*M A and B are “similar” if B = M-1AM for some matrix M . B then has the same eigenvalues as A .
6.5: Symmetric Matrices, Real Eigenvalues, Orthogonal Eigenvectors Symmetric matrices have n perpendicular eigenvectors and n real eigenvalues.
7.2: Positive Definite Matrices, S=A'*A A positive definite matrix S has positive eigenvalues, positive pivots, positive determinants, and positive energy vT Sv for every vector v. S = AT A is always positive definite if A has independent columns.
7.2b: Singular Value Decomposition, SVD The SVD factors each matrix A into an orthogonal matrix U times a diagonal matrix Σ (the singular value) times another orthogonal matrix VT : rotation times stretch times rotation.
7.3: Boundary Conditions Replace Initial Conditions A second order equation can change its initial conditions on y(0) and dy/dt(0) to boundary conditions on y(0) and y(1) .
8.1: Fourier Series A Fourier series separates a periodic function F(x) into a combination (infinite) of all basis functions cos(nx) and sin(nx) .
8.1b: Examples of Fourier Series Even functions use only cosines (F(–x) = F(x) ) and odd functions use only sines. The coefficients an and bn come from integrals of F(x) cos(nx ) and F(x) sin(nx ).
8.1c: Fourier Series Solution of Laplace's Equation Inside a circle, the solution u (r , θ) combines rn cos(n θ) and rn sin(n θ). The boundary solution combines all entries in a Fourier series to match the boundary conditions.
8.3: Heat Equation The heat equation ∂u /∂t = ∂2u /∂x2 starts from a temperature distribution u at t = 0 and follows it for t > 0 as it quickly becomes smooth.