From the series: Differential Equations and Linear Algebra
Gilbert Strang, Massachusetts Institute of Technology (MIT)
The direction field for dy/dt = f(t,y) has an arrow with slope f at each point t, y. Arrows with the same slope lie along an isocline.
OK. So we've moved on into Chapter 3. Chapter 1 and 2 were about equations we could solve, first order equations, chapter one; second order equations in chapter 2, often linear, constant coefficient sometimes. Now we take any equation. And I'll start with first order.
First derivative is some function and not a linear function, so I don't expect a formula. A solution will exist. But I won't have a formula for the solution. But I can make a picture of the solution. You see what's happening as time goes on. And so that's today's lecture, is a picture.
So this function, whatever it is, gives the slope of y. That's the slope. And it will be the slope of the arrows that I will draw in this picture. So here's a picture that started, y, t. And the slope of the arrows is f. And here is my example. Well, you will see. I chose a constant coefficient linear equation. Because I could find a solution. So 2 minus y, I know from that minus sign that I'm going to have exponential decay in the null solution. And then y equal to 2 is a very special particular solution, a constant.
And in my picture, y equal to 2, it jumps out. Because when y is 2, when y is 2 the slope is 0. So all my arrows on the y equal to 2 line, have slope 0. So that's a very special line. And since the solution follows the arrows that's the whole point. The solution follows the arrows. Because the arrows tell the slope.
So if I'm on that line, the solution just follows those arrows, and stays on the line. y equal to 2 is a fixed point, fixed point of the solution, a fixed point for the equation. And the question is, if I don't start at y equal to 2, do I move toward 2 or away from it? OK.
So I can see from the formula what the answer is going to be. If I start with some other value, some other value of c not 0, then there will be a null solution part. But as t gets large that goes to 0. So I move toward 2. Now let's see that in the picture.
So let me-- I'm drawing the arrows first. So this is all time starting at if y is 0, then if y is 0 then dy dt is 2. So I draw arrows with slope 2, along the y equals 0 line. This is the y equals 0 line. All my arrows have slope 2.
Now what else? So that's a few arrows that show what will-- so the solution if it starts there, will start in the direction of that arrow. But then I have to see what the other arrows are for other values of y. Because right away the solution y will change. And the slope will change. And that's it needs more arrows. Well, actually it needs way more arrows than I can possibly draw.
Let me draw another line of arrows when y is 1, along that line, along the line y equal 1. When y is 1, 2 minus 1 is 1. The slope is 1. f is 1. And my arrows have slope 1. So all along here, the arrows go up. Those went up steeply with slope 2. Now the arrows will go up. So I'll have arrows that are going a 45-degree angle, slope 1.
Do you see? I hope you begin to see the picture here. The solution might start there. It would start with that slope. But it will curve down. Because the arrows are not so steep. As I go upward, the arrows are getting flat. And so the curve that follows the arrows has to flatten out, flatten out, flatten out. The arrows are still, at that point the arrows are still slope 1. But it's flattening out. And it's never going to cross this line. And it will run closer and closer to that line. And wherever it starts, if it starts at time t equals to 1 there, it'll do the same thing. And it will stay just below the other one.
Do you see what the pictures are looking like? If it starts at different times, so these are different times. These are different starts. Yeah, really we're used to, at t equals 0, we're used to giving y of 0. So this is starting at 0. This is starting at 2. Starting at 1 would be a higher start. What about starting at 4?
Suppose y of 0 is 4. That point is t equals 0, y equal 4. So that point is y of 0 equal 4. What's the graph of the solution with that start? Actually, I could figure out what the solution would be if y of 0 was 4, I'd have 2 plus 2 e to the minus t. At t equals 0, that's 4. And it fits. It solves the equation. And it's going to be its graph. I should be able learn that from the arrows.
So along this line of y equal 4, all the arrows when y is 4, the slope is minus 2. So these arrows from these points, go down with slope minus 2. But the solution starts down. So it starts like that. But then it has to follow the new arrows. And the new arrows are not so steep. So the new arrows are I have slope minus 1. I hope my picture is showing the steeper slope 2 along this line, and the flatter slope 1, or rather minus 1 downwards, along this line. So it just follows along here.
Well of course it's just a mirror image of that one. It's a mirror image of that one. I'm trying to show the graph of all solutions from all starts, the whole plane. Actually I could go, t could go to minus infinity. And y could go all the way from minus infinity up, all the way up. I could fill the whole board here with arrows, and then with solutions. And the solutions would follow the arrows, the arrows changing slope and actually in this case, all solutions wherever you started, would approach 2.
And that's what the formula says. But we get that information from the arrows with no formula. Let me show you a next example. And here's our next example. The logistic equation, it's not linear. So it's going to be more interesting.
And do you remember the solution? You remember maybe the trick with the logistic equation was 1 over the solution, gave a linear equation and expression like that. OK. Time to draw arrows.
OK. When y is 0-- so here's y-- when y is 0, the slope is 0. So I have a whole line of flat. I have a flat horizontal line. That's the solution, y equals 0 fixed point. Also we have another fixed point. When y is 1, 1 minus 1 is 0. Slope is 0. Slope stays 0. The arrows all have zero slope along the line y equal 1. So there is another solution, which doesn't do anything exciting. It just stays at 1. y equal 1 is another fixed point, steady state, whatever words we want to use.
But again, the real picture is what about other starts. What about a start at 1/2? Well if it starts at-- if y is 1/2 half at the starting time, what is the slope? 1/2 minus 1/4 is 1/4. So the slope is upwards, but not very steep. The slope along the-- and it doesn't depend on t in these examples. So that slope is the same as long as y is 1/2, doesn't matter what the time is. y equals 1/2 gives me a 1/2 minus 1/4, which is a 1/4. It gives me that slope.
What about the slope 1/4? So 1/4, I have 1/4 minus 1/16. I think that's 3/16. So it's beginning to climb upward. So it's upwards again. But 3/16 is a little-- I don't know if I'm going to get the picture too brilliantly. The slope, as soon it-- if it just starts a little above 0, what happens to the solution that starts a little bit above 0?
It climbs. Because if y is above 0, say if it starts between 0 and 1, if y is between 0 and 1, then y is bigger than y squared. And the slope is positive. And it goes up. So do you see what it's doing? The slope will just, if it starts a little bit above, it'll have a small slope. But that slope will gradually increase. But then actually at this point, the slope is 3/4 minus whatever it is. It slows down, still going upwards. y is still bigger than y squared.
You recognize what the curve is going to look like. So there is an S curve. It's an S curve. Which we saw for the logistic equation, and here we have a formula for it. Well, the whole point of today's video was we don't need a formula. So you don't need that. The arrows will tell you that it starts up slowly. It gets only-- that's the biggest slope it gets. And then it starts down. The slope goes down again. But it's still a positive slope. Still climbing, climbing, climbing, and approaching 1.
Now that's sort of a sandwich in the picture. But it could start with a negative. So what happens if it starts at y equal minus 1? The slope, if y is minus 1, we have minus 1, minus 1, a slope of minus 2. That's a steeper serious downward slope. So the solution that starts here has-- that's tangent. You see that it's tangent to the arrow, because it has the same slope as the arrow. And it comes down. But as it goes down, the slopes are getting steeper. Whoops, not flatter, but steeper.
For example, if y is minus 2, I have minus 2, minus 4 is minus 6. So as soon as it gets down to minus 2, the slope has jumped way down to minus 6. So here is the-- it falls right off. It's a drop-off curve, a drop-off curve. It falls right off actually to infinity. It never makes it out to-- it falls down to y equal minus infinity in a fixed time, in a definite time.
And so here's a whole region of curves going down to minus infinity. Here is a whole region. What happens in this region? Suppose y starts at plus 2? Well, I have 2 minus 4. So the slope is negative. The slope is negative up here. Yeah. And this is the big picture. The slope, the arrows are positive below this line. They're upward. They were downward here. They're slowly upward in this sandwich. And then up above, they're downward again.
So if slopes are coming down, and they drop into actually it's a symmetric picture. Really-- no reason not to go backwards in time. Where are these coming from? They're all coming from curves. The whole plane is full of curves. And these start at plus infinity. They drop into 2. These start below 0, and they drop off to minus infinity. And then the real interest in studying population was these.
Can you do one more example? Let me take a third example that has a t in the function. So the arrows won't be the same along the whole line. In fact, the arrows will be the same. So if I have 1 plus t minus y, that's the f, equal a constant. Then that's a curve-- well, it's actually a straight line. It's actually a 45-degree line in this plane. And along that line the f, this is the f, the f of t and y, the arrow slope. The arrows slopes are the same along that line. That line is called an isocline. This is called an I-S-O, meaning the same, cline, meaning slope. So that's an isocline.
Here's an isocline. It's a 45-degree line. That's the 45-degree line, 1 plus t minus y equal 1. Let me draw the 45-degree line 1 plus t minus y equals 0. So it's a little bit higher. OK. Now arrows, and then put in the curves, the solution curves that match the arrows.
So the arrows have this slope along that line. Along this line, 1 plus t minus y, they have slope 0. Oh, interesting. At every point on the line the slope is 0. Because this is the slope of the arrows. At every point on this line, the slope is 1, also very interesting. Because that's right along the line.
So here we have a solution line. That must be a solution line. That's the line where y is t. That's a very big 45-degree important line. Because if y equals t, if y equals t then dy dt should be 1. And it is 1 for y equals t. So that's a solution line with that solution.
Now what about a line with 1 plus t minus y equal minus 1, a line? If 1 plus t minus y is minus 1, if f is minus 1, the slope is negative. So what does that mean? If 1 plus t minus y is minus 1, the slope is negative. So at points on this line, the slope is going downwards. Oh, interesting. I wasn't quite expecting that. Let me just see if I got a suitable picture. Why is it not right?
If 1 plus t minus y is-- oh, I'm sorry. This is the line, y equal 1 plus t. I think what I'm expecting to see is I'm expecting to see it from the formula too that as time goes on, this part goes to 0, and y goes to t. I believe that all the solutions will approach this y equal to t. I think their slopes, their slopes here-- darn, that's not right. Their slopes should be coming upwards. Yeah, let me-- I can figure that out.
If t is let's say 1, and y is 0. OK. If t is 1, and y is 0, I have a slope of 2. Good. OK. There's a point t equal to 1. Here is 0, 0. Here's the point t equal to 1, y equals 0. The slope came out to be 2. It went up that way. So along that line, the slopes are going up. Along this line, the slopes are right on the line. On this line the slopes are flat, and the curve is moving toward the line.
I'll just draw the beautiful picture now of the solution. So the solutions look like this. They are-- this is the big line. You've got to keep your eye on that line. Because that's the steady state line that all solutions are approaching. So if you have the idea of arrows to show the slope, fitting solution curves through tangent to the arrows, and sometimes having a formula to confirm that you did it right, you get a picture like this. So that's the idea of first order equations, which are graphed in the y-t plane. And the arrows tell you the derivative. Thanks.
1.1: Overview of Differential Equations Linear equations include dy/dt = y, dy/dt = –y, dy/dt = 2ty . The equation dy/dt = y *y is nonlinear.
1.2: The Calculus You Need The sum rule, product rule, and chain rule produce new derivatives from the derivatives of xn , sin(x ) and ex . The Fundamental Theorem of Calculus says that the integral inverts the derivative.
1.4b: Response to Exponential Input, exp(s*t) With exponential input, est , from outside and exponential growth, eat , from inside, the solution, y(t), is a combination of two exponentials.
1.4c: Response to Oscillating Input, cos(w*t) An oscillating input cos(ωt ) produces an oscillating output with the same frequency ω (and a phase shift).
1.4d: Solution for Any Input, q(t) To solve a linear first order equation, multiply each input q(s) by its growth factor and integrate those outputs.
1.4e: Step Function and Delta Function A unit step function jumps from 0 to 1. Its slope is a delta function: zero everywhere except infinite at the jump.
1.5: Response to Complex Exponential, exp(i*w*t) = cos(w*t)+i*sin(w*t) For linear equations, the solution for f = cos(ωt ) is the real part of the solution for f = eiωt . That complex solution has magnitude G (the gain).
1.6: Integrating Factor for a Constant Rate, a The integrating factor e-at multiplies the differential equation, y’=ay+q, to give the derivative of e-at y: ready for integration.
1.6b: Integrating Factor for a Varying Rate, a(t) The integral of a varying interest rate provides the exponent in the growing solution (the bank balance).
1.7: The Logistic Equation When –by2 slows down growth and makes the equation nonlinear, the solution approaches a steady state y( ∞) = a/b.
1.7c: The Stability and Instability of Steady States Steady state solutions can be stable or unstable – a simple test decides.
2.1: Second Order Equations For the oscillation equation with no damping and no forcing, all solutions share the same natural frequency.
2.1b: Forced Harmonic Motion With forcing f = cos(ωt ), the particular solution is Y *cos(ωt ). But if the forcing frequency equals the natural frequency there is resonance.
2.3: Unforced Damped Motion With constant coefficients in a differential equation, the basic solutions are exponentials est . The exponent s solves a simple equation such as As2 + Bs + C = 0 .
2.3c: Impulse Response and Step Response The impulse response g is the solution when the force is an impulse (a delta function). This also solves a null equation (no force) with a nonzero initial condition.
2.4: Exponential Response - Possible Resonance Resonance occurs when the natural frequency matches the forcing frequency — equal exponents from inside and outside.
2.4b: Second Order Equations With Damping A damped forced equation has a particular solution y = G cos(ωt – α). The damping ratio provides insight into the null solutions.
2.5: Electrical Networks: Voltages and Currents Current flowing around an RLC loop solves a linear equation with coefficients L (inductance), R (resistance), and 1/C (C = capacitance).
2.6: Methods of Undetermined Coefficients With constant coefficients and special forcing terms (powers of t , cosines/sines, exponentials), a particular solution has this same form.
2.6b: An Example of Method of Undetermined Coefficients This method is also successful for forces and solutions such as (at2 + bt +c) est : substitute into the equation to find a, b, c .
2.6c: Variations of Parameters Combine null solutions y1 and y2 with coefficients c1(t) and c2(t) to find a particular solution for any f(t).
2.7: Laplace Transform: First Order Equation Transform each term in the linear differential equation to create an algebra problem. You can then transform the algebra solution back to the ODE solution, y(t) .
2.7b: Laplace Transform: Second Order Equation The second derivative transforms to s2Y and the algebra problem involves the transfer function 1/ (As2 + Bs +C).
3.1: Pictures of the Solutions The direction field for dy/dt = f(t,y) has an arrow with slope f at each point t, y . Arrows with the same slope lie along an isocline.
3.2: Phase Plane Pictures: Source, Sink Saddle Solutions to second order equations can approach infinity or zero. Saddle points contain a positive and also a negative exponent or eigenvalue.
3.2b: Phase Plane Pictures: Spirals and Centers Imaginary exponents with pure oscillation provide a “center” in the phase plane. The point (y, dy/dt) travels forever around an ellipse.
3.2c: Two First Order Equations: Stability A second order equation gives two first order equations for y and dy/dt . The matrix becomes a companion matrix.
3.3: Linearization at Critical Points A critical point is a constant solution Y to the differential equation y’ = f(y) . Near that Y , the sign of df/dy decides stability or instability.
3.3b: Linearization of y'=f(y,z) and z'=g(y,z) With two equations, a critical point has f(Y,Z) = 0 and g(Y,Z) = 0. Near those constant solutions, the two linearized equations use the 2 by 2 matrix of partial derivatives of f and g .
3.3c: Eigenvalues and Stability: 2 by 2 Matrix, A Two equations y’ = Ay are stable (solutions approach zero) when the trace of A is negative and the determinant is positive.
5.1: The Column Space of a Matrix, A An m by n matrix A has n columns each in R m . Capturing all combinations Av of these columns gives the column space – a subspace of R m .
5.4: Independence, Basis, and Dimension Vectors v 1 to v d are a basis for a subspace if their combinations span the whole subspace and are independent: no basis vector is a combination of the others. Dimension d = number of basis vectors.
5.5: The Big Picture of Linear Algebra A matrix produces four subspaces – column space, row space (same dimension), the space of vectors perpendicular to all rows (the nullspace), and the space of vectors perpendicular to all columns.
5.6: Graphs A graph has n nodes connected by m edges (other edges can be missing). This is a useful model for the Internet, the brain, pipeline systems, and much more.
6.1: Eigenvalues and Eigenvectors The eigenvectors x remain in the same direction when multiplied by the matrix (A x = λx ). An n x n matrix has n eigenvalues.
6.2: Diagonalizing a Matrix A matrix can be diagonalized if it has n independent eigenvectors. The diagonal matrix Λis the eigenvalue matrix.
6.3: Solving Linear Systems d y /dt = A y contains solutions y = eλt x where λ and x are an eigenvalue / eigenvector pair for A .
6.4: The Matrix Exponential, exp(A*t) The shortest form of the solution uses the matrix exponential y = eAt y (0) . The matrix eAt has eigenvalues eλt and the eigenvectors of A.
6.4b: Similar Matrices, A and B=M^(-1)*A*M A and B are “similar” if B = M-1AM for some matrix M . B then has the same eigenvalues as A .
6.5: Symmetric Matrices, Real Eigenvalues, Orthogonal Eigenvectors Symmetric matrices have n perpendicular eigenvectors and n real eigenvalues.
7.2: Positive Definite Matrices, S=A'*A A positive definite matrix S has positive eigenvalues, positive pivots, positive determinants, and positive energy vT Sv for every vector v. S = AT A is always positive definite if A has independent columns.
7.2b: Singular Value Decomposition, SVD The SVD factors each matrix A into an orthogonal matrix U times a diagonal matrix Σ (the singular value) times another orthogonal matrix VT : rotation times stretch times rotation.
7.3: Boundary Conditions Replace Initial Conditions A second order equation can change its initial conditions on y(0) and dy/dt(0) to boundary conditions on y(0) and y(1) .
8.1: Fourier Series A Fourier series separates a periodic function F(x) into a combination (infinite) of all basis functions cos(nx) and sin(nx) .
8.1b: Examples of Fourier Series Even functions use only cosines (F(–x) = F(x) ) and odd functions use only sines. The coefficients an and bn come from integrals of F(x) cos(nx ) and F(x) sin(nx ).
8.1c: Fourier Series Solution of Laplace's Equation Inside a circle, the solution u (r , θ) combines rn cos(n θ) and rn sin(n θ). The boundary solution combines all entries in a Fourier series to match the boundary conditions.
8.3: Heat Equation The heat equation ∂u /∂t = ∂2u /∂x2 starts from a temperature distribution u at t = 0 and follows it for t > 0 as it quickly becomes smooth.