From the series: Differential Equations and Linear Algebra
Gilbert Strang, Massachusetts Institute of Technology (MIT)
An oscillation equation d2y/dt2 + Sy = 0 has 2n solutions (sines and cosines). Solutions use the eigenvectors of S.
OK. Now, I'm going to have differential equations, systems of equations, so there'll be matrices and vectors, using symmetric matrix. They'll be second order. So second order, second derivative, that y is the vector. And S is the symmetric matrix. And that's the first time we've been prepared for the most fundamental equation of physics, of mechanics, oscillating springs-- so many applications-- rotating torques. It's very important in applications. The finite element, giant finite element codes, are solving equations like that all the time. And we don't have a damping term here, so it-- or a forcing term, so it's the null solutions that I'm going to look for to match initial conditions. I don't have a forcing term.
OK. So the real central equation always looks like that. This is-- Newton's law, is what this is-- mass times acceleration. So M will be a matrix, often a diagonal matrix, telling me the masses. Remember, I have n equations here, so I have n masses, as you'll see. And I have, let's say, a bunch of springs connecting those masses. And then there's a matrix K in multiplying y itself, and that's always called the stiffness matrix.
So, actually, in applications, the first job is to take the problem and create these matrices. I'll give you an example, but let's suppose we've got them, and how do we solve them? We look for, as we always do, solutions where time is separate from the vector x. I substitute that into the equation, So I get M, second derivative will bring down the i omega, twice. E to the i omega t x, right? Plus, this term, K times e to the i omega t x, should be 0. So I'm just substituting the expected form for the solution. In that form, that exponential factor can cancel. And, I see, I have an eigenvalue problem. Let me just look at that eigenvalue problem. I'm going to put that on the opposite side. But what i squared is giving me minus one. I'm just going to be left with Kx. Well, let me put this on the other side, because it's got a minus. And then when I put it over there, will be a plus.
M omega squared x. That's an eigenvalue problem. Here is the eigenvector. There is the eigenvalue. Oh, but we have two matrices. That's something a little new. Not new to MATLAB however. The MATLAB command to find these eigenvalues, let me call those eigenvalues lambda, so lambda will now be omega squared, because two derivatives brought down omega twice. But we have our two matrices, so the MATLAB command would be i of K and M. If you define the matrices, K and M, and you call that command, it will produce the eigenvalues and the eigenvectors x for this, you could say, generalized eigenvalue problem, two matrix eigenvalue problem. It's got a K, as usual, and then it has an M. But many, many times, M will be a multiple of the identity and present no problem.
OK, so that-- this is the eigenvalue problem that we reached. And that's the command that would solve it. OK. That's the first step, is to look for solutions of that special form.
Now let's do a little count. How many solutions are we expecting? How many initial conditions do we have? So we initially, we give y at 0 of course, the initial condition, the position. But we also give, in a second order equation, we also give the initial velocity, y prime of 0. And those are vectors, because those tell us the initial condition of n masses. And so I have n numbers from y of 0 and n more numbers. Two n, all together, initial conditions. I'm going to need two n solutions. I'm going to need two n solutions to match two n initial conditions and solve the-- solve the equation.
OK, so what do they look like? All right. Here, I've gone ahead to put an application up. So, again, I'm taking the masses to be equal. Here are the masses, M, M, and M. And so I end up with a 3 by 3 matrix. I have three unknowns, n is 3 for this problem. And I've got 4 springs. These are springs-- maybe I make them look a little springier. OK. And they're connected at the top to a fixed support and at the bottom to a fixed support and they're connected to each other.
Do you see what will happen? As I, maybe I, the initial condition, I drag all those masses down? As my initial condition, I let go? Then they will go up and down, up and down, just the way springs always do. And they will solve, their position will be the solution to my differential equation. M y double prime plus Ky equals 0. So I'm not, I'm just starting the motion, and then backing off. So I'm not forcing-- it's not forced motion. It's pure oscillation, pure oscillation. Right.
But coupled, several oscillators are a couple, that's what's new. We know all about this equation when y is just a scalar, just one equation. We know that, and that led us to the square root of K over M. That was just a 1 by 1 eigenvalue problem, and now we will have a 3 by 3 eigenvalue problem. The mass matrix is simple. Here's what the stiffness matrix would look like, if all those springs were the same. Just, I wanted to see what kind of a matrix shows up in the problem. And time to write down solutions.
OK, so what are we remembering from solutions? We're remembering that solutions look like this. I have, but I'll have three possible eigenvectors, because I have 3 by 3 matrices. So that will give me three solutions. But I want six because I have six initial conditions all together. Let me write those six solutions down.
Y, the solution is, sub constant times the cosine of omega t, times the first eigenvector. And another constant times the sine of omega t, times the second eigenvector-- times the first eigenvector, sorry. The first eigenvector, I have three eigenvectors, and for each of them, I get two solutions, one a cosine and one a sine. And the frequencies, there would be omega 1, and you are remembering that lambda, lambda is omega squared. So if I write omega 1 there, it's the square root of lambda 1. That's the square root of lambda 1.
So I've got two solutions so far, coming from the first eigenvector at its eigenvalue, and with the cosine and a sine. And then, I'll also have for the A2 and a B2, and an A3 and a B3. So I just, briefly, A2, B2 with omega, using omega 2 and x2. And then they'll be an A3 and a B3 using omega 3 and the eigenvector x3.
That's pretty stupid looking. This is what I meant to represent. I don't want to rewrite all that with 2's, and then rewrite it again with 3's. That's what the solution looks like.
OK. How do we-- when we match the initial condition, what happens? I said t equals 0, right? When I said t equals 0, the sines disappear. So it's the A's that match y of 0. So A's match y of 0. And when I look at the initial velocity, the derivative, the derivative of the cosine is the sine, and it's 0. But the derivative of the sine is the cosine and it's 1, so I'll see that the B's match y prime of 0.
I'm trying to get the total count to be six. So there are three, three initial positions, and three A's. There are three initial velocities, cause there are three masses, and there are three B's. I get a perfect match, a total of six. Six constants, six numbers to match. It all works.
OK. Now, do I want to try an example? Sure. Let me end with a particular example. I better go to 2 by 2 for an example. So can I do the same problem 2 by 2? So my problem is going to be y double prime plus S-- there's a K, there's a division by m, and there's a 2, 2 minus 1, minus 1 y equals 0. That's my equation.
Now, I'm speaking now, about the problem with a spring, a first mass m, a spring, the second mass m, and a spring. So two masses, two equal masses, three equal springs. That's my equation.
So what's a solution? How do I solve it? I need the eigenvalues and the eigenvectors of my matrix. So here's my matrix S. That's my matrix S. It's symmetric, it's got physical constants there. The stiffness of the springs divided by the masses, k over m, we're expecting that same k over m that always shows up. And we need the eigenvectors there. And what are they? The eigenvalues of that are 1 times k over m, And 3 times k over m. Cause the trace is 4. 2 plus 2 is 4. The determinant is 3. 4 minus 1 is 3. Those are the two eigenvalues. And these are the omega squareds, remember.
OK. That's how I start the system from rest. So physically I pull down these masses, or maybe I push that one up and pull that one down, whatever I like. I hold them for a moment and I let go. I don't give them an initial velocity. They start from rest, so the B's will be 0. So y prime of 0, y prime of 0 is going to be 0 from rest. And that will give me B's are 0.
So my solution will be, my A cosine of, so the eigenvalues cosine, I take the square root of k over m t times the first eigenvector-- one of the eigenvectors. The eigenvectors of this, probably are 1, 1. And then the other eigenvector, and that will have a cosine of the square root of omega, I have to take the square root of that, so that's the square root of 3, k over m t, times its eigenvector, which I think is 1 minus 1. It's going to be perpendicular to that one.
I've solved the problem. The A1 and the A2 are determined by the initial condition.
Now do you see what's happening in the motion? That's the last thing, last point for this video. This motion with a 1, 1 eigenvector, the two masses are in sync. They're growing together, up and down. That's one eigenvector of the problem. And it has a certain frequency that they go up and down-- a square root of k over m, our old friend. But also, with two masses, they can go against each other, like this motion. That's coming from this eigenvector, and it happens at a higher frequency. So those are going-- the final solution is a combination of the masses moving together at a little slower oscillation, and the masses moving opposite each other at a faster oscillation. Some combination of those two is the solution. And then if we had three masses, there would be three oscillations. One where all three are going together, one where the outside ones are opposite, and one where all three are, I see opposite signs.
It's a beautiful subject. Highly developed, and highly important applications. But that's the nicest solution you could hope for.
OK that's a second order system, solved. Good.
1.1: Overview of Differential Equations Linear equations include dy/dt = y, dy/dt = –y, dy/dt = 2ty . The equation dy/dt = y *y is nonlinear.
1.2: The Calculus You Need The sum rule, product rule, and chain rule produce new derivatives from the derivatives of xn , sin(x ) and ex . The Fundamental Theorem of Calculus says that the integral inverts the derivative.
1.4b: Response to Exponential Input, exp(s*t) With exponential input, est , from outside and exponential growth, eat , from inside, the solution, y(t), is a combination of two exponentials.
1.4c: Response to Oscillating Input, cos(w*t) An oscillating input cos(ωt ) produces an oscillating output with the same frequency ω (and a phase shift).
1.4d: Solution for Any Input, q(t) To solve a linear first order equation, multiply each input q(s) by its growth factor and integrate those outputs.
1.4e: Step Function and Delta Function A unit step function jumps from 0 to 1. Its slope is a delta function: zero everywhere except infinite at the jump.
1.5: Response to Complex Exponential, exp(i*w*t) = cos(w*t)+i*sin(w*t) For linear equations, the solution for f = cos(ωt ) is the real part of the solution for f = eiωt . That complex solution has magnitude G (the gain).
1.6: Integrating Factor for a Constant Rate, a The integrating factor e-at multiplies the differential equation, y’=ay+q, to give the derivative of e-at y: ready for integration.
1.6b: Integrating Factor for a Varying Rate, a(t) The integral of a varying interest rate provides the exponent in the growing solution (the bank balance).
1.7: The Logistic Equation When –by2 slows down growth and makes the equation nonlinear, the solution approaches a steady state y( ∞) = a/b.
1.7c: The Stability and Instability of Steady States Steady state solutions can be stable or unstable – a simple test decides.
2.1: Second Order Equations For the oscillation equation with no damping and no forcing, all solutions share the same natural frequency.
2.1b: Forced Harmonic Motion With forcing f = cos(ωt ), the particular solution is Y *cos(ωt ). But if the forcing frequency equals the natural frequency there is resonance.
2.3: Unforced Damped Motion With constant coefficients in a differential equation, the basic solutions are exponentials est . The exponent s solves a simple equation such as As2 + Bs + C = 0 .
2.3c: Impulse Response and Step Response The impulse response g is the solution when the force is an impulse (a delta function). This also solves a null equation (no force) with a nonzero initial condition.
2.4: Exponential Response - Possible Resonance Resonance occurs when the natural frequency matches the forcing frequency — equal exponents from inside and outside.
2.4b: Second Order Equations With Damping A damped forced equation has a particular solution y = G cos(ωt – α). The damping ratio provides insight into the null solutions.
2.5: Electrical Networks: Voltages and Currents Current flowing around an RLC loop solves a linear equation with coefficients L (inductance), R (resistance), and 1/C (C = capacitance).
2.6: Methods of Undetermined Coefficients With constant coefficients and special forcing terms (powers of t , cosines/sines, exponentials), a particular solution has this same form.
2.6b: An Example of Method of Undetermined Coefficients This method is also successful for forces and solutions such as (at2 + bt +c) est : substitute into the equation to find a, b, c .
2.6c: Variations of Parameters Combine null solutions y1 and y2 with coefficients c1(t) and c2(t) to find a particular solution for any f(t).
2.7: Laplace Transform: First Order Equation Transform each term in the linear differential equation to create an algebra problem. You can then transform the algebra solution back to the ODE solution, y(t) .
2.7b: Laplace Transform: Second Order Equation The second derivative transforms to s2Y and the algebra problem involves the transfer function 1/ (As2 + Bs +C).
3.1: Pictures of the Solutions The direction field for dy/dt = f(t,y) has an arrow with slope f at each point t, y . Arrows with the same slope lie along an isocline.
3.2: Phase Plane Pictures: Source, Sink Saddle Solutions to second order equations can approach infinity or zero. Saddle points contain a positive and also a negative exponent or eigenvalue.
3.2b: Phase Plane Pictures: Spirals and Centers Imaginary exponents with pure oscillation provide a “center” in the phase plane. The point (y, dy/dt) travels forever around an ellipse.
3.2c: Two First Order Equations: Stability A second order equation gives two first order equations for y and dy/dt . The matrix becomes a companion matrix.
3.3: Linearization at Critical Points A critical point is a constant solution Y to the differential equation y’ = f(y) . Near that Y , the sign of df/dy decides stability or instability.
3.3b: Linearization of y'=f(y,z) and z'=g(y,z) With two equations, a critical point has f(Y,Z) = 0 and g(Y,Z) = 0. Near those constant solutions, the two linearized equations use the 2 by 2 matrix of partial derivatives of f and g .
3.3c: Eigenvalues and Stability: 2 by 2 Matrix, A Two equations y’ = Ay are stable (solutions approach zero) when the trace of A is negative and the determinant is positive.
5.1: The Column Space of a Matrix, A An m by n matrix A has n columns each in R m . Capturing all combinations Av of these columns gives the column space – a subspace of R m .
5.4: Independence, Basis, and Dimension Vectors v 1 to v d are a basis for a subspace if their combinations span the whole subspace and are independent: no basis vector is a combination of the others. Dimension d = number of basis vectors.
5.5: The Big Picture of Linear Algebra A matrix produces four subspaces – column space, row space (same dimension), the space of vectors perpendicular to all rows (the nullspace), and the space of vectors perpendicular to all columns.
5.6: Graphs A graph has n nodes connected by m edges (other edges can be missing). This is a useful model for the Internet, the brain, pipeline systems, and much more.
6.1: Eigenvalues and Eigenvectors The eigenvectors x remain in the same direction when multiplied by the matrix (A x = λx ). An n x n matrix has n eigenvalues.
6.2: Diagonalizing a Matrix A matrix can be diagonalized if it has n independent eigenvectors. The diagonal matrix Λis the eigenvalue matrix.
6.3: Solving Linear Systems d y /dt = A y contains solutions y = eλt x where λ and x are an eigenvalue / eigenvector pair for A .
6.4: The Matrix Exponential, exp(A*t) The shortest form of the solution uses the matrix exponential y = eAt y (0) . The matrix eAt has eigenvalues eλt and the eigenvectors of A.
6.4b: Similar Matrices, A and B=M^(-1)*A*M A and B are “similar” if B = M-1AM for some matrix M . B then has the same eigenvalues as A .
6.5: Symmetric Matrices, Real Eigenvalues, Orthogonal Eigenvectors Symmetric matrices have n perpendicular eigenvectors and n real eigenvalues.
7.2: Positive Definite Matrices, S=A'*A A positive definite matrix S has positive eigenvalues, positive pivots, positive determinants, and positive energy vT Sv for every vector v. S = AT A is always positive definite if A has independent columns.
7.2b: Singular Value Decomposition, SVD The SVD factors each matrix A into an orthogonal matrix U times a diagonal matrix Σ (the singular value) times another orthogonal matrix VT : rotation times stretch times rotation.
7.3: Boundary Conditions Replace Initial Conditions A second order equation can change its initial conditions on y(0) and dy/dt(0) to boundary conditions on y(0) and y(1) .
8.1: Fourier Series A Fourier series separates a periodic function F(x) into a combination (infinite) of all basis functions cos(nx) and sin(nx) .
8.1b: Examples of Fourier Series Even functions use only cosines (F(–x) = F(x) ) and odd functions use only sines. The coefficients an and bn come from integrals of F(x) cos(nx ) and F(x) sin(nx ).
8.1c: Fourier Series Solution of Laplace's Equation Inside a circle, the solution u (r , θ) combines rn cos(n θ) and rn sin(n θ). The boundary solution combines all entries in a Fourier series to match the boundary conditions.
8.3: Heat Equation The heat equation ∂u /∂t = ∂2u /∂x2 starts from a temperature distribution u at t = 0 and follows it for t > 0 as it quickly becomes smooth.