From the series: Differential Equations and Linear Algebra
Gilbert Strang, Massachusetts Institute of Technology (MIT)
The integrating factor e-at multiplies the differential equation, y’=ay+q, to give the derivative of e-aty: ready for integration.
OK. This is our last look at the first order linear differential equation that you see up here. The dy dt is ay, that's the interest rate growing in the bank example. y is our total balance. And q of t is our deposits or withdrawals.
Only one change. We allow the interest rate a to change with time. This we didn't see before. Now we will get a formula. It will be a formula we had before when a was constant. And now we'll see it. It looks a little messier, but the point is, it can be done. We can solve that equation by a new way.
So that's really the other point. Everybody in the end likes these integrating factors. And I will call it m. And let me show you what it is and how it works. What it is is the solution to the null equation, with a minus sign. With a minus sign. dM dt equals minus a of tM. No source term. We can solve that equation.
If a is constant-- and I'll keep that case going because that's the one with simple, recognizable formulas. If a is a constant, we're looking for the function M whose derivative is minus aM. And that function is e to the minus at.
The derivative brings down the minus a that we want. In case a is varying, we can still to solve this equation. It will still be an exponential of minus something. But what we have to put here when I take the derivative of M, the derivative of that will come down. So I want the integral of a here. And then the derivative of the integral is a minus a, coming down as it should.
So I want minus the integral of a. And can I introduce dummy variables, say a of T dT, just to make the notation look right. OK. You see that, again, the derivative of M is always with an exponential. It's always the exponential times the derivative of the exponent. And the derivative of that exponent is minus a. Because by the fundamental theorem of calculus, if I integrate a and take its derivative, I get a again. And it's that a that I want.
Now, why do I want this M? How does it work? Here's the reason M succeeds. Look at the derivative of M times y. That's a product. So I'll use the product rule. I get the derivative of y times M, and then I get the derivative of M times y. But the derivative of M is minus a of tM, so I better put the derivative of M is minus a of tM times y.
But what have I got here? Factor out an M and that's just dy dt minus ay, dy dt minus ay is q. So when I factor out the M, I just have q. All together, this is M times q. Look, my differential equation couldn't look nicer. Multiplying by M made it just tell us that a derivative is a right-hand side. To solve that equation, we just integrate both sides.
So if you'll allow me to take that step, integrate both sides and see what I've got, that will give us the formula we know when we're in the constant case, and the formula we've never seen when t is varying. And then I'll do an example. Let me do an example right away.
Suppose a of t, instead of being constant, is growing. The economy is really in hyperinflation. Take that example if a of t is, let's say, 2t. Interest rate started low and moves up, then growth is going to be faster and faster as time goes on. And what will be the integral of 2t?
The integral of 2t is t squared, so M, in that case, will be e to the minus a t squared. Sorry, there's no a anymore. a is just the 2t. e to the minus t squared. With a minus sign, it's dropping fast. In a minute, we'll have a plus sign there and we'll see the growth. Do you see that this is the integrating factor when a of t happens to be 2t?
OK. Now I come back to this equation and integrate both sides to get the answer. OK. All right. The integral of My, of the derivative, the integral of the derivative is just M of t y of t minus M of 0 y of 0. That's the integral on the left side. And on the right side, I have the integral of M times q from 0 to t.
And again, I'm going to put in an integration variable different from t just to keep things straight. OK. So now I've got a formula for y. It involves the M. Actually, the y is multiplied by M, I better divide by-- first of all, do we remember what M of 0 is?
That's the growth factor at 0. It's just 1. Nothing's happened. It's the exponential of 0 in our formulas for M. M of 0 is 1. That's where M starts. So M of 0 is 1. I can remove that.
OK. And now-- oh, let me put that on the other side so this will be equals y of 0 plus that. OK. And now if I divide by M, I have my answer. So those are the steps. Find the integrating factor. Do the integration, which is now made easy because I have a perfect derivative whose integral I just have to integrate. And then put in what M is, and divide by it so that I get y.
OK. So I'm dividing by M. So what is 1 over M? Well, M has this minus sign in the exponent. 1 over M will have a plus sign. M here has e to the minus t squared. 1 over M will be e to the plus t squared.
So when I divide by M, I get y of t. This will be 1 over M. That will be e to the plus the integral of a of t dt y of 0. That's the null solution. That's the solution that's growing out of y of 0. And now I have plus the integral from 0 to t of-- remember, I'm dividing by M. And that's e to the plus the integral from 0 to s of a times q of s ds.
OK. Oh, just a moment. I'm dividing by M and I had an M there. Oh, wait a minute. I haven't got it right here. So I want to know what is M at time s divided by M at time t? So this was the integral from 0 to s. This is an integral from 0 to t. And both are in the exponent.
This is-- can I say it here? This is a e to the-- divided by M is the integral from 0 to t. And then I'm multiplying by e to the minus the integral from 0 to s. The rule for exponents is if I have a product of two exponentials, I add the exponents. When I add this to this, this knocks off the lower half of the integral. I'm left with the integral from s to t of a.
So this was an integral of a minus an integral of a. Let me do our example. Our example up here. Example-- M of t will be-- when a is equal to 2t, this was the example a equal to 2t. The first time we've been able to deal with a varying interest rate. So the integral of 2t is t squared. From the-- is e to the t squared. And I subtract the lower limit, s squared. That's the growth factor.
That's the growth factor from time s to time t. When a was constant, that exponent was just a times t minus s. That told us the time. But now, a is varying and the growth factor between s and t is e to the t squared minus s squared. So that's what goes in here. Let me-- that's the growth factor.
May I just put it in here? In this example, it's e to the t squared minus s squared. Instead of e to the a t minus s, I now have t squared minus s squared, because I had an integral of a of t, and a is not constant anymore. This is my example. And I don't know if you like this formula. Can I just describe it again?
This was an integral from 0 to t, so that would be-- this part would be e to the t squared. That's the growth factor that multiplies the initial deposit. The growth factor that multiplies the later deposit is e to the t squared minus s squared. And we allow deposits all the way from s equals 0 to t. So when we add those up, we get that sum.
We've solved an equation that we hadn't been able to solve before. That's a small triumph in differential equations. Small, admittedly. I'd rather move next to non-linear equations, which we have not touched. And that's a big deal. Thank you.
1.1: Overview of Differential Equations Linear equations include dy/dt = y, dy/dt = –y, dy/dt = 2ty . The equation dy/dt = y *y is nonlinear.
1.2: The Calculus You Need The sum rule, product rule, and chain rule produce new derivatives from the derivatives of xn , sin(x ) and ex . The Fundamental Theorem of Calculus says that the integral inverts the derivative.
1.4b: Response to Exponential Input, exp(s*t) With exponential input, est , from outside and exponential growth, eat , from inside, the solution, y(t), is a combination of two exponentials.
1.4c: Response to Oscillating Input, cos(w*t) An oscillating input cos(ωt ) produces an oscillating output with the same frequency ω (and a phase shift).
1.4d: Solution for Any Input, q(t) To solve a linear first order equation, multiply each input q(s) by its growth factor and integrate those outputs.
1.4e: Step Function and Delta Function A unit step function jumps from 0 to 1. Its slope is a delta function: zero everywhere except infinite at the jump.
1.5: Response to Complex Exponential, exp(i*w*t) = cos(w*t)+i*sin(w*t) For linear equations, the solution for f = cos(ωt ) is the real part of the solution for f = eiωt . That complex solution has magnitude G (the gain).
1.6: Integrating Factor for a Constant Rate, a The integrating factor e-at multiplies the differential equation, y’=ay+q, to give the derivative of e-at y: ready for integration.
1.6b: Integrating Factor for a Varying Rate, a(t) The integral of a varying interest rate provides the exponent in the growing solution (the bank balance).
1.7: The Logistic Equation When –by2 slows down growth and makes the equation nonlinear, the solution approaches a steady state y( ∞) = a/b.
1.7c: The Stability and Instability of Steady States Steady state solutions can be stable or unstable – a simple test decides.
2.1: Second Order Equations For the oscillation equation with no damping and no forcing, all solutions share the same natural frequency.
2.1b: Forced Harmonic Motion With forcing f = cos(ωt ), the particular solution is Y *cos(ωt ). But if the forcing frequency equals the natural frequency there is resonance.
2.3: Unforced Damped Motion With constant coefficients in a differential equation, the basic solutions are exponentials est . The exponent s solves a simple equation such as As2 + Bs + C = 0 .
2.3c: Impulse Response and Step Response The impulse response g is the solution when the force is an impulse (a delta function). This also solves a null equation (no force) with a nonzero initial condition.
2.4: Exponential Response - Possible Resonance Resonance occurs when the natural frequency matches the forcing frequency — equal exponents from inside and outside.
2.4b: Second Order Equations With Damping A damped forced equation has a particular solution y = G cos(ωt – α). The damping ratio provides insight into the null solutions.
2.5: Electrical Networks: Voltages and Currents Current flowing around an RLC loop solves a linear equation with coefficients L (inductance), R (resistance), and 1/C (C = capacitance).
2.6: Methods of Undetermined Coefficients With constant coefficients and special forcing terms (powers of t , cosines/sines, exponentials), a particular solution has this same form.
2.6b: An Example of Method of Undetermined Coefficients This method is also successful for forces and solutions such as (at2 + bt +c) est : substitute into the equation to find a, b, c .
2.6c: Variations of Parameters Combine null solutions y1 and y2 with coefficients c1(t) and c2(t) to find a particular solution for any f(t).
2.7: Laplace Transform: First Order Equation Transform each term in the linear differential equation to create an algebra problem. You can then transform the algebra solution back to the ODE solution, y(t) .
2.7b: Laplace Transform: Second Order Equation The second derivative transforms to s2Y and the algebra problem involves the transfer function 1/ (As2 + Bs +C).
3.1: Pictures of the Solutions The direction field for dy/dt = f(t,y) has an arrow with slope f at each point t, y . Arrows with the same slope lie along an isocline.
3.2: Phase Plane Pictures: Source, Sink Saddle Solutions to second order equations can approach infinity or zero. Saddle points contain a positive and also a negative exponent or eigenvalue.
3.2b: Phase Plane Pictures: Spirals and Centers Imaginary exponents with pure oscillation provide a “center” in the phase plane. The point (y, dy/dt) travels forever around an ellipse.
3.2c: Two First Order Equations: Stability A second order equation gives two first order equations for y and dy/dt . The matrix becomes a companion matrix.
3.3: Linearization at Critical Points A critical point is a constant solution Y to the differential equation y’ = f(y) . Near that Y , the sign of df/dy decides stability or instability.
3.3b: Linearization of y'=f(y,z) and z'=g(y,z) With two equations, a critical point has f(Y,Z) = 0 and g(Y,Z) = 0. Near those constant solutions, the two linearized equations use the 2 by 2 matrix of partial derivatives of f and g .
3.3c: Eigenvalues and Stability: 2 by 2 Matrix, A Two equations y’ = Ay are stable (solutions approach zero) when the trace of A is negative and the determinant is positive.
5.1: The Column Space of a Matrix, A An m by n matrix A has n columns each in R m . Capturing all combinations Av of these columns gives the column space – a subspace of R m .
5.4: Independence, Basis, and Dimension Vectors v 1 to v d are a basis for a subspace if their combinations span the whole subspace and are independent: no basis vector is a combination of the others. Dimension d = number of basis vectors.
5.5: The Big Picture of Linear Algebra A matrix produces four subspaces – column space, row space (same dimension), the space of vectors perpendicular to all rows (the nullspace), and the space of vectors perpendicular to all columns.
5.6: Graphs A graph has n nodes connected by m edges (other edges can be missing). This is a useful model for the Internet, the brain, pipeline systems, and much more.
6.1: Eigenvalues and Eigenvectors The eigenvectors x remain in the same direction when multiplied by the matrix (A x = λx ). An n x n matrix has n eigenvalues.
6.2: Diagonalizing a Matrix A matrix can be diagonalized if it has n independent eigenvectors. The diagonal matrix Λis the eigenvalue matrix.
6.3: Solving Linear Systems d y /dt = A y contains solutions y = eλt x where λ and x are an eigenvalue / eigenvector pair for A .
6.4: The Matrix Exponential, exp(A*t) The shortest form of the solution uses the matrix exponential y = eAt y (0) . The matrix eAt has eigenvalues eλt and the eigenvectors of A.
6.4b: Similar Matrices, A and B=M^(-1)*A*M A and B are “similar” if B = M-1AM for some matrix M . B then has the same eigenvalues as A .
6.5: Symmetric Matrices, Real Eigenvalues, Orthogonal Eigenvectors Symmetric matrices have n perpendicular eigenvectors and n real eigenvalues.
7.2: Positive Definite Matrices, S=A'*A A positive definite matrix S has positive eigenvalues, positive pivots, positive determinants, and positive energy vT Sv for every vector v. S = AT A is always positive definite if A has independent columns.
7.2b: Singular Value Decomposition, SVD The SVD factors each matrix A into an orthogonal matrix U times a diagonal matrix Σ (the singular value) times another orthogonal matrix VT : rotation times stretch times rotation.
7.3: Boundary Conditions Replace Initial Conditions A second order equation can change its initial conditions on y(0) and dy/dt(0) to boundary conditions on y(0) and y(1) .
8.1: Fourier Series A Fourier series separates a periodic function F(x) into a combination (infinite) of all basis functions cos(nx) and sin(nx) .
8.1b: Examples of Fourier Series Even functions use only cosines (F(–x) = F(x) ) and odd functions use only sines. The coefficients an and bn come from integrals of F(x) cos(nx ) and F(x) sin(nx ).
8.1c: Fourier Series Solution of Laplace's Equation Inside a circle, the solution u (r , θ) combines rn cos(n θ) and rn sin(n θ). The boundary solution combines all entries in a Fourier series to match the boundary conditions.
8.3: Heat Equation The heat equation ∂u /∂t = ∂2u /∂x2 starts from a temperature distribution u at t = 0 and follows it for t > 0 as it quickly becomes smooth.