Chapter 11 Optimization and Lagrange Multipliers
“The wise man doesn't give the right answers, he poses the right questions.” - Claude Levi-Strauss
Probably the most applied concept in all of calculus is finding the maxima and minima of functions. In industry, these can be problems from engineering, such as trying to design an airplane wing that yields the maximum lift and stability while at the same time minimizing the drag coefficient. This way, we build a plane that flies easily while using less fuel. Since planes measure fuel consumption in gallons per second, a small change in wing design can result in considerable profit for the company (and a big raise for you). Have you noticed the addition of the upward turned tips at the ends of the airplane wings in recent years?
In the financial markets, mutual funds are sets of stocks. People may buy shares of the fund instead of buying shares of individual stocks. Mutual fund managers (and their clients) want to choose groups of stocks that will increase in value and make them (and their clients) rich. Thus, a fund manager wants to design a mutual fund that will maximize profits for the investors, but because investors fear volatility (large fluctuations in the value of their portfolios), the manager also wants to minimize the volatility of the mutual fund. This is an example of an optimization problem with what is called a constraint because you want to maximize profit but are constrained by the customers' concerns about volatility.
Mathematicians have spent a considerable amount of time in industry working on both of these interesting problems that are representative of “real-world” applications. In mathematics, the difference between being able to understand or apply a formula to such a problem and the ability to derive or create your own formulas for the problems is the difference between working as an engineer, mutual fund manager, or biologist on a team (a wonderful job in it's own right) and working in a think tank such as Bell Labs or Los Alamos or MSRI (the Mathematical Sciences Research Institute) where you are tackling the problems that no one can solve and creating the mathematics that will be implemented by the teams in industry.
In Calculus I you solved optimization or max-min problems by setting the derivative of a function to zero to tell you where the rate of change (or slope) of the function was zero. In Calculus III we do exactly the same thing. Except that instead of solving \(f'=0\) we are solving \(\nabla f=0\) and we are seeking a horizontal tangent plane instead of a horizontal tangent line. The procedures are very similar and both find the point in the domain of the functions where potential maxima and minima are attained.
Definition 11.1.
A critical point of \(f:{\re}^2 \to \re\) is any point \(x \in \re^2\) where \(\nabla f(x)\) is zero or undefined.
Since \(\nabla f = 0\) translates to \(\nabla f(x,y) = (f_x (x,y), f_y (x,y)) = (0,0)\) we are seeking points \((x,y)\) in the plane where both \(f_x(x,y) =0\) and \(f_y(x,y) = 0\text{,}\) or where at least one is undefined. Review the definitions for “local minimum, local maximum, and inflection point” for functions from \(\re\) to \(\re\) (i.e. from Calculus I). You may look these up in a book or on the web.
Recall from Definition 10.33 that an \(\epsilon\) - neighborhood of the point \((s,t)\) in \(\re^2\) is the set of all points in \(\re^2\) that are a distance of less than \(\epsilon\) away from \((s,t).\)
Definition 11.2.
If \(f:\re^2 \to \re\) and \(x\) is in the domain of \(f\text{,}\) then we say \((x,f(x))\) is a local minimum if \(f(x)\lt f(y)\) for every \(x\) in some \(\epsilon\)-neighborhood of \(x\text{.}\)
Local maximum is defined similarly. A function may have infinitely many local minima and maxima.
Problem 11.3.
Find the minimum of \(f(x,y)=x^2-2x+y^2-4y+5\) in two different ways.
First, set \(\nabla f(x,y)=0\) and solve for \((x,y)\text{.}\)
Second, complete the square to write \(f\) as \(f(x,y) = (x-a)^2 + (y-b)^2\) and graph.
Problem 11.4.
Let \(f(x,y)=8y^3+12x^2-24xy\text{.}\)
Find all critical points of \(f.\)
Sketch \(f\) using any software to verify your answers.
Use any software to solve the whole problem. In other words, use software to compute your partial derivatives and solve for the roots of these partial derivatives.
Recall in Calculus I that if \(x\) were a critical point for \(f\) then \((x,f(x))\) could have been a minimum, maximum, or an inflection point for \(f.\) Inflection points were critical points where the function \(f\) switched concavity (i.e. where the first derivative is zero and the second derivative switches signs). In Calculus III the analogous points are critical points that are neither maxima nor minima and we call these saddle points.
Definition 11.5.
If \(f:{\re}^2 \to \re\) is differentiable at \(x\text{,}\) then we say \((x,f(x))\) is a saddle point if \(\nabla f(x)=0\) and no matter how small an \(\epsilon >0\) we choose, there are points \(y\) and \(z\) in the \(\epsilon\)-neighborhood of \(x\) so that \(f(x)\lt f(y)\) and \(f(x)>f(z)\text{.}\)
This definition of a saddle point says that if we are at a saddle point and we decide to walk away from it, then there are paths away from the critical point along which \(f\) increases and paths away from the point along which \(f\) decreases. Given a critical point, how do we determine if it was a maximum, minimum, or saddle point? That is, how do we classify the critical points of \(f\text{?}\) How did we do it in Calculus I? We restate the Second Derivative Test which is a slick way to classify the the critical points of the single-variable, real-valued functions from Calculus I. Notice how nicely it parallels the next theorem for classifying the multi-variable, real-valued functions of Calculus III.
Theorem 11.6.
Second Derivative Test I. If \(f: \re \to \re\) is differentiable and \(f'(x)=0\) then,
If \(f''(x) > 0\text{,}\) then \((x,f(x))\) is a minimum.
If \(f''(x) \lt 0\text{,}\) then \((x,f(x))\) is a maximum.
If \(f''\) switches signs at \(x\text{,}\) then \((x,f(x))\) is an inflection point.
Here is a sweet theorem for classifying critical points of functions of two variables.
Theorem 11.7.
Second Derivative Test II. If \(f:{\re}^2 \to \re\) and \(\nabla f(u)=0\) and \(D=f_{xx}(u)f_{yy}(u)-(f_{xy}(u))^2\text{,}\) then
\((u,f(u))\) is a local min if \(D>0\) and \(f_{xx}(u)>0\text{.}\)
\((u,f(u))\) is a local max if \(D>0\) and \(f_{xx}(u)\lt 0\text{.}\)
\((u,f(u))\) is a saddle point if \(D\lt 0\text{.}\)
No information if D=0.
Example 11.8.
Classify all critical points of a function \(f\) with gradient \(\nabla f(x,y) = (xy-2x-3y+6, yx-y+4x-4).\)
Problem 11.9.
Compute the critical points of \(f(x,y)=xy^2-6x^2-3y^2\) and classify these critical points as local maxima, local minima, or saddle points.
Problem 11.10.
Compute and classify the critical points of \(f(x,y)=xy+\dsp{\frac{2}{x}+\frac{4}{y}}\text{.}\)
Problem 11.11.
Compute and classify the critical points of \(\dsp f(x,y)=e^{-(x^2+y^2-4y)}\text{.}\)
The next problem reminds you of an important aspect of max/min problems from Calculus I. If you wanted the local maxima and minima of a function like \(f(x) = x^2\) on \([-1,3]\text{,}\) then you checked not only the places where \(f'=0\) but also the values of \(f\) at the endpoints (boundary) of the interval. The same must be done for functions of several variables. When you applied this technique, you were applying the Extreme Value Theorem.
Theorem 11.12.
Extreme Value Theorem for Functions of One Variable. If \(f: [a,b] \to \re\) is a differentiable function, then \(f\) has a maximum and a minimum on this interval.
The corresponding theorem for real-valued functions of two variables requires at least an intuitive idea of the notions of what it means for a set to be closed and bounded. A set \(S\) is bounded if there is a number \(M\) so that \(| x | \le M\) for all \(x \in S.\) A set \(S\) is closed if it contains its boundary points. Think of the open and closed intervals. An open interval does not contain its end points, the points on the boundary of the set. A closed interval does contain its boundary points. The set of all points \((x,y)\) satisfying \(x^2 + y^2 \leq 9\) is closed, while the set of all points \((x,y)\) satisfying \(x^2 + y^2 \lt 9\) is not closed.
Definition 11.13.
A set \(S\) in \(\re^2\) is closed if it contains all its boundary points.
Theorem 11.14.
Extreme Value Theorem for Functions of Two Variables. If \(M\) is a closed and bounded set and \(f: M \to \re\) is a differentiable function, then \(f\) has a maximum and a minimum on \(M\text{.}\)
Example 11.15.
Optimize \(f(x,y) = 2x^2 + y^2 + 1\) subject to \(g=0\) where \(g(x,y) = x^2 + y^2 - 1\) in four ways!
Problem 11.16.
Let \(T(x,y)=2x^2+y^2-y\) be the temperature at the point \((x,y)\) on the circular disk of radius 1 centered at \((0,0).\)
Find the critical points of T.
Find the minimum and maximum of T over the circle (perimeter), \(x^2+y^2=1\) by parameterizing the circle.
Find the minimum and maximum of T over the disk, \(x^2+y^2 \leq 1.\)
For the next problem, you will need to parameterize each of the four line segments that form the line and check the maximum and minimum of the function over not only the interior of the square, but also over each of the four lines.
Problem 11.17.
Find the maximum and minimum of \(f(x,y) = 2x^2 - 3y^2 + 10\) over the rectangular region, \(S = \{ (x,y) | 0 \leq x \leq 3, 2 \le y \le 4 \}.\)
Problem 11.18.
Find all maxima and minima of \(f(x,y)=x^2-y^2+4y\) over the rectangular region \(R = \{ (x,y) | -1 \leq x \leq 1, -3 \le y \le 3 \}.\)
Problem 11.19.
Find all maxima and minima of \(f(x,y)=2x^2 + 3y^2 +2\) over all points on or inside the triangle with vertices, \((-2,0)\text{,}\) \((0,2)\text{,}\) and \((2,0)\text{.}\)
In Calculus I an \(n^{th}\) degree polynomial will have at most \(n-1\) critical points. What about in Calculus III? Here is a \(6^{th}\) degree polynomial which has far more than 5 critical points. All of these may be found by hand and easily seen when graphed.
Problem 11.20.
Find all thirteen critical points of \(f(x,y) = x^3y^3 - x^3y -3xy^3 + 3xy +1.\)
Problem 11.21.
Let \(f(x,y) = x^3 - y^3\) and \(p = (2,4).\) Find the direction in which \(f\) increases the most rapidly.
Problem 11.22.
Ted is riding his mountain bike and is at altitude (in feet) of \(A(x,y) = 5000e^{-(3x^2+y^2)/100}.\) What is my slope of descent or ascent if I am riding in the direction \(\oa {(-1,1)}\) starting at the point, \((10,10, 5000e^{-4})?\) In what direction should I travel to ascend the most rapidly? To descend the most rapidly?
Problem 11.23.
Let \(\dsp T(x,y,z) = \frac{10}{x^2 + y^2 + z^2}\) and \(\dsp \oa r(t) = (t \cos(\pi t) , t \sin(\pi t) , t ).\) If \(T(x,y,z)\) represents the temperature in space at the point \((x,y,z)\) and \(\oa r(t)\) represents Ted's position at time t, then compute \((T \circ \oa r)'(3)\) and explain what this number represents in a complete sentence. To check your answer, compute it in two ways. First compose the functions and take the derivative. Second, use the chain rule.
We have tackled optimization problems before when we found maxima and minima of functions like \(f(x,y)=x^2+x^2y+y^2+4\) from Problem 11.3. We have also sought maxima and minima of curves (or paths) on surfaces when we found the maxima and minima of \(T(x,y)=2x^2+y^2-y\) over the circle \(x^2+y^2+1\) as in Problem 11.16. Problem 11.16 is called a constrained optimization problem because we want a maxima or a minima of \(T\) subject to the constraint that it is above the unit circle. The method of Lagrange multipliers is a slick way to tackle constrained optimization problems.
Theorem 11.24.
Lagrange Multipliers. Suppose that \(f, g: \re^2 \to \re.\) To maximize (or minimize) the function \(f\) subject to the constraint \(g=0\) we solve the two equations,
\(\nabla f(x)=\lambda \nabla g(x)\)
\(g(x)=0\)
for \(x\) and for \(\lambda.\) The variable \(\lambda\) is called the Lagrange multiplier, and \(x\) is the point at which \(f\) is maximized (or minimized).
Example 11.25.
Minimize the function \(f(x,y) = y^2-x^2\) over the region \(g(x,y)\leq 0\) where \(g(x,y) = \frac{x^2}{4}+y^2 -1.\)
Problem 11.26.
Find any possible maxima or minima of \(f(x,y)=x^2+y^2\) subject to \(xy=3\) in three ways. First use Lagrange multipliers by putting \(g(x,y) = xy-3\) so that the constraint is \(g(x,y) = 0.\) Second, substitute \(y=3/x\) into the equation and solve. Third, sketch \(f\) and identify the portion of \(f\) that is above the equation, \(xy=3.\)
Problem 11.27.
Find any minima and maxima of \(f(x,y)=4x^2+y^2-4xy\) subject to \(x^2+y^2=1\text{.}\) It may be helpful to eliminate \(\lambda\) first.
Problem 11.28.
Find the minimum of \(f(x,y,z)=3x+2y+z\) subject to \(9x^2+4y^2-z=0\) via Lagrange multipliers.