12_Total_Differential.tex

\section{Differentiability and the Total Differential}\label{sec:total_differential}

We studied \textbf{differentials} in \autoref{sec:differentials}, where \autoref{def:differential}  states that if $y=f(x)$ and $f$ is differentiable, then $\dd y=\fp(x)\dd x$. One important use of this differential is in Integration by Substitution. Another important application is approximation. Let $\Delta x = \dd x$ represent a change in $x$. When $\dd x$ is small, $\dd y\approx \Delta y$, the change in $y$ resulting from the change in $x$. Fundamental in this understanding is this: as $\dd x$ gets small, the difference between $\Delta y$ and $\dd y$ goes to 0. Another way of stating this: as $\dd x$ goes to 0, the \emph{error} in approximating $\Delta y$ with $\dd y$ goes to 0.

We extend this idea to functions of two variables. Let $z=f(x,y)$, and let $\Delta x = \dd x$ and $\Delta y=\dd y$ represent changes in $x$ and $y$, respectively. Let $\Delta z = f(x+\dd x,y+\dd y) - f(x,y)$ be the change in $z$ over the change in $x$ and $y$. Recalling that $f_x$ and $f_y$ give the instantaneous rates of $z$-change in the $x$- and $y$-directions, respectively, we can approximate $\Delta z$ with $\dd z = f_x\dd x+f_y\dd y$; in words, the total change in $z$ is approximately the change caused by changing $x$ plus the change caused by changing $y$. In a moment we give an indication of whether or not this approximation is any good. First we give a name to $\dd z$.

\begin{definition}[Total Differential]\label{def:total_differential}
Let $z=f(x,y)$ be continuous on an open set $S$. Let $\dd x$ and $\dd y$ represent changes in $x$ and $y$, respectively. Where the partial derivatives $f_x$ and $f_y$ exist, the \textbf{total differential of $z$} is \index{total differential}\index{partial derivative!total differential}
\[\dd z = f_x(x,y)\dd x + f_y(x,y)\dd y.\]
\end{definition}

\youtubeVideo{C1Xcj5Xmngc}{Differentials of Functions of Two Variables}

\begin{example}[Finding the total differential]\label{ex_total_diff_10}
Let $z = x^4e^{3y}$. Find $\dd z$.
\solution
We compute the partial derivatives: $f_x = 4x^3e^{3y}$ and $f_y = 3x^4e^{3y}$. Following \autoref{def:total_differential}, we have
\[\dd z = 4x^3e^{3y}\dd x+3x^4e^{3y}\dd y.\]
\end{example}

We \emph{can} approximate $\Delta z$ with $\dd z$, but as with all approximations, there is error involved. A good approximation is one in which the error is small. At a given point $(x_0,y_0)$, let $E_1$ and $E_2$ be functions of $\dd x$ and $\dd y$ such that $E_1\dd x+E_2\dd y$ describes this error. Then
\begin{align*}
\Delta z &= \dd z + E_1\dd x+ E_2\dd y \\
		&= f_x(x_0,y_0)\dd x+f_y(x_0,y_0)\dd y + E_1\dd x+E_2\dd y.
\end{align*}
If the approximation of $\Delta z$ by $\dd z$ is good, then as $\dd x$ and $\dd y$ get small,  so does $E_1\dd x+E_2\dd y$. The approximation of $\Delta z$ by $\dd z$ is even better if, as $\dd x$ and $\dd y$ go to 0, so do $E_1$ and $E_2$. This leads us to our definition of differentiability.

\begin{definition}[Multivariable Differentiability]\label{def:multi_differentiability}
Let $z=f(x,y)$ be defined on an open set $S$ containing $(x_0,y_0)$ where $f_x(x_0,y_0)$ and $f_y(x_0,y_0)$ exist. Let $\dd z$ be the total differential of $z$ at $(x_0,y_0)$, let $\Delta z = f(x_0+\dd x,y_0+\dd y) - f(x_0,y_0)$, and let $E_1$ and $E_2$ be functions of $\dd x$ and $\dd y$ such that 
\index{differentiable}\index{derivative!multivariable differentiability}\index{multivariable function!differentiability}
\[\Delta z = \dd z + E_1\dd x + E_2\dd y.\]
\begin{enumerate}
	\item $f$ is \textbf{differentiable at $(x_0,y_0)$} if%, given $\epsilon >0$, there is a $\delta >0$ such that if $\norm{\bracket{\dd x,\dd y}}< \delta$, then $\norm{\bracket{E_1,E_2}}< \epsilon$. That is, as $\dd x$ and $\dd y$ go to 0, so do $E_1$ and $E_2$.
%	\[\lim_{(\dd x,\dd y)\to(0,0)}\norm{\bracket{E_1,E_2}}=0.\]
	\[\lim_{(\dd x,\dd y)\to(0,0)}E_1=0\qquad\text{and}\qquad\lim_{(\dd x,\dd y)\to(0,0)}E_2=0.\]
	\item $f$ is \textbf{differentiable on $S$} if $f$ is differentiable at every point in $S$. If $f$ is differentiable on $\mathbb{R}^2$, we say that $f$ is \textbf{differentiable everywhere}.
\end{enumerate}
\end{definition}

\begin{example}[Showing a function is differentiable]\label{ex_totaldiff1}
Show $f(x,y) = xy+3y^2$ is differentiable using \autoref{def:multi_differentiability}.
\solution
We begin by finding $f(x+\dd x,y+\dd y)$, $\Delta z$, $f_x$ and $f_y$.
\begin{align*}
f(x+\dd x,y+\dd y) &= (x+\dd x)(y+\dd y) + 3(y+\dd y)^2 \\
						&= xy + x\dd y+y\dd x+\dd x \dd y + 3y^2+6y\dd y+3\dd y^2.
\end{align*}
$\Delta z = f(x+\dd x,y+\dd y) - f(x,y)$, so
\[\Delta z = x\dd y + y\dd x + \dd x \dd y + 6y\dd y+3\dd y^2.\]
It is straightforward to compute $f_x = y$ and $f_y = x+6y$. Consider once more $\Delta z$:
\begin{align*}
\Delta z &= x\dd y + y\dd x + \dd x \dd y + 6y\dd y+3\dd y^2 \qquad \text{ (now reorder)}\\
		&= y\dd x + x\dd y+6y\dd y+ \dd x \dd y + 3\dd y^2\\
		&= \underbrace{(y)}_{f_x}\dd x + \underbrace{(x+6y)}_{f_y}\dd y + \underbrace{(\dd y)}_{E_1}\dd x+\underbrace{(3\dd y)}_{E_2}\dd y\\
		&= f_x\dd x + f_y\dd y + E_1\dd x+E_2\dd y.
\end{align*}
With $E_1 = \dd y$ and $E_2 = 3\dd y$, it is clear that as $\dd x$ and $\dd y$ go to 0, $E_1$ and $E_2$ also go to 0. Since this did not depend on a specific point $(x_0,y_0)$, we can say that $f(x,y)$ is differentiable for all pairs $(x,y)$ in $\mathbb{R}^2$, or, equivalently, that $f$ is differentiable everywhere.
\end{example}

Our intuitive understanding of differentiability of functions $y=f(x)$ of one variable was that the graph of $f$ was ``smooth.'' A similar intuitive understanding of functions $z=f(x,y)$ of two variables is that the surface defined by $f$ is also ``smooth,'' not containing cusps, edges, breaks,  etc. The following theorem
%states that differentiable functions are continuous, followed by another theorem that 
provides a more tangible way of determining whether a great number of functions are differentiable or not.

% differentiability requires total derivative requires partial derivatives requires continuity
%\begin{theorem}[Continuity and Differentiability of Multivariable Functions]\label{thm:diff_cont_multi}
%Let $z=f(x,y)$ be defined on an open set $S$ containing $(x_0,y_0)$. 
%If $f$ is differentiable at $(x_0,y_0)$, then $f$ is continuous at $(x_0,y_0)$.
%\index{multivariable function!differentiability}\index{multivariable function!continuity}
%\end{theorem}

\begin{theorem}[Differentiability of Multivariable Functions]\label{thm:differentiable}
Let $z=f(x,y)$ be defined on an open set $S$. If $f_x$ and $f_y$ are both continuous on $S$, then $f$ is differentiable on $S$.
\index{multivariable function!differentiability}
\end{theorem}

The theorems assure us that essentially all functions that we see in the course of our studies here are differentiable (and hence continuous) on their natural domains. There is a difference between \autoref{def:multi_differentiability} and \autoref{thm:differentiable}, though: it is possible for a function $f$ to be differentiable yet $f_x$ or $f_y$ is \emph{not} continuous. Such strange behavior of functions is a source of delight for many mathematicians.  When this happens, we need to use other methods to determine whether or not $f$ is differentiable at that point.

%For instance, consider the function 
%\[
% f(x,y)=
% \begin{cases}\frac{xy}{x^2+y^2} & (x,y)\neq (0,0) \\ 0 & (x,y) = (0,0)\end{cases}
%\]
%We can find $f_x(0,0)$ and $f_y(0,0)$ using \autoref{def:partial_derivative}:
%\begin{align*}
%f_x(0,0) &= \lim_{h\to 0} \frac{f(0+h,0) - f(0,0)}{h} \\
%				&= \lim_{h\to 0} \frac{0}{h^2} = 0;\\
%f_y(0,0) &= \lim_{h\to 0} \frac{f(0,0+h) - f(0,0)}{h} \\
%				&= \lim_{h\to 0} \frac{0}{h^2} = 0.
%\end{align*}
%
%Both $f_x$ and $f_y$ \emph{exist} at $(0,0)$, but they are not continuous at $(0,0)$, as 
%\[f_x(x,y) = \frac{y(y^2-x^2)}{(x^2+y^2)^2} \qquad \text{and}\qquad f_y(x,y) = \frac{x(x^2-y^2)}{(x^2+y^2)^2} \] are not continuous at $(0,0)$. (Take the limit of $f_x$ as $(x,y)\to(0,0)$ along the $x$- and $y$-axes; they give different results.) So even though $f_x$ and $f_y$ \emph{exist} at every point in the $x$-$y$ plane, they are not continuous. Therefore it is possible, by \autoref{thm:differentiable}, for $f$ to not be differentiable.
%
%Indeed, it is not. One can show that $f$ is not continuous at $(0,0)$ (see \autoref{ex_multilimit4}), and by \autoref{thm:diff_cont_multi}, this means $f$ is not differentiable at $(0,0)$.

\subsection{Approximating with the Total Differential}

By the definition, when $f$ is differentiable $\dd z$ is a good approximation for $\Delta z$ when $\dd x$ and $\dd y$ are small. We give some simple examples of how this is used here.

\begin{example}[Approximating with the total differential]\label{ex_totaldiff2}
Let $f(x,y)=\sqrt{x}\sin y$. Approximate $f(4.1,0.2)$.
\solution
We can approximate $f(4.1,0.2)$ using $f(4,0)=0$. Without calculus, this is the best approximation we could reasonably come up with. The total differential gives us a way of adjusting this initial approximation to hopefully get a more accurate answer.

We let $\Delta z = f(4.1,0.2) - f(4,0)$. The total differential $\dd z$ is approximately equal to $\Delta z$, so
\begin{equation}
f(4.1,0.2) - f(4,0) \approx \dd z
\quad \Rightarrow \quad
f(4.1,0.2) \approx \dd z + f(4,0).\label{eq:totaldiff2}
\end{equation}
To find $\dd z$, we need $f_x$ and $f_y$.
\begin{align*}
f_x(x,y) &= \frac{\sin y}{2\sqrt{x}} \quad\Rightarrow&
f_x(4,0) &= \frac{\sin0}{2\sqrt{4}}=0 \\
f_y(x,y) &= \sqrt{x}\cos y \quad\Rightarrow&
f_y(4,0) &= \sqrt{4}\cos0=2
\end{align*}
Approximating $4.1$ with 4 gives $\dd x = 0.1$; approximating $0.2$ with $0$ gives $\dd y=0.2$. Thus
\[
\dd z(4,0) = f_x(4,0)(0.1) + f_y(4,0)(0.2)
=0(0.1) + 2(0.2)
=0.4.
\]
Returning to \autoeqref{eq:totaldiff2}, we have
\[f(4.1,0.2) \approx 0.4 + 0 = .4.\]
We, of course, can compute the actual value of $f(4.1,0.2)$ with a calculator; to 5 places after the decimal, this is $0.40228$. Obviously our approximation is quite good.
\end{example}

The point of the previous example was \emph{not} to develop an approximation method for known functions. After all, we can very easily compute $f(4.1,0.2)$ using readily available technology. Rather, it serves to illustrate how well this method of approximation works, and to reinforce the following concept:
\begin{center}
	``New position = old position $+$ amount of change,'' so\\
	``New position $\approx$ old position + approximate amount of change.''
\end{center}

In the previous example, we could easily compute $f(4,0)$ and could approximate the amount of $z$-change when computing $f(4.1,0.2)$, letting us approximate the new $z$-value.

It may be surprising to learn that it is not uncommon to know the values of $f$, $f_x$ and $f_y$ at a particular point without actually knowing the function $f$. The total differential gives a good method of approximating $f$ at nearby points.

\begin{example}[Approximating an unknown function]\label{ex_totaldiff3}
Given that $f(2,-3) = 6$, $f_x(2,-3) = 1.3$ and $f_y(2,-3) = -0.6$, approximate $f(2.1,-3.03)$.
\solution
The total differential approximates how much $f$ changes from the point $(2,-3)$ to the point $(2.1,-3.03)$. With $\dd x = 0.1$ and $\dd y = -0.03$, we have
\begin{align*}
\dd z &= f_x(2,-3)\dd x + f_y(2,-3)\dd y\\
		&= 1.3(0.1) + (-0.6)(-0.03) \\
		&= 0.148.
\end{align*}
The change in $z$ is approximately $0.148$, so we approximate $f(2.1,-3.03)\approx 6.148$.
\end{example}

\subsection{Error/Sensitivity Analysis}

The total differential gives an approximation of the change in $z$ given small changes in $x$ and $y$. We can use this to approximate error propagation; that is, if the input is a little off from what it should be, how far from correct will the output be? We demonstrate this in an example.
\index{sensitivity analysis}\index{total differential!sensitivity analysis}

\begin{example}[Sensitivity analysis]\label{ex_totaldiff4}
A cylindrical steel storage tank is to be built that is 10ft tall and 4ft across in diameter. It is known that the steel will expand/contract with temperature changes; is the overall volume of the tank more sensitive to changes in the diameter or in the height of the tank?
\solution
A cylindrical solid with height $h$ and radius $r$ has volume $V = \pi r^2h$. We can view $V$ as a function of two variables, $r$ and $h$. We can compute partial derivatives of $V$:
\[\frac{\partial V}{\partial r} = V_r(r,h) = 2\pi rh \qquad \text{and}\qquad \frac{\partial V}{\partial h} = V_h(r,h) = \pi r^2.\]
The total differential is $\dd V = (2\pi rh)\dd r + (\pi r^2)\dd h.$ When $h = 10$ and $r = 2$, we have $\dd V = 40\pi \dd r + 4\pi \dd h$.
Note that the coefficient of $\dd r$ is $40\pi%\approx 125.7
$; the coefficient of $\dd h$ is a tenth of that%, approximately $12.57$
. A small change in radius will be multiplied by $40\pi$, whereas a small change in height will be multiplied by $4\pi$. Thus the volume of the tank is more sensitive to changes in radius than in height.
\end{example}

The previous example showed that the volume of a particular tank was more sensitive to changes in radius than in height. Keep in mind that this analysis only applies to a tank of those dimensions. A tank with a height of 1ft and radius of 5ft would be more sensitive to changes in height than in radius.

One could make a chart of small changes in radius and height and find exact changes in volume given specific changes. While this provides exact numbers, it does not give as much insight as the error analysis using the total differential.

\subsection{Differentiability of Functions of Three Variables}

The definition of differentiability for functions of three variables is very similar to that of functions of two variables. We again start with the total differential.

\begin{definition}[Total Differential]\label{def:total_differential3}
Let $w=f(x,y,z)$ be continuous on an open set $S$. Let $\dd x$, $\dd y$ and $\dd z$ represent changes in $x$, $y$ and  $z$, respectively. Where the partial derivatives $f_x$, $f_y$ and $f_z$ exist, the \textbf{total differential of $w$} is
\index{total differential}\index{partial derivative!total differential} 
\[\dd w = f_x(x,y,z)\dd x + f_y(x,y,z)\dd y+f_z(x,y,z)\dd z.\]
\end{definition}

This differential can be a good approximation of the change in $w$ when $w = f(x,y,z)$ is \textbf{differentiable}.

\begin{definition}[Multivariable Differentiability]\label{def:multi_differentiability3}
Let $w=f(x,y,z)$ be defined on an open set $S$ containing $(x_0,y_0,z_0)$ where $f_x(x_0,y_0,z_0)$, $f_y(x_0,y_0,z_0)$ and $f_z(x_0,y_0,z_0)$ exist. Let $\dd w$ be the total differential of $w$ at $(x_0,y_0,z_0)$, let $\Delta w = f(x_0+\dd x,y_0+\dd y,z_0+\dd z) - f(x_0,y_0,z_0)$, and let $E_1$, $E_2$ and $E_3$ be functions of $\dd x$, $\dd y$ and $\dd z$  such that
\index{differentiable}\index{derivative!multivariable differentiability}\index{multivariable function!differentiability}
\[\Delta w = \dd w + E_1\dd x + E_2\dd y + E_3\dd z.\]
\begin{enumerate}
	\item $f$ is \textbf{differentiable at $(x_0,y_0,z_0)$} if%, given $\epsilon >0$, there is a $\delta >0$ such that if $\norm{\bracket{\dd x,\dd y,\dd z}}< \delta$, then $\norm{\bracket{E_1,E_2,E_3}}<\epsilon$. 
	\begin{align*}
	 \lim_{(\dd x,\dd y,\dd z)\to(0,0,0)}E_1 &=0,\\
	 \lim_{(\dd x,\dd y,\dd z)\to(0,0,0)}E_2 &=0,\qquad\text{and}\\
	 \lim_{(\dd x,\dd y,\dd z)\to(0,0,0)}E_3 &=0.
	\end{align*}
	\item	$f$ is \textbf{differentiable on $S$} if $f$ is differentiable at every point in $S$. If $f$ is differentiable on $\mathbb{R}^3$, we say that $f$ is \textbf{differentiable everywhere}.
\end{enumerate}
\end{definition}

Just as before, this definition gives a rigorous statement about what it means to be differentiable that is not very intuitive. We follow it with a theorem similar to \autoref{thm:differentiable}.

% again, differentiability requires total derivative requires partial derivatives requires continuity
%\begin{theorem}[Continuity and Differentiability of Functions of Three Variables]\label{thm:differentiable3}
%Let $w=f(x,y,z)$ be defined on an open set $S$ containing $(x_0,y_0,z_0)$. 
%\begin{enumerate}
%\item	If $f$ is differentiable at $(x_0,y_0,z_0)$, then $f$ is continuous at $(x_0,y_0,z_0)$.
%\item If $f_x$, $f_y$  and $f_z$ are continuous on $S$, then $f$ is differentiable on $B$.
%\index{multivariable function!differentiability}\index{multivariable function!continuity}
%\end{enumerate}
%\end{theorem}

\begin{theorem}[Differentiability of Functions of Three Variables]\label{thm:differentiable3}
Let $w=f(x,y,z)$ be defined on an open set $S$ containing $(x_0,y_0,z_0)$. If $f_x$, $f_y$, and $f_z$ are continuous on $S$, then $f$ is differentiable on $B$.
\index{multivariable function!differentiability}\index{multivariable function!continuity}
\end{theorem}

This set of definition and theorem extends to functions of any number of variables. The theorem again gives us a simple way of verifying that most functions that we encounter are differentiable on their natural domains.\bigskip

This section has given us a formal definition of what it means for a functions to be ``differentiable,'' along with a theorem that gives a more accessible understanding. The following sections return to notions prompted by our study of partial derivatives that make use of the fact that most functions we encounter are differentiable.

\printexercises{exercises/12-04-exercises}