<jupyter_start><jupyter_text>Calculus Review: Introduction[Integrals](http://en.wikipedia.org/wiki/Integral) and [derivatives](http://en.wikipedia.org/wiki/Derivative) are widely used in scientific modeling and simulations.  Integration, along with its inverse operationdifferentiation, are the two main operations of calculus.  In this lecture notebook we will review some of the basic concepts from Calculus, including going all the way backto discussing functions and their graphs, and the concept of the slope of a line; looking at limits andderivatives, and how the slop of a function relates to the derivative; and then at the concept of integrationand integrals, and specifying and solving differential equations.This notebook is only meant to be a quick review.  It is hoped and expected that at some point you have studied at leastthe basics of these topics in secondary education and during your undergraduate degree.  Most students should beable to follow these materials and understand the important concepts who have at least a good understanding ofalgebra and geometry, especially equations of lines and functions, as a starting point to understandingthe calculus mathematics.The following suggested materials have more information and activities that you may use to supplement thisreview.  Some of the examples and materials used in this review come from examples in these links:- The [Kahn Academy courses on  Differential and Integral calculus](https://www.khanacademy.org/math/calculus-1?t=classes)    as well as [Integral Calculus](https://www.khanacademy.org/math/integral-calculus) would be good   reviews for weeks 7 and 8 materials.- There are multiple good online textbooks for studying or reviewing differential and integral Calculus.  There are several listed [here](https://www.math.ucdavis.edu/resources/learning/online-textbooks/)  (I may be assigning some readings from here when we review Probability and Linear Algebra as well).  I also like the [MIT Online Calculus book](https://ocw.mit.edu/resources/res-18-001-calculus-online-textbook-spring-2005/textbook/).- [Basic Calculus Refresher (UW-Madison)](http://pages.stat.wisc.edu/~ifischer/calculus.pdf) - a lot of the content   of this review came from this 20 page calculus refresher.- Kahn Academy [Introduction to Integral Calculus](https://www.khanacademy.org/math/ap-calculus-ab/ab-integration-new/ab-6-1/v/introduction-to-integral-calculus)- [Understanding Calculus in 10 min](https://www.youtube.com/watch?v=WjJ-kpgps1c)  Functions and GraphsWe have been using functions in both the mathematical sense and the programming sense extensively in thisclass.  In the mathematical sense, the simplest function is a function of a single independent variable$x$.  We often also use a dependent variable $y$ and say that "y is a **function** of x", written thisway\begin{equation}y = f(x)\end{equation}The simplest function we can imagine is a constant function, for example, \begin{equation}y = 3\end{equation}Here you may ask how is this a function of the independent variable $x$?  In this case it is not, $y$ doesnot depend on $x$, although we can also think of it like this:\begin{equation}y = 0 \cdot x + 3\end{equation}We could also graph this constant function.  It is a pretty uninteresting plot, since the value of $y$ is always 3 no matter what the value of $x$ is:<jupyter_code># Example of constant function, it doesn't matter what the
# independent variable x value is, y is always a constant 3
# y = 3
def f(x):
    return 0*x + 3


x = np.linspace(-3, 3)
plt.plot(x, f(x));
plt.ylim([-1, 4])
plt.grid();<jupyter_output><empty_output><jupyter_text>Slope of a Line and the Slope-Intercept EquationThe slope of a line is simply the ratio of the change in $y$, written as $\Delta y$ to the change in $x$, $\Delta x$\begin{equation}m = \frac{\Delta y}{\Delta x}\end{equation}(m is often used to represent the slop of a line).  For our previous equation $y = 0x + 3$ youshould see that the slope of the line is 0, because $y$ never changesthus the ratio is 0.  We say that the line has a slope of 0 in this case. Lets try a different function, say\begin{equation}y = 2x\end{equation}Here is the graph of this line.<jupyter_code>def f(x):
    return 2*x
    
x = np.linspace(-3, 3)
plt.plot(x, f(x));
plt.xlim([-6, 6])
plt.grid();<jupyter_output><empty_output><jupyter_text>Here the slope $m = 2$ because for every change of $x$ $\Delta x = 1$, $y$ will change by 2 $\Delta y = 2$and thus the ratio is $m = \Delta y / \Delta x = 2 / 1 = 2$If we add another term to the equation\begin{equation}y = 2x + 3\end{equation}The graph of the equation now looks like this:<jupyter_code>def f(x):
    return 2*x + 3

x = np.linspace(-3, 3)
plt.plot(x, f(x));
plt.xlim([-6, 6])
plt.grid();<jupyter_output><empty_output><jupyter_text>Here the slope of the line is still the same $m = 2$.  But when $x = 0$ $y$ has a value of $3$.Whenever you rearrange a linear function into the form $y = mx + b$, this is known as the slope-interceptform of the equation of the line.  $m$ will be the slope, which gives the ratio of how $y$ changes to the amountthat $x$ changes.  The constant $b$ will be the intercept of the line.  When $x = 0$ this will be the value of $y$or in other words this will be the intercept location, the value of $y$ when x changes from negative to positive (atthe point $x = 0$ ).At this point you should understand the following about the slope and the equation of a line- A line always has a constant slope.  At all places on the line, the slope is always the same $m$.  The  slope never changes.- The slope-intercept equation of a line allows you to directly see the slope of the line and the intercept  location where the line crosses the axis at $x = 0$.- A positive slope, like $+2$ above will describe a line that goes up, as $x$ gets bigger $y$ gets bigger.  You can of  course have a negative slope, in which case the line goes down, as $x$ gets bigger $y$ decreases.- A slope of 0 is a constant function, and defines a flat line as seen on its graph.  Other Functions of a Single VariableA line is a fairly simple function of a single varible.  We can of course have much more complex functions, evenif we still restrict ourself to functions of a single variable $f(x)$. Here are a few examples\begin{equation}y = x^2 + 2x + 1 \\y = \frac{2}{3}x^3 - 5x^2 + 5 \\y = x^{-1} \\y = 2^x\end{equation}The first 2 equations are examples of polynomial functions.  Anytime $y$ is a functions of a  power of $x$like $x^1, x^2, x^3  \cdots$ is an example of a polynomial function.  The first function with a power of $x^2$is known as a quadratic function and the second with $x^3$ is a cubic function.  The last equation is anexample of an exponential function (the $x$ appears in the exponent of one of the terms of the function).We can of course graph these equations in a similar way as we did for our linear equations.  They definea unique mapping from any value of $x$ to a unique single value of $y$, so by mapping many values of $x$ to $y$ over some rangewe can see the shape of the equation in that range.  Here are the quadratic and cubic equations from above plotted separately.<jupyter_code>def quadratic_function(x):
    return x**2

def cubic_function(x):
    return 2/3 * x**3 - 5 * x**2 + 5

x = np.linspace(-5, 5)

plt.subplots(1,2)
plt.subplot(121)
plt.plot(x, quadratic_function(x))
plt.grid();


plt.subplot(122)
plt.plot(x, cubic_function(x))
plt.grid();<jupyter_output><empty_output><jupyter_text>Limits and DerivativesThe higher order polynomial functions, and in fact most any more complex function of the variable $x$no longer have a constant slope.  The rate of change of the line at any point $x$ of the function willdepend on the function properties at that point.  Lets say we want to know what the slope is of the basic quadratic function $y = x^2$ is at the point where $x = 2$.We could use the equation for the slope of a line, and calculate the ratio of the change of $y$ to $x$like this:<jupyter_code>def f(x):
    return x**2

plt.figure(figsize=(10,10))
x = np.linspace(0, 5)
plt.plot(x, f(x));


# slope when x changes from 2 to 5 is
deltay = f(5.0) - f(2.0)
deltax = 5.0 - 2.0
m = deltay / deltax
print("slope when x changes from 2 to 5: ", m)
plt.plot([2.0, 5.0], [f(2.0), f(5.0)], 'k--')

# slope when x changes from 2 to 4 is
deltay = f(4.0) - f(2.0)
deltax = 4.0 - 2.0
m = deltay / deltax
print("slope when x changes from 2 to 4: ", m)
plt.plot([2.0, 4.0], [f(2.0), f(4.0)], 'k--')

# slope when x changes from 2 to 3 is
deltay = f(3.0) - f(2.0)
deltax = 3.0 - 2.0
m = deltay / deltax
print("slope when x changes from 2 to 3: ", m)
plt.plot([2.0, 3.0], [f(2.0), f(3.0)], 'k--')

# slope when x changes from 2 to 3 is
deltay = f(2.1) - f(2.0)
deltax = 2.1 - 2.0
m = deltay / deltax
print("slope when x changes from 2 to 2.5: ", m)
plt.plot([2.0, 2.1], [f(2.0), f(2.1)], 'k--')

# plot the tangent line, instantaneous slope is 4
# instantaneous slope (derivative) is 4, so line is y = 4x + b that goes through point (2,4), so y = 4x -4
# plot tangent at x=1.5 to x=3.5
x1 = 1.0
y1 = 4 * x1 - 4
x2 = 3.5
y2 = 4 * x2 - 4
plt.plot([x1, x2], [y1, y2], 'r-')
plt.plot(2, 4, 'ro');
plt.grid();

# example of slope as delta x decreases from 1 in powers of 10
x = 2
deltax = 10

for i in range(8):
    deltax = deltax / 10
    deltay = f(x + deltax) - f(x)
    m = deltay / deltax
    print("slope at x=%f for deltax=%0.16f is:%0.16f" % (x, deltax, m))<jupyter_output>slope when x changes from 2 to 5:  7.0
slope when x changes from 2 to 4:  6.0
slope when x changes from 2 to 3:  5.0
slope when x changes from 2 to 2.5:  4.099999999999998
slope at x=2.000000 for deltax=1.0000000000000000 is:5.0000000000000000
slope at x=2.000000 for deltax=0.1000000000000000 is:4.1000000000000014
slope at x=2.000000 for deltax=0.0100000000000000 is:4.0099999999998914
slope at x=2.000000 for deltax=0.0010000000000000 is:4.0009999999996992
slope at x=2.000000 for deltax=0.0001000000000000 is:4.0001000000078335
slope at x=2.000000 for deltax=0.0000100000000000 is:4.0000100000270322
slope at x=2.000000 for deltax=0.0000010000000000 is:4.0000010006480116
slope at x=2.000000 for deltax=0.0000001000000000 is:4.0000000911533098<jupyter_text>You should intuitively see that as we make the change in $x$ smaller ($\Delta x$) the lines they define that wemeasure the slope of are approaching the red line.  The red line represents the tangent to the curve right at the pointwe want to measure the slope $x = 2$.  For a quadratic curve like this one, the slope is not constant, it is constantlychanging.  Each location on the equation will have a slightly different slope.  But we can measure the"instantaneous slope", also called the "instantaneous rate of change" at a given point.  This is simply the slopeof the red line shown.  More formally, in the limit as $\Delta x$ gets smaller and smaller and eventually reaches0, the lines we measure the slope of get closer and closer to the red line, the tangent line which is the trueinstantaneous slope of the equation at the point $x = 2$.  Formally we can use the concept of a limit to specifythis idea:\begin{equation}m_{\text{tan}} = \lim\limits_{\Delta x \to 0} \frac{\Delta y}{\Delta x} = \lim\limits_{\Delta x \to 0} \frac{f(x + \Delta x) - f(x)}{\Delta x} \end{equation}This idea, denoted by the notation $\frac{dy}{dx}$ is called the **derivative** of the function $y = f(x)$ measured instantaneously at a given point $x$.  The derivative is another name for this notionof the measure of the instantaneous rate of change, or the slope of the function at a particular point onthe function.There are many different notations to represent this concept of the derivative:$\frac{d f(x)}{dx}$, $\frac{d}{dx} f(x)$, $f'(x)$. (The last one is sometimes referred to as "prime notation.") It is possible to come up with a formula that can be used to calculate the derivitaive of a function atany point.  For example, lets look quickly at the function $y = f(x) = x^2$ above. For any value of $x$,using the definition above, we have the following expression\begin{equation}m = \frac{\Delta y}{\Delta x}   =  \frac{(x + \Delta x)^2 - x^2}{\Delta x}   = \frac{x^2 + 2x \Delta x + (\Delta x)^2 - x^2}{\Delta x}  = \frac{\Delta x (2x + \Delta x)}{\Delta x}  = 2x + \Delta x\end{equation}The previous sequence may look complicated, but we are using simple algebra to expand and cancel out terms, justwith the maybe unfamiliar term $\Delta x$ in the expressions.  The result of this rearrangement though showssomething interesting.  This says simply that the slope of any line at any point of the functionis $2x + \Delta x$ for a given $\Delta$.  But also it says, in the limit, when $\Delta x = 0$, theinstantaneous slope will be simply $2x$.  Or in other words we have shown that for our function $f(x) = x^2$\begin{equation}\frac{d f(x)}{dx} = 2x\end{equation}This means that the derivative (instantaneous rate of change) of the function is $2x$ at any point $x$of this function.  We can use this expression for the derivative of our function $y = x^2$to determine the instantaneous rate of change at any point of our function.  For example:<jupyter_code>def f(x):
    """The simple quadratic function we are using y = x^2
    """
    return x**2

def df(x):
    """The derivative of our equation, can be used to determine the instantaneous
    rate of change, or in other words the slope, of the equation at any particular
    point x.
    """
    return 2 * x;

def plot_tangent_line(x):
    """A little utility function, this will plot a tangent line
    on the current figure at the location x, making use of the
    slope of the function f(x) using its derivative df(x), and
    plotting a line that goes through x with that slope
    """
    # determine value of y at the point x, and the slope of the line
    # at point x
    y = f(x)
    m = df(x)
    
    # y = mx + b
    # we have y and m for our needed line, need to determine
    # correct value of be, so rearrange
    # b = y - mx
    # and determine the correct intercept of the tangent line
    b = y - m * x
    
    # we will plot a line from -0.5 to +0.5 of x
    deltax = 0.75
    x0 = x - deltax
    x1 = x + deltax
    
    # we have the equation of the line, and we have two point x0, x1, determine
    # the points y0, y1 on the line
    y0 = m * x0 + b
    y1 = m * x1 + b
    
    # now we can plot the tangent line
    plt.plot([x0, x1], [y0, y1], 'r--')
    
    # plot a marker at the tangent point
    plt.plot(x, y, 'ro')
    
    # annotate the plot
    plt.text(x+0.05, y, 'x = %f\nm = %f' % (x, m))
    
    
# plot the original function
x = np.linspace(-5.0, 5.0)
plt.plot(x, f(x))
plt.grid();

# plot some tangent lines at different points, using the derivative (slope) of
# the function to do it.  We use a small utility function to plot several
# example tangent lines
plot_tangent_line(-4)
plot_tangent_line(-2)
plot_tangent_line(0)
plot_tangent_line(2)
plot_tangent_line(4)<jupyter_output><empty_output><jupyter_text>General Rules of DerivationUsing similar methods to this, we can also analyze functions like $y = x^3$ or $y = x^4$.  In general if you dothis you can easily see that a general rule exists for any power function $x^p$.  Namely if $y = f(x) = x^p$then**Power Rule**\begin{equation}\frac{d}{dx} (x^p) = p x^{p-1}\end{equation}This means that for example for the function $y = x^5$ the derivative of this function can be calculatedas\begin{equation}\frac{dy}{dx} = f'(x) = 5 x^{4}\end{equation}Likewise through similar analysis it is possible to derive other general rules of derivation.  Here are a fewthat are useful:**Exponential Rule**If $y = f(x) = b^x$ then\begin{equation}\frac{d}{dx} (b^x) = b^x \ln(b)\end{equation}If $y = f(x) = e^{ax}$ then\begin{equation}\frac{d}{dx} (e^{ax}) = a e^{ax}\end{equation}**Logarathim Rule**if $y = f(x) = \log_b(x)$ then\begin{equation}\frac{d}{dx} (\log_b(x)) = \frac{1}{x} \frac{1}{\ln(b)}\end{equation}as a special case, when $(b = e)$ then\begin{equation}\frac{d}{dx} (\ln(x)) = \frac{1}{x}\end{equation} Properties of DerivativesGiven some general rules like this, we can define several properties of derivatives, that allow us todetermine exact equations for the derivatives of many different functions and combinations of functions. 1. Derivative of a **constant**2. **Sum and Difference Rules**3. **Product Rule**4. **Quotient Rule**5. **Chain Rule**In a introductory calculus class you often spend a lot of time deriving the derivativeof mathematical expressions and learning to apply these rules to determine the symbolicderivative for many types of functions.  In this class we do not really calculatesymbolic solutions for the derivatives of functions, because we are learning aboutcomputational (approximate) methods to calculate things like the derivative offunctions.  So if you are rusty or haven't learned or practiced how to determine symbolicallythe expression for the derivative of a function, it is not really that important for thisclass, as long as you understand the basic concept of a derivative and what it isand what it means.  With a solid understanding of the concept of a derivative, we canthen better understand the uses we might put this concept to to solve other types of problems, including its relationship to the concept of an integral.  Applications of Derivatives: Roots and Minima / Maxima Finding roots of an EquationIt is often useful to find what are known as the **roots** of an equation, i.e. the location where a function$f(x) = 0$, or in other words the values where the graph of $f(x)$ intersects the x-axis.  Algebraically this canbe extremely tedious or even impossible, so we often use numerical techniques.  We will later look at a few suchroot finding techniques.  The most basic technique known as the Newton method, or the Netwon-Raphson method, startswith an initial guess $x_0$ and then produces a sequences of values (i.e.it defines a series) that converges to a numerical solutionof a root.  By iterating the expression\begin{equation}x_i = x_{i-1} - \frac{f(x_{i-1})}{f'(x_{i-1})}\end{equation}Take for example the function\begin{equation}f(x) = x^3 - 21 x^2 + 135x - 220\end{equation}If we explore the graph of this funtion, it appears to have a root somewhere between 2.4 and 2.6<jupyter_code>def f(x):
    return x**3 - 21 * x**2 + 135 * x - 220

x = np.linspace(2.0, 3.0)
plt.plot(x, f(x))
plt.grid();<jupyter_output><empty_output><jupyter_text>Starting with an initial value of $x_0 = 2$ we can iterate using the previous described Newton's methodlike this:<jupyter_code>def f(x):
    return x**3 - 21 * x**2 + 135 * x - 220

def df(x):
    """Derivative of f(x)
    """
    return 3 * x**2 - 42 * x + 135

NUM_ITER = 10
x = np.zeros(NUM_ITER)
x[0] = 2.0
print(0, x[0])
for i in range(1, NUM_ITER):
    x[i] = x[i-1] - f(x[i-1]) / df(x[i-1])
    print(i, x[i])<jupyter_output>0 2.0
1 2.4126984126984126
2 2.461290404582591
3 2.461940602324453
4 2.4619407179495005
5 2.461940717949505
6 2.4619407179495045
7 2.461940717949506
8 2.4619407179495045
9 2.461940717949506<jupyter_text>We can confirm that this final value in the sequence is indeed a root of the equationby plugging it back into the function.<jupyter_code>print(x[9], f(x[9]))<jupyter_output>2.461940717949506 5.684341886080802e-14<jupyter_text>Since this is a numerical method, the result won't be exactly 0, but the functionshould essentially be very close to 0 at the point we numerically determined is aroot.  You can also check the graph of the function below and see that the functioncrosses the x axis at this value of x.To understand why this works, look at the expression for the series.  We aresubtracting the expression\begin{equation}\frac{f(x)}{f'(x)}\end{equation}each time to come up with the next value in the series.  Remember we are trying to findthe root of the equation $y = f(x)$.  So when $f(x)$ is 0 or close to 0, thisexpression is going to be 0 or close to 0, and we won't be subtracting much from thecurrent value in the series.  Also we need to combine information about whether $f(x)$ is positive or negativealong with information about whether the derivative (slope) is positive or negativeto determine whether we want to make $x$ bigger or smaller to move in adirection closer to a root.  Look at the figure for our example function.  At thestarting point of $x_0 = 2$, the function $f(x) < 0$  thus $f(x)$ is negative.The root in our vicinity is to the right (we need $x$ to increase).  The fact thatthe derivative (slope) is positive at the point $x_0 = 2$ means that if we increase $x$ we should be moving up the curve to a location closer to where $f(x) = 0$.  Thusby combining information about the sign of $f(x)$ and the sign of the slope$f'(x)$ this lets us determine if we need to increase or decrease our next valueof $x$.  In this particular case, since $f(x)$ is negative and the derivative $f'(x)$is positive, the expression ends up being negative, and since we are subtracting a negativewe end up increasing $x$ in the next step.  If instead we had started at$x_0 = 3$, the function $f(x)$ would be positive at this location, and the derivativewould also be positive, which would indicate we need to subtract some amount from$x$ to go towards the root.  The same argument/intuition can be made for when thederivative is negative, in which case which direction you need to go to approach theroot will be reversed from our example here, but again the general Newton's methodexpression will work to move you closer to the root of the equation.  Minimum and Maximum of EquationsOften it is useful to be able to "minimize" or "maximize" a function.  This meansthat we want to find the location where the function is at its largest or smallestvalue (often restricted to some range).A function will be at a local maximum or minimum when its derivative is 0. \begin{equation}f'(x) = 0\end{equation}Take for example again the same function we have been using\begin{equation}f(x) = x^3 - 21 x^2 + 135x - 220\end{equation}<jupyter_code>def f(x):
    return x**3 - 21 * x**2 + 135 * x - 220

x = np.linspace(2, 12, 1000)
#x = np.linspace(2.45, 2.47)
plt.plot(x, f(x))
plt.grid();<jupyter_output><empty_output><jupyter_text>If we take the derivative of the function and set it to 0 to findthe locations of the minima and maxima we have\begin{equation}f'(x) = 3x^2 - 42x + 135 \\3x^2 - 42x + 135 = 0 \\3(x - 5) (x - 9) = 0\end{equation}We could have used the quadratic equation to determine the solution to setting ourderivative to 0, but in this case we can also relatively easily factor the quadraticexpression to easily see where the roots must be located.Thus wee see that when $x = 5$ or $x = 9$ these will be solutions to this equation,and from the graph we see that these are indeed the locations of the minimum and themaximum of the function.  Integrals of FunctionsA definite integral of a continuous function $f(x)$ is a measure of the area under the graph of $f$ insome interval $[a, b]$.  The integral of a function is related to the derivative of the function.  It is nottoo difficult to derive this association.  Lets say you have a continuous function $f(x)$ and you want tofind the area under the curve of this graph in the interval $[a, x]$ from some *fixed* lower value $a$ toany *variable* upper value $x$.<jupyter_code>def f(x):
    """An example of a specific function we want to find the area under the
    graph of.
    """
    return x * (x - 3) * (x - 6) + 20

plt.figure(figsize=(16, 12))
x = np.linspace(0, 6.25)
plt.plot(x, f(x), 'k-', linewidth=3)
plt.grid();

# a = 0, x = 4, deltax = 2
# represent F(x), function to calculate area from fixed a to x
a = 0
b = 4
deltax = 2
z = 5.35
x = np.linspace(a, b)
plt.fill_between(x, f(x), color='lightblue')

# represent F(x + deltax) - F(x)
deltax = 2
x = np.linspace(b, b+deltax)
plt.fill_between(x, f(x), color='lightgreen');

# label the graph
# first remove the specific tics and set general/example tic labels
plt.yticks([0])
plt.xticks([a, b, z, b+deltax], ['$a$', '$x$', '$z$', '$x + \Delta x$'], fontsize=20);

# label the functions
plt.xlim([0, 7.6])
plt.text(2.75, 25, '$y = f(x)$', fontsize=20)
plt.text(1.5, 12, '$F(x)$', fontsize=20)
plt.text(6.25, 15, '$f(z)$', fontsize=20)
plt.arrow(6.25, 15, z-6.05, f(z)-14.4, head_width=0.1, head_length=0.5, color='black')
plt.text(6.25, 7, '$F(x + \Delta x) - F(x)$', fontsize=20)
plt.arrow(6.25, 7, -0.5, 0, head_width=0.2, head_length=0.1, color='black')

# add lines for width and height of area
plt.plot([z, z], [0, f(z)], 'k--')
plt.plot([b, b+deltax], [f(z), f(z)], 'k--');<jupyter_output><empty_output><jupyter_text>Let $F(x)$ = Area under the graph of $f$ in the interval $[a, x]$.then $F(x + \Delta x)$ = Area under the graph of $f$ in the interval $[a, x + \Delta x]$if we take the difference of these two areas we have \begin{equation}F(x + \Delta x) - F(x) = \text{Area under the graph of} f \; \text{in the interval} \; [x, x + \Delta x]\end{equation}The area of this difference (the green bit of the figure) has to be equal to the area of a rectangle with height $f(z)$,where $z$ is some value in the interval $[x, x + \Delta x]$, and with width of $\Delta x$,thus the area of the previous difference can be written as:\begin{equation}F(x + \Delta x) - F(x) = f(z) \Delta x\end{equation}rewriting we have\begin{equation}\frac{F(x + \Delta x) - F(x)}{\Delta x} = f(z)\end{equation}This should look a little familiar.  This is the finite difference equation we used (but with the function $F$) indeveloping the concept of the derivative.  If we take this expression in the limit again as $\Delta x \to 0$we see that the right hand side $f(z)$ becomes $f(x)$ (because z is some value in the interval from$[x, x + \Delta x]$, and as $\Delta x \to 0$, $z$ simply becomes $x$).  Thus we have\begin{equation}\frac{d}{dx} F(x) = f(x)\end{equation}$F$ is called the **antiderivative** of $f$.  In other words, $F$ is the function suchthat, if you take the derivative of it $F'(x)$ you will end up the the function$f(x)$.  In essence, if you do the inverse of the process of determiningthe derivative of a function, we can use this antiderivative to calculate values of the function $F$ in order toexactly measure the area under the curve of functions $f$.We formally express this concept of calculating the area under the graph of a function $f$ from itsantiderivative using the symbol for the definite integral of a function like this:\begin{equation}F(x) = \int_a^b f(x) \; dx\end{equation} Properties of IntegralsIntegrals possess the analogues of the properties of derivatives.  For example the antiderivativeof a constant and of the sum and difference of functions work in the same way as for derivatives.The Power Rule, Logarithm Rule and Exponential Rule work basically in an inverted manner to calculatethe antiderivative of a function. Calculating Areas using Definite IntegralsIf you can calculate the antiderivative of a function $f$, you can use this to calculate the area underthe graph of the curve of the function $f$ on any finite interval $[a, b]$.For example, take the function\begin{equation}f(x) = x^3 (1 - x^4)^2\end{equation}The graph of this function in the interval $[0, 1]$ looks like this.<jupyter_code>def f(x):
    return x**3 * (1 - x**4)**2

a = 0
b = 1.0
x = np.linspace(a, b, 1000)
plt.plot(x, f(x), 'k-', linewidth=3)
plt.fill_between(x, f(x))
plt.grid();<jupyter_output><empty_output><jupyter_text>As a rough estimate, since the maximum height of the function is 0.2, we know that the area of the box is 0.2.The area under the graph looks like it is probably less than $1/2$ of the total area of this bounding box, soI would expect an area of a bit under 0.1 for this function.We can calculate the antiderivative of the function.  This function is a polynomial, so it is probably easiestto simply exapand the terms and then calculate the antiderivative of the sum/difference of the terms, like this:\begin{equation}\int_0^1 x^3 (1 - x^4)^2 \; dx \\\int_0^1 x^3 (1 - 2x^4 + x^8) \; dx \\\int_0^1 x^3 - 2x^7 + x^{11} \; dx \\\end{equation}The antiderivative of simple terms raised to a power will use the reverse of the Power Rule, ths we have theantiderivative\begin{equation}F(X) = \Big[ \frac{x^4}{4} - \frac{2 x^8}{8} + \frac{x^{12}}{12} \Big]_0^1\end{equation}(We leave it as an exercise for the student to prove to yourself that if youtake the derivative $\frac{d}{dx} F(X)$ you get the original function $f(x)$).)Notice the notation of the 0 and 1 on the expression for the antiderivative here.  To use thisantiderivative to calculate the area under the curve exactly, you take\begin{equation}F(b) - F(a)\end{equation}thus we can calculate the area of this particular function in the interval $[a=0, b=1]$ as\begin{equation}\Big[ \frac{1^4}{4} - \frac{2 \cdot 1^8}{8} + \frac{1^{12}}{12} \Big] - \Big[ \frac{0^4}{4} - \frac{2 \cdot 0^8}{8} + \frac{0^{12}}{12} \Big] =\frac{1}{12} - 0 = \frac{1}{12} = 0.083333\end{equation}This is the general procedure for calculating a definite integral symbolically(an exact solution).  You first need to derive the expressionfor the antiderivative, using the inverse of the derivative rules, if you can.  If such an antiderivateexpression is possible, then the area under the curve for any definite interval can be calculated by takingthe difference of the antiderivative on the two ends of the interval.  Acknowledgements and Versions<jupyter_code>%load_ext version_information

%version_information numpy, scipy, matplotlib<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1028160453808752, 'fraction_numerical': 0.0398095623987034, 'mean_word_length': 4.124048442906575, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 50, 'https://': 6, 'lorem ipsum': 0, 'www.': 5, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 15, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '568694', 'license': 'CC-BY-3.0', 'n_tokens_mistral': 10067, 'n_tokens_neox': 9151, 'n_words': 4292, 'path': 'lectures/review/U00-3-Calculus-Review.ipynb', 'repo_name': 'tgrasty/CSCI574-Machine-Learning', 'size': '29616'}