<jupyter_start><jupyter_text>Introduction Since Newton invented calculus, differentiating a function has been essential to the advancement of humanity. Calculating the derivative of a function is crucial to finding the extrema for a function and determining zeros for a function, two operations that are central to optimization (1). Optimization is essential, especially due to the exploding popularity of machine learning, where minimizing a loss function is the essential task. Often, we can find the symbolic/analytical solution to the derivative of a function, however this has become increasingly complex and computationally expensive as our functions/equations have grown in size and complexity. Numerically solving differential equations forms a cornerstone of modern science and engineering and is intimately linked with machine learning; however this method suffers from rounding errors and numerical instability. Many of these issues can be solved using Automatic Differentiation (AD) because AD can calculate the exact derivative up to machine precision (2). The logic and processes behind AD enables it to be implemented using computer code, making it easily accessible for use by scientists and mathematicians. This python package will implement the forward mode of AD.   How to use the package  Installation  Download from PyPI (for basic users)Ideally, the user should have Anaconda installed (https://www.anaconda.com/download/).The user can, but does not need to, create a virtual environment using `virtualenv`. If the user has Anaconda installed, then the user will already have `virtualenv` installed, but if they do not, the user can the user can run the following command in the terminal:    sudo easy_install virtualenvThen, to create a virtual environment, the user runs the following command in the terminal:    virtualenv envTo activate the virtual environment, the user runs the following command in the terminal:    source env/bin/activateTo deactivate the virtual environment, the user runs the following command in the terminal:    deactivateWhether or not the user uses a virtual environment, they will need to install the package.    pip install autodiff-pyAll the necessary dependencies will be installed along with `autodiff-py`. The user can then start using the package in their code with an import statement such as    import autodiff as ad     Download from Github repo (for developers)Ideally, the a developer should have Anaconda installed (https://www.anaconda.com/download/).To install the package, a developer will download or clone the repository onto their local machine. Then, the user can, but does not need to, create a virtual environment using `virtualenv`. If the user has Anaconda installed, then the user will already have `virtualenv` installed. If the user is not using Anaconda, he or she can run the following command in the terminal:    sudo easy_install virtualenvThen, to create a virtual environment, the user runs the following command in the terminal in the package directory:    virtualenv envTo activate the virtual environment, the user runs the following command in the terminal in the package directory:    source env/bin/activateTo deactivate the virtual environment, the user runs the following command in the terminal in the package directory:    deactivateWhether or not the user uses a virtual environment, he or she will need to have the necessary dependencies. The package directory includes a *requirements.txt* file that lists the necessary dependencies. The user can easily install all the dependencies with the following command:    pip install -r requirements.txtAfter installing the necessary dependencies, the user can then start using the package in their code with an import statement such as    import autodiff as adTo run the tests for the package, the user can run the following command in terminal in the package folder:    pytest<jupyter_code>#install the package autodiff-py
!pip install autodiff-py<jupyter_output><empty_output><jupyter_text>Usage For example, suppose $f(x, y) = x^2 + 2xy + y^2$, and $x = 1, y = 2$.​$f(1, 2) = 9\\\frac{\partial f}{\partial x} = 2x + 2y \implies \frac{\partial f}{\partial x}|_{x = 1, y = 2} = 6 \\\frac{\partial f}{\partial y} = 2x + 2y \implies \frac{\partial f}{\partial y}|_{x = 1, y = 2} = 6 $​To solve this problem using our package, the user will run the following straightforward code.<jupyter_code>import autodiff as ad

# Define x and y scalars with variable name and value. The actual python variable names do not matter.
x = ad.Scalar('x', 1)
y = ad.Scalar('y', 2)
# Express f in terms of x and y. User does not need to define any more scalar objects besides the basic ones x and y above. 
f = x ** 2 + 2 * x * y + y ** 2
# Get value of f(1, 2)
f.getValue()
# Get partial derivative with respect to x. Pass in variable names as String in list.
f.getGradient(['x'])
# Get partial derivative with respect to y
f.getGradient(['y'])
# Get both partial derivatives
f.getGradient(['x', 'y'])
# Get all partial derivatives for the function 'f'
f.getDeriv()<jupyter_output><empty_output><jupyter_text>Also included in our package is an optimization library, `optimize`. The package includes Newton's method for root finding and also Quasi-Newton's methods and gradient descent for minimum finding. The benefit of this package is that the user does not need to use any functionality related to `autodiff-py` themselves; they just need to define the function (which only takes in one required argument) they want to optimize/find the root of and give the initial guess as a list/array. For example, suppose we want to find the root of the function $\vec{f}(x_1, x_2, x_3) = [2x_1 + 1, -2x_1 - x_2^2 + 3, -3 x_3 + 1]$. The roots of this function will be $(x_1, x_2, x_3) = (-0.5, 2, 1/3)$ and $(x_1, x_2, x_3) = (-0.5, -2, 1/3)$. We can use the function `newtons_method` to solve for a root. The root found will change depending on the initial starting position. We can also specify the method used to calculate the step size for each iteration in `newtons_method`.<jupyter_code>import autodiff.optimize as optimize
import numpy as np

def f(x):
    x1 = x[0]
    x2 = x[1]
    x3 = x[2]
    return np.array([2 * x1 + 1, -2 * x1 - x2 ** 2 + 3, -3 * x3 + 1])
#Returns two elements in a tuple. First element corresponds to position of root, the second the number of iterations
#it took to converge
#First root
optimize.newtons_method(f, [1, 1, 0])[0]
#Second root
optimize.newtons_method(f, [1, -1, 0])[0]
#Use gmres action step update method
optimize.newtons_method(f, [1, 1, 0], method = 'gmres_action')[0]
optimize.newtons_method(f, [1, -1, 0], method = 'gmres_action')[0]<jupyter_output><empty_output><jupyter_text>Suppose we want to find the minimum of the function $f(x_1, x_2) = 2(x_1-1)^2+3x_2^2$, which is found at $(x_1, x_2) = (1, 0)$. We can use the function `gradient_descent` or `quasi_newtons_method` depending on the method we want. `quasi_newtons_method` can solve for the minimum using one of the specified Quasi-Newton's methods.<jupyter_code>def f(x):
    return 2 * ((x[0] - 1) ** 2) + 3 * (x[1] ** 2)
#Returns two elements in a tuple. First element corresponds to position of minimum, the second the number of iterations
#it took to converge
#Gradient descent
optimize.gradient_descent(f, [11, 41])[0]<jupyter_output><empty_output><jupyter_text>Background​The following mathematical concepts and background are required for understanding automatic differentiation:​ 1. Differential calculus​Differential calculus is a subfield of calculus concerned with the study of the rates at which quantities change.For example, given the function: \begin{align} f\left(x\right) &=  {x^{2}}      \end{align}  Increment x by h: \begin{align} f\left(x+h\right) &=  {(x+h)^{2}}      \end{align}  Apply the finite difference approximation to calculate the slope:  \begin{align} \frac{f\left(x+h\right) - f\left(x\right) }{h} \end{align} Simplify the equation:  \begin{align} &= \frac{x^{2}+2xh+h^{2}-x^{2} }{h}\\ &= \frac{2xh+h^{2}}{h}\\ &=2x+h \end{align}  Set $h\rightarrow 0$:   \begin{align} 2x +0 &= 2x  \end{align}  The derivative is then defined is:\begin{align} \lim_{h\to0} \frac{f\left(x+h\right) - f\left(x\right) }{h} \end{align}  2. Elementary functions and their derivatives|       Function $f(x)$                |       Derivative $f^{\prime}(x)$                || :-------------------:  | :------------------------------------------------------------------------------:  || ${c}$           | $0$         || ${x}$           | $1$         || ${x^{n}}$           | ${nx^{n-1}}$         || $\frac{1}{x}$     | $\frac{-1}{x^{2}}$     || $ln{x}$     | $\frac{1}{x}$     || $\sin(x)$           |   $\cos(x)$         || $\cos(x)$           |   $-\sin(x)$         || $\tan(x)$           |   $\dfrac{1}{\cos^2(x)}$         || $\exp(x)$           |   $\exp(x)$         || ${a^{x}}$           |   ${a^{x}\ln{a}}$         |  3. The chain rule$^{(1)}$For a function $h(u(t))$, the derivative of $h$ with respect to $t$ can be expressed as:$$\dfrac{\partial h}{\partial t} = \dfrac{\partial h}{\partial u}\dfrac{\partial u}{\partial t}.$$If the function is expressed as a combination of multiple variables that are expressed in terms of t, i.e. $h(u(t), v(t))$, the the derivative of $h$ with respect to $t$ can be expressed as:$$\frac{\partial h}{\partial t} = \frac{\partial h}{\partial u}\frac{\partial u}{\partial t} + \frac{\partial h}{\partial v}\frac{\partial v}{\partial t}$$Note that we are only looking at scalar variables in this case, but this idea can be extended to vector variables as well.  For any $h\left(y\left(x\right)\right)$ where $y\in\mathbb{R}^{n}$ and $x\in\mathbb{R}^{m}$,    \begin{align}    \nabla_{x}h = \sum_{i=1}^{n}{\frac{\partial h}{\partial y_{i}}\nabla y_{i}\left(x\right)}.  \end{align} 4. The graph structure of calculations and forward accumulationForward accumulation is computing the derivative using the chain rule starting from the inner most derivative to the outer most derivative, where we assume the most basic variables have seed values. Using a graph helps visualize forward accumulation. For example,\begin{align} f\left(x,y\right) &= \frac{x}{y} +cos(x)sin(y)\\ x &= y = 1\end{align} ![](img/graph_eg.png)| Trace | Elementary Function | Current Value | Elementary Function Derivative | &nbsp; &nbsp; $\nabla_{x}$ Value &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; $\nabla_{y}$ Value &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| :---: | :-----------------: | :-----------: | :----------------------------: | :-----------------: | :-----------------: | :-----------------: || $w_{1}$ | $1$ | $1$ | $\dot{w_1}$ | $1$ | $0$ || $w_{2}$ | $1$ | $1$ | $\dot{w_2}$ | $0$ | $1$ || $w_{3}$ | $cos{(w_1})$ | $cos{(1)}$ | $-sin{(w_1)}\dot{w_1}$ | $-sin(1)$ | $0$ || $w_{4}$ | $sin{(w_2})$ | $sin{(1)}$ | $cos{(w_2)}\dot{w_2}$ | $0$ | $cos{(1)}$ || $w_{5}$ | $w_3\dot w_4$ | $sin{(1)}cos{(1)}$ | $w_4\dot{w_3} + w_3\dot{w_4}$ | $-sin^2{(1)}$ | $cos^2{(1)}$ || $w_{6}$ | $w_1 / w_2$ | $1$ | $\dot{w_1}/w_2 - w_1 \dot{w_2}/ w_2^2$ | $1$ | $-1$ || $w_{7}$ | $w_5 + w_6$ | $sin{(1)}cos{(1)} + 1$ | $\dot{w_5} + \dot{w_6}$ | $-sin^2{(1)} + 1$ | $cos^2{(1)}-1$ |  5. OptimizationOften times, we want to optimize a convex function, which is a function where a line segment between any two points on the graph of the function lies above or on the graph. A differentiable function is at a critical point when the derivative/gradient is equal to $0$. For convex functions, there is a minimum at the critical point. Thus, we are interested in finding when the derivative/gradient of a function is equal to $0$. It is important to understand that many optimization/root-finding methods are iterative in nature; in iteration $k$ the position is changed using step: $x_{k+1} = x_{k} + \Delta x_k$. In root-finding methods, we can solve for $\Delta x_k$ using the following equation:$J_f\Delta x_k  = -f(x_k)$where $J_f$ is the Jacobian, the matrix of first order derivatives, of $f(x)$ at iteration $k$.Similarly, in most optimization methods, we can solve for $\Delta x$ using the following equation:$B_k\Delta x_k = -\alpha \nabla f(x_k)$where $B$ is the Hessian, the matrix of second order derivatives, of $f(x)$ at iteration $k$, $\alpha$ is found through backtracking line search, and $\nabla f(x)$ is the gradient of $f(x)$In small cases, solving for the inverse of $J$ and $H = B^{-1}$ would not be a problem. However, as the dimension gets larger and larger, solving for the inverse becomes very computationally intensive. As such, one must find other ways of solving for $\Delta x$. One could use methods such as LU decomposition or the Generalized minimal residual method (Gmres) to solve for it. For optimization, one could also use Quasi-Newton methods to estimate the inverse of $B$. At each iteration, the inverse of the Hessian can be approximated using an update equation. For example, for DFP, to estimate $H_k$, the inverse of $B_k$, $H_k = H_k + \frac{\Delta x_k \Delta x_k^T}{\Delta x_k^T y_k} - \frac{H_k y_k y_k^T H_k}{y_k^T H_k y_k}$where $y_k = \nabla f(x_{k+1}) -  \nabla f(x_{k})$More update equations can be found on Wikipedia's Quasi-Newton method page (3).  Software Organization The directory structure is:        ``` autodiff\          autodiff\                    __init__.py                    functions.py                    scalar.py                    optimize.py                    vector.py          tests\                    test_composite.py                    test_functions.py                    test_scalar.py                     test_optimize.py                    test_vector.py          docs\                    milestone1.ipynb                    milestone2.ipynb                    Final_milestone.ipynb          README.md          requirements.txt          setup.cfg          LICENSE                ```   **Basic modules and what they do?**This module aims to compute forward automatic differentiation. The module we implemented can efficiently compute the derivatives of a function with automatic differentiation of a scalar input. The *scalar.py* file contains the objects which compute the value and the derivative of scalar variables. The dunder methods add, sub, mul, truediv, pow, iadd, isub, imul, idiv, ipow are implemented in this module. The *function.py* file contains sine, cosine, power, exponential, and other mathematical functions. Both *scalar.py* and *function.py* return the value and the derivative.The *optimize.py* file contains functions for find the roots and minima of user-defined functions.**Where do tests live? How are they run? How are they integrated?**We are using both `TravisCI` and `Coveralls` to test our module. Each test function exists in the tests folder, and it utilizes the `pytest` package.**How can someone install your package?**Our package has been released in `PyPI` under the name `autodiff-py`. Therefore, our auto-differentiating software can be downloaded at https://github.com/cs207FinalProjectGroup/cs207-FinalProject.git.  Implementation Details  *Scalar* ClassThis package relies on the *Scalar* class, which represents scalar variables. To initialize a *Scalar* class object, the user will pass in a string that represents the variable name (i.e. 'x', 'y', 'x1', etc.) and the value of the variable. The user also has the option to specify the starting value for the derivative; if a starting value for the derivative is not specified, then it defaults to $1$. A *Scalar* object holds two attributes: 1) the value of the variable `_val` at the current step and 2) a dictionary `_deriv` containing the derivative or partial derivatives (keys will be the names of the variables (i.e *x* and *y*) and the values will be the derivative value with respect to each variable). The user should utilize the methods `getValue()` and `getDeriv()` to obtain the value and derivative(s) of their Scalar object.<jupyter_code>import autodiff as ad

x = ad.Scalar('x',2);
print('Value of x: ', x.getValue()); #should be 2.0
print('Derivative of x:', x.getDeriv()['x']); #should be 1.0

y = ad.Scalar('y', val=5, deriv=3.2);
print('Value of y: ', y.getValue()); #should be 5.0
print('Derivative of y: ', y.getDeriv()['y']); #should be 3.2<jupyter_output>Value of x:  2.0
Derivative of x: 1.0
Value of y:  5.0
Derivative of y:  3.2<jupyter_text>Storing the partial derivatives with respect to each variable allows us to easily compute additional derivatives with respect to each variable when we are performing mathematical operations because we can update each partial derivative individually. When a *Scalar* object is initialized, by default `_deriv` will just be a dictionary with the only key being the string the user passes in with value 1. A user can access the value of a *Scalar* object using the *getValue()* method and access the derivative (or partial derivatives) for the object through the *getDeriv()* method. The user can also get the derivatives/partial derivatives as a numpy array with the *getGradient()* method, which takes in a list of strings, with each element representing the variable to take the derivative with respect to, as an argument. The dunder methods __add__, __sub__, __mul__,  __truediv__, __pow__, __iadd__, __isub__, __imul__, __itruediv__, __ipow__ (and the right equivalents for the ones that have one) have been overwritten so that they return a new *Scalar* object with an updated value and derivatives. Thus, the adding or substracting two *Scalars* or raising a *Scalar* to the power of another *Scalar* does not change the values or derivatives of the original *Scalar* objects. By overwriting these methods, we are implementing forward accumulation, as the orders of operation allows us to traverse the chainrule starting from the inside.<jupyter_code>#addition example
x,y = ad.Scalar('x', 2),ad.Scalar('y', 5);
val = x+y;
print(val.getValue()); #should be 7.0
print(val.getDeriv()['x']); #should be 1.0
print(val.getDeriv()['y']); #should be 1.0
print();
print(x.getValue(),',', y.getValue()); #x, y should retain original values
#pow example
x=ad.Scalar('x', 2);
val = x**2;
print(val.getValue()) #should be 4.0
print(val.getDeriv()['x']) #should be 4.0
#division example
x,y=ad.Scalar('x', 3),ad.Scalar('y', 2);
val = x/y;
print(val.getValue()) # should be 1.5
print(val.getDeriv()['x']); #should be 0.5
print(val.getDeriv()['y']); #should be -0.75
print();
print(x.getValue(),',', y.getValue()); #x, y should retain original values<jupyter_output>1.5
0.5
-0.75

3.0 , 2.0<jupyter_text>We have also implemented the functions **sin**, **cos**, **tan**, **arcsin**, **arccos**, **arctan**, **log**, **ln**, **power**, and **exp**. All of these functions can take in an `int`, `float`, or *Scalar* object. If only an `int`/`float` is provided, then a `float` is returned, but if *Scalar* object is provided, then a new *Scalar* object. No changes are made to the value or derivatives of the original *Scalar* object passed in. Within these functions, the *numpy* functions *sin*, *cos*, *tan*, *arcsin*, *arccos*, *arctan*, *log*, *ln* and *exp* are used to calculate the appropriate values. The trigonometric functions  **sin(x)**, **cos(x)**, and **tan(x)** each take in an `int`, `float`, or *Scalar* object *x* and apply the respective trigonometric function to the *x*. If a *x* is a *Scalar* object, the trigonometric function is applied to the *x._val* attribute and the derivative(s) is updated accordingly.<jupyter_code>import autodiff as ad
import numpy as np 

# Sine functions
x = ad.Scalar('x', 2.0) # float input 
val = ad.sin(x)
print(val.getValue()); #shoud be 0.90929742682
print(val.getDeriv()['x']); #should be -0.41614683654


# Cosine functions
x = ad.Scalar('x', 0) # integer input 
val = ad.cos(x)
print(val.getValue()); #should be 1.0
print(val.getDeriv()['x']); #should be 0.0

# tan function
x = ad.Scalar('x', 2.0); # float input 
val = ad.tan(x);
print(val.getValue()); #shoudl be -2.185039863261519
print(val.getDeriv()['x']); #should be 5.7743992040419174<jupyter_output>0.9092974268256817
-0.4161468365471424
1.0
-0.0
-2.185039863261519
5.774399204041917<jupyter_text>The **power(x1, x2)** function raises *x1* to the power of *x2*. *x1* and *x2* can be any combination of `ints`, `floats`, or *Scalar* objects. If only ints and floats are provided, then **power** will return a `float` with value *x1* raised to the power of *x2*. If at least one *Scalar* object is provided, then **power** works just like using the **__pow__** operator and returns a new *Scalar* object without changing any values in the original *Scalar* object(s).<jupyter_code>#power example
x = 5.0
p = 3.0
print (ad.power(x, p)) # should be 125.0<jupyter_output>125.0<jupyter_text>The **sqrt(x)** function raises *x* to the power of 0.5. *x1* can be a an `int`, `float`, or *Scalar* objects. If only an `int` or `float` is provided, then **sqrt** will return a `float` with value *x* raised to the power of 0.5. If a *Scalar* object is provided, then **sqrt** works just like using the **__pow__** operator and returns a new *Scalar* object without changing any values in the original *Scalar* object(s).<jupyter_code># sqrt example

b = ad.Scalar('b', 4)
b_sqrt = ad.sqrt(b)
b = ad.Scalar('b', 4)
b_sqrt = ad.sqrt(b)
print (b_sqrt.getValue()); #should be 2.0
print (b_sqrt.getDeriv()['b']) #should be 0.25<jupyter_output>2.0
0.25<jupyter_text>The **exp(x)** function raises *e* to the power of *x* , where *x* can be an `int`, `float`, or *Scalar* object. If *x* is an `int` or `float`, then a `float` with value equal to *e* raised to the power of *x* is returned. If *x* is a Scalar object, then a new *Scalar* is returned with an updated value and derivative(s).<jupyter_code>#exponential example
x = 1.0
print (ad.exp(x)) # should be 2.718281828459045<jupyter_output>2.718281828459045<jupyter_text>The **log(x, base)** function give the log of *x*, where *x* can be an int, float, or Scalar object. This function takes in an `int`, `float`, or *Scalar* object for *x* and an `int` or `float` for *base*. It applies log with base *base* to the to the value of *x* and updates the derivative appropriately. If the argument is an `int` or `float`, then the function returns a `float`. If the argument is a *Scalar* object, the function returns a new *Scalar* object with the updated value and derivative. The **ln(x)** function returns the log with base $e$ of *x* and behaves identically to  **log(x, base)**.<jupyter_code># log functions 

x = ad.log(ad.Scalar('x', 10), 10)
print (x.getValue()) # shoud be 1.0
print (x.getDeriv()['x']) # should be 0.043429448190325175

x = ad.ln(ad.Scalar('x', np.e))
print (x.getValue()) # should be 1.0
print (x.getDeriv()['x']) # should be 0.36787944117144233<jupyter_output>1.0
0.043429448190325175
1.0
0.36787944117144233<jupyter_text>The **logistic(x)** function takes in an `int`, `float`, or *Scalar* object and applies the logistic function to the value. If the argument is an `int` or `float`, then the function returns a `float`. If the argument is a *Scalar* object, the function returns a new *Scalar* object with the updated value and derivative.<jupyter_code># logistic example

x = ad.Scalar('x', 8)
y = ad.logistic(x)
print (y.getValue())
print(y.getDeriv()['x'])

x = ad.Scalar('x', 0)
y = ad.logistic(x)
print(y.getValue())
print(y.getDeriv()['x'])<jupyter_output>0.9996646498695336
0.0003352376707564743
0.5
0.25<jupyter_text>We have also implemented the functions **sinh**, **cosh**, and **tanh**. All functions work with `np.sinh`, `np.cosh`, `np.tanh` to calculate the appropriate values. The hyperbolic functions **sinh(x)**, **cosh(x)**, and **tanh(x)** each take in an in `int`, `float`, or *Scalar* object *x* and apply the resprective hyperbolic functions to the *x*. If a *x* is a *Scalar* object, the hyperbolic function is applied to the *x._val* attribute and the derivative(s) is updated accordingly.<jupyter_code># Sinh functions
x = ad.Scalar('x', 2.0) # float input 
val = ad.sinh(x)
print(val.getValue());  # shoud be 3.6268604078470186
print(val.getDeriv()['x']); # should be 3.7621956910836314

# Cosh functions
x = ad.Scalar('x', 0) # integer input 
val = ad.cosh(x)
print(val.getValue()); # should be 1.0
print(val.getDeriv()['x']); # should be 0.0

# tanh function
x = ad.Scalar('x', 2.0); # float input 
val = ad.tanh(x);
print(val.getValue()); # shoudl be 0.9640275800758169
print(val.getDeriv()['x']); # should be 0.07065082485316443<jupyter_output>3.6268604078470186
3.7621956910836314
1.0
0.0
0.9640275800758169
0.07065082485316443<jupyter_text>The __eq__ and __ne__ dunder methods have been implemented to compare *Scalar* objects.__eq__ checks if two *Scalar* objects have equivalent values for both the value and the derivative. __ne__ checks if two *Scalar* objects are not equal.<jupyter_code># __eq__, __ne__ example

x = ad.Scalar('x', 1)
x2 = ad.Scalar('x', 1)
print (x == x2) # should be true 
print (not (x != x2)) # should be true 

x = ad.Scalar('x', 1) 
x2 = ad.Scalar('x', 2.0) 
print (x != x2) # should be true 
print (not (x == x2)) # should be true<jupyter_output>True
True
True
True<jupyter_text>*Vectors*There is not an individual class for vectors/arrays. Users can create a vector using the `create_vector` function, which takes in a name for the vector and a list/array of starting values for each *Scalar* object in the vector. This function creates a `numpy` array of *Scalar* objects that is the same length as the list/array of starting values that is passed in. Each *Scalar* object in the vector is given a name equivalent to the name of the vector concatenated with the index of the object in the vector plus one (e.g. if a list `[1,2,3]` is passed in for the starting values and the vector is name `'v'`, then the *Scalar* objects are name `'v1'`, `'v2'`, and `'v3'`).  A user can also specify the starting derivative value for each *Scalar* object in the vector/array through the `seed_vector` argument.<jupyter_code>v = ad.create_vector('v', values=[1, 2], seed_vector=[3.1, 4.2]);
print(v[0].getValue()) # should print 1.0
print(v[0].getDeriv()['v1']); #should print 3.1

print(v[1].getValue()) # should print 2.0
print(v[1].getDeriv()['v2']); #should print 4.2<jupyter_output>1.0
3.1
2.0
4.2<jupyter_text>A user can use the function `get_value(vector)` to get the values of all of the *Scalar* objects in the vector. The function returns a `numpy` array of the values.<jupyter_code>ad.get_value(v)<jupyter_output><empty_output><jupyter_text>To obtain the derivatives (or partial derivatives) for each *Scalar* object in the vector, the user can use the `get_deriv(vector)` function. This will return a `numpy` array of dictionaries containing the derivatives (or partial derivatives) for each *Scalar* object.<jupyter_code>ad.get_deriv(v)<jupyter_output><empty_output><jupyter_text>Since a vector is just a `numpy` array of *Scalar* objects, all of the dunder methods implemented for the *Scalar* class (__add__, __sub__, __mul__,  __truediv__, __pow__, __iadd__, __isub__, __imul__, __idiv__, __ipow__, and the right equivalents) will be used when the same operations are done on the vector. Similar to `numpy` methods, the operations are conducted element-wise (i.e. in an addition operation between a vector of *Scalar* objects and a float, the float is added to each *Scalar* object). Each operation generates a new vector of *Scalar* objects with updated values and derivatives, which importantly leaves the vector to which the operation was applied unchanged.<jupyter_code>#demonstration for addition with vectors of Scalar objects
w = (v + [1,4]) * 2;

print('Values for v: ', ad.get_value(v));
print('Derivatives for v: ', ad.get_deriv(v));

print('Values for w: ', ad.get_value(w));
print('Derivatives for w: ', ad.get_deriv(w));<jupyter_output>Values for v:  [1. 2.]
Derivatives for v:  [{'v1': 3.1} {'v2': 4.2}]
Values for w:  [ 4. 12.]
Derivatives for w:  [{'v1': 6.2} {'v2': 8.4}]<jupyter_text>The mathematical functions **sin**, **arccos**, **sqrt**, **power**, **ln**, and others can also be applied to a vector of *Scalar* objects. Like the dunder methods for *Scalar* objects, these functions will be applied element-wise and generate a new vector of updated *Scalar* objects.<jupyter_code>z = ad.sin(v);

print('Values for v: ', ad.get_value(v));
print('Derivatives for v: ', ad.get_deriv(v));

print('Values for z: ', ad.get_value(z));
print('Derivatives for z: ', ad.get_deriv(z));<jupyter_output>Values for v:  [1. 2.]
Derivatives for v:  [{'v1': 3.1} {'v2': 4.2}]
Values for z:  [0.84147098 0.90929743]
Derivatives for z:  [{'v1': 1.6749371481912334} {'v2': -1.7478167134979983}]<jupyter_text>A user may also easily obtain the Jacobian of the vector using the `get_jacobian(vector, variables)` method. `variables` is a list of the variable names in the order in which the partial derivatives should be organized in the resulting Jacobian matrix.<jupyter_code>x = ad.create_vector('x', [1, 2, 3])
y = ad.create_vector('y', [5, 8, -7])
z = x - y

jacobian = ad.get_jacobian(z, ['x1', 'y2', 't'])
print(jacobian); #notice that derivative with respect to 't' is all zeroes<jupyter_output>[[ 1.  0.  0.]
 [ 0. -1.  0.]
 [ 0.  0.  0.]]
{'fraction_non_alphanumeric': 0.1295404066290791, 'fraction_numerical': 0.03130018793780967, 'mean_word_length': 4.168844931119746, 'pattern_counts': {'":': 0, '<': 67, '<?xml version=': 0, '>': 67, 'https://': 3, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '106121', 'license': 'MIT', 'n_tokens_mistral': 9458, 'n_tokens_neox': 8841, 'n_words': 3816, 'path': 'docs/documentation.ipynb', 'repo_name': 'cs207FinalProjectGroup/cs207-FinalProject', 'size': '29265'}