<commit_before># Autograd

Autograd can automatically differentiate native Python and Numpy code. It can handle a large subset of Python's features, including loops, ifs, recursion and closures, and it can even take derivatives of derivatives of derivatives. It uses reverse-mode differentiation (a.k.a. backpropagation), which means it can efficiently take gradients of scalar-valued functions with respect to array-valued arguments. The main intended application is gradient-based optimization.

# Example use

```python
>>> import autograd.numpy as np             # Thinly-wrapped numpy
>>> from autograd import grad               # The only autograd function you may ever need
>>> def tanh(x):                            # Define a function
...     return (1.0 - np.exp(-x))  / ( 1.0 + np.exp(-x))
... 
>>> grad_tanh = grad(tanh)                  # Obtain its gradient function
>>> grad_tanh(1.0)                          # Evaluate the gradient at x = 1.0
0.39322386648296376
>>> (tanh(1.0001) - tanh(0.9999)) / 0.0002  # Compare to finite differences
0.39322386636453377
>>>
>>> # We can differentiate as many times as we like
... grad_tanh_2 = grad(grad_tanh)           # 2nd derivative
>>> grad_tanh_3 = grad(grad_tanh_2)         # 3rd derivative
>>> grad_tanh_4 = grad(grad_tanh_3)         # etc.
>>> grad_tanh_5 = grad(grad_tanh_4)
>>> grad_tanh_6 = grad(grad_tanh_5)
>>>
>>> import matplotlib.pyplot as plt         # Plot the results, just for fun
>>> x = np.linspace(-7, 7, 200)
>>> plt.plot(x, map(tanh, x),
...          x, map(grad_tanh, x),
...          x, map(grad_tanh_2, x),
...          x, map(grad_tanh_3, x),
...          x, map(grad_tanh_4, x),
...          x, map(grad_tanh_5, x),
...          x, map(grad_tanh_6, x))
>>> plt.show()
```

<img src="examples/tanh.png" width="600">

## Docs
<commit_msg>Broke up example into chunks<commit_after># Autograd

Autograd can automatically differentiate native Python and Numpy code. It can handle a large subset of Python's features, including loops, ifs, recursion and closures, and it can even take derivatives of derivatives of derivatives. It uses reverse-mode differentiation (a.k.a. backpropagation), which means it can efficiently take gradients of scalar-valued functions with respect to array-valued arguments. The main intended application is gradient-based optimization.

Example use:

```python
>>> import autograd.numpy as np  # Thinly-wrapped numpy
>>> from autograd import grad    # The only autograd function you may ever need
>>>
>>> def tanh(x):                 # Define a function
...     return (1.0 - np.exp(-x))  / ( 1.0 + np.exp(-x))
... 
>>> grad_tanh = grad(tanh)       # Obtain its gradient function
>>> grad_tanh(1.0)               # Evaluate the gradient at x = 1.0
0.39322386648296376
>>> (tanh(1.0001) - tanh(0.9999)) / 0.0002  # Compare to finite differences
0.39322386636453377

We can continue to differentiate as many times as we like:

```python
>>> grad_tanh_2 = grad(grad_tanh)           # 2nd derivative
>>> grad_tanh_3 = grad(grad_tanh_2)         # 3rd derivative
>>> grad_tanh_4 = grad(grad_tanh_3)         # etc.
>>> grad_tanh_5 = grad(grad_tanh_4)
>>> grad_tanh_6 = grad(grad_tanh_5)
>>>
>>> import matplotlib.pyplot as plt
>>> x = np.linspace(-7, 7, 200)
>>> plt.plot(x, map(tanh, x),
...          x, map(grad_tanh, x),
...          x, map(grad_tanh_2, x),
...          x, map(grad_tanh_3, x),
...          x, map(grad_tanh_4, x),
...          x, map(grad_tanh_5, x),
...          x, map(grad_tanh_6, x))
>>> plt.show()
```

<img src="examples/tanh.png" width="600">

## Docs

{'fraction_non_alphanumeric': 0.13277545327754534, 'fraction_numerical': 0.04630404463040446, 'mean_word_length': 2.880952380952381, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 107, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4451736', 'n_tokens_mistral': 1283, 'n_tokens_neox': 1178, 'n_words': 363}