<jupyter_start><jupyter_text>Neural Networks From Scratch - 01 - A Tiny Neuron is Born in a Cookie NightWe are going to implement neural networks from scratch, that is no tensorflow, pytorch, scikit-learn etc.Most of the material is taken from my another [repository](https://gitlab.com/QmAuber/basit-ysa-python/-/blob/master/ysa/defterler/YapayZekaMatematik.ipynb)Here is the list of packages we will be using:<jupyter_code>import numpy as np ## for matrix operations, you can excange it with something else
# or implement your own array operations, we don't use anything fancy here
from typing import List, Tuple, Callable<jupyter_output><empty_output><jupyter_text>DerivativesThe first thing you need to understand when implementing Neural Networks is by and large, derivatives. So what are derivatives ? Grossly reduced, they are operators that help us to understand how functions behave. Notice that derivative**s**, so there are more than one type.The simplest derivative can be expressed mathematically as:$$f'(x) = \lim_{h\to 0} \frac{f(x+h) - f(x)}{h}$$Here we assume that: $f: \mathbb{R} \to \mathbb{R}$. You can read this as the following:*Function f takes in a real number and outputs a real number*In code this would make:<jupyter_code>def derivative(f: Callable[[float], float], x: float, h=1e-8) -> float:
    "Takes the derivative of f"
    f_t = (f(x + h) - f(x)) / h
    return f_t<jupyter_output><empty_output><jupyter_text>Simply put I add a tiny bit of something to the argument of f and apply the f to both the original and the added argument, then observe the difference relative to added value. The more I do this, I understand better and better how the function behaves.Well this is all and good but what if ... just what if the function f takes in multiple values, like a list of values ?So mathematically what if the definition of $f$ is something like the following:$$f: \mathbb{R}^N \to R$$If this is the case, since I only have a single $h$ to add, this raises a question. Which value should receive the $h$ while taking the derivative ? The answer is whichever you want. But then we would not be able to observe *completely* how the function behaves: YES, that is why this way of taking the derivative is called **partial** derivative, and it is specified as the following: $$f'(x_1, ..., x_n) = \lim_{h\to0} \frac{f(x_1, ..., x_i+h, ..., x_n) - f(x_1, ..., x_n)}{h}$$In code this would make:<jupyter_code>def partial_derivative(f: Callable[[List[float]], float], 
                arguments: list, 
                argument_position: int, 
                h=1e-8) -> np.ndarray:
    """
    Basic implementation of partial derivative
    """
    arguments_copy = arguments.copy() # in order to leave original arguments untouched
    arguments_copy[argument_position] = arguments_copy[argument_position] + h
    return (f(arguments_copy) - f(arguments)) / h<jupyter_output><empty_output><jupyter_text>Well what if I want to observe, the **complete** behaviour of the function $f$.Then I basically apply the partial derivative to all of the arguments. This is called taking the **gradient** of the function.In code:<jupyter_code>def gradient(f: Callable[[List[float]], float], 
                arguments: list, 
                h=1e-8) -> List[float]:
    """
    Basic implementation of gradient computation
    """
    partials: List[float] = []
    for argument_pos, argument in enumerate(arguments):
        partial = partial_derivative(f=f, arguments=arguments, argument_position=argument_pos)
        partials.append(partial)
    return partials<jupyter_output><empty_output><jupyter_text>Let's test our functions:<jupyter_code># simple derivative test
def testfn(x): return x * x
d = derivative(testfn, x=5)
print("x^2 derivative = 2x => 5^2 derivative 2 * 5", 
      10 == round(d, 3))
# testing partial derivative
def test_part_fn(xs: list): return (xs[0] ** 2) + (xs[0] * xs[1]) + (xs[1] ** 2)
# x^2 + xy + y^2
# f_x' = 2x + y , f_y' = 2y + x
xd = partial_derivative(f=test_part_fn, arguments=[2, 6], argument_position=0)
yd = partial_derivative(f=test_part_fn, arguments=[2, 6], argument_position=1)
print("f(x,y) = x^2 + xy + y^2, x=2, y=6")
print("f_x'(x,y) = 2x + y => 4 + 6 = 10")
print("f_y'(x,y) = 2y + x => 2 + 12 = 14")
print("f_x'(x,y) = 10", round(xd, 3) == 10)
print("f_y'(x,y) = 14", round(yd, 3) == 14)
# test gradient
partials = gradient(f=test_part_fn, arguments=[2, 6])
print("∇f(x,y), gradient of f, that is [f_x', f_y'] => [10, 14]")
print("∇f(x,y)= [10, 14], ", [round(p, 3) for p in partials] == [10, 14])<jupyter_output>∇f(x,y), gradient of f, that is [f_x', f_y'] => [10, 14]
∇f(x,y)= [10, 14],  True<jupyter_text>What if the signature of the function $f$ is the following:$$f: \mathbb{R} \to \mathbb{R}^n$$This is called a vector function, simply a function that outputs a vector. It can also be interpreted as $f(x) = [f_1(x), f_2(x), \dots, f_n(x)]$, that is we can interpret it as a set of functions instead of a single function.When taking the derivative of this function, we have two cases:- The member functions, let's call them, sub functions of $f$ are known- The member functions of $f$ are not knownWhen we know the member function, the derivative is simply:$$f'(x) = [f'_1(x), f'_2(x), \dots, f'_n(x)]$$When we don't know the member functions, the derivative is:$$f'(x) = \lim_{h\to 0} \frac{f(x+h) - f(x)}{h}$$The only difference here is that the division is a division of a vector by a scalar.So in code:<jupyter_code>def derivative_of_vector_function_with_known_functions(
    fs: List[Callable[[float], float]], 
    x: float,
    h=1e-8
)-> List[float]:
    "Compute derivative of a vector function when member functions are known"
    derivatives = []
    for f in fs:
        d = derivative(f, x)
        derivatives.append(d)
    return derivatives

def derivative_of_vector_function(
    f: Callable[[float], List[float]], 
    x: float, 
    h=1e-8) -> List[float]:
    "Derivative of a vector function where member functions are not known"
    return list((np.array(f(x + h)) - np.array(f(x))) / h)<jupyter_output><empty_output><jupyter_text>Let's test all that<jupyter_code># testler
def testfn(x): return [x*x, 2 * x, x - 1]
# turevi bunun 2x, 2, 1
vk = derivative_of_vector_function_with_known_functions(fs=[lambda x: x*x, 
                                     lambda x: 2 * x, 
                                     lambda x: x -1],
                                 x=3)
print("with known functions", 
      [round(v, 3) for v in vk] == [6, 2, 1])
vnk = derivative_of_vector_function(f=testfn, x=3)
print("with unknown functions", 
      [round(v, 3) for v in vnk] == [6, 2, 1])<jupyter_output>with known functions True
with unknown functions True<jupyter_text>So far we have covered the following function signatures:- $f: \mathbb{R} \to \mathbb{R}$ - goes with simple derivative- $f: \mathbb{R}^n \to \mathbb{R}$ - goes with partial derivative and the gradient- $f: \mathbb{R} \to \mathbb{R}^n$ - goes with vector derivativeNow what if the function signature is the last conceivable option, that is:- $f: \mathbb{R}^n \to \mathbb{R}^m$How would we compute the derivative of such a function ?The logic is a mix of what we do with vector functions and gradient.Again we can interpret the $f$ as a set of functions whose number of elements equals to $m$ where each function takes $n$ arguments. Basically as the following matrix:$f(x) = \pmatrix{f_1(x_1, ..., x_n)  \\f_2(x_1, ..., x_n) \\... \\f_m(x_1, ..., x_n)}$Then the derivative of such a function would simply be:$$f'(x)= J(f) = \pmatrix{f'_1(x_1, ..., x_n)  \\f'_2(x_1, ..., x_n) \\... \\f'_m(x_1, ..., x_n)}$$If we make it even more explicit:$$J(f) = \pmatrix{f_{x_1, 1}'(x) = \frac{f(x_1+h, ..., x_n) - f(x)}{h} & ... & f_{x_n, 1}'(x) = \frac{f(x_1, ..., x_n+h) - f(x)}{h} \\f_{x_1, 2}'(x) = \frac{f(x_1+h, ..., x_n) - f(x)}{h} & ... & f_{x_n, 2}'(x) = \frac{f(x_1, ..., x_n+h) - f(x)}{h} \\\vdots & ... & \vdots \\f_{x_1, m}'(x) = \frac{f(x_1+h, ..., x_n) - f(x)}{h} & ... & f_{x_n, m}'(x) = \frac{f(x_1, ..., x_n+h) - f(x)}{h}}$$Here is an example: $ f(x,y,z)=[(2x+y),3z/2,y2+z,6x]$. The derivative matrix of $f$ would be:$$J(f) = \pmatrix{2 & 1 & 0 \\0 & 0 & \frac{3}{2} \\0 & 2y & 1 \\6 & 0 & 0}$$This derivative matrix is called **Jacoby matrix**. Computing this matrix is informally called taking/evaluating the jacobian of a function.The previous question arises again: What if we don't know the member functions ?Then taking the jacobian means simply:$$f'(x) = \lim_{h\to 0} \frac{f(x+h) - f(x)}{h}$$The only difference is that we divide a matrix by a scalar at the end.In code this would be:<jupyter_code>def jacoby_with_known_functions(fs: List[Callable[[List[float]], float]], 
           args: List[float], h=1e-8) -> List[List[float]]:
    j_mat = []
    for f in fs:
        d = gradient(f=f, arguments=args, h=h)
        j_mat.append(d)
    return j_mat

def jacoby_pure_python(f: Callable[[List[float]], List[float]], 
           args: List[float], 
           h=1e-8) -> List[List[float]]:
    ""
    nargs = [a for a in args] 
    j_mat = []
    for i, a in enumerate(args):
        narg = a + h
        nargs[i] = narg
        d = list((np.array(f(nargs)) - np.array(f(args))) / h)
        j_mat.append(d)
        nargs[i] = a
    return j_mat

def jacobian(f: Callable[[np.ndarray], np.ndarray], 
                args: np.ndarray, h=1e-8) -> np.ndarray:
    """
    Compute jacobian with numpy
    """
    result = f(args)
    j_mat = np.zeros((*args.shape, *result.shape), dtype=np.float)
    for index in np.ndindex(args.shape):
        narg = args.copy()
        narg[index] += h
        j_mat[index] = (f(narg) - f(args)) / h
    return j_mat.reshape((result.size, args.size))
def f1(arg: List[float]): return 2*arg[0] + arg[1]
def f2(arg: List[float]): return (3 * arg[2]) / 2
def f3(arg: List[float]): return arg[1]*arg[1] + arg[2]
def f4(arg: List[float]): return 6*arg[0]
        
def testfn(arg: List[float]): 
    return [
    f1(arg), 
    f2(arg),
    f3(arg),
    f4(arg)
]
j_known = jacoby_with_known_functions(
    fs=[f1,f2,f3,f4], args=[2,1, 3])
print("jacobian with known functions: ",
      np.around(np.array(j_known), 3))
j_unknown = jacoby_pure_python(f=testfn, args=[2,1,3])
print("jacobian with unknown function: ", np.around(np.array(j_unknown), 3))
def testfn1(arg):
    return np.array([arg[0].sum() * 2, 
                     arg[1].sum() * 3, 
                     arg[2].sum() * 4], 
                    dtype=np.float)

def testfn2(arg):
    return np.array([arg[0] * 2, 
                     arg[1] * 3, 
                     arg[2] * 4], 
                    dtype=np.float)


j_np_n1 = jacobian(f=testfn1, 
                      args=np.array([[1, 2, 3, 4], 
                                           [5, 6, 7, 8],
                                           [9, 10, 11, 12]
                                          ], 
                                          dtype=np.float)
                     )

j_np_n2 = jacobian(f=testfn2, 
                     args=np.array(
                         [[1, 2, 3, 4],
                          [5, 6, 7, 8],
                          [9, 10, 11, 12]
                         ], 
                         dtype=np.float)
                    )


print("jacobian with numpy: ") 
print(np.around(j_np_n1, 3))

print("jacobian with numpy: ")
print(np.around(j_np_n2, 3))<jupyter_output>jacobian with numpy: 
[[2. 0. 0. 2. 0. 0. 2. 0. 0. 2. 0. 0.]
 [0. 3. 0. 0. 3. 0. 0. 3. 0. 0. 3. 0.]
 [0. 0. 4. 0. 0. 4. 0. 0. 4. 0. 0. 4.]]
jacobian with numpy: 
[[2. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [0. 2. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 2. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 2. 0. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 3. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 3. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 3. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 3. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 4. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 4. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 4. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 4.]]<jupyter_text>We have one last option to consider and that is composite functions.Armed with the knowledge of jacobian, we can apply the derivative to most of the functions in everyday use. However what if our function is composed of a sequential application of a series of functions, so what if it is something like: $f(x) = k(p(x))$Notice that this is not truly the same case with the vector functions or the jacobian. In jacobian, or vector functions, the order of evaluation, or let's say the order in which we apply the functions or the derivative does not change the result. Here we have a clear sense of time, a sequential relation. The function $p$ has to be evaluated first.In any case how do we compute its derivative ?Let's start with the simplest case again:Suppose that $f,k,p: \mathbb{R} \to \mathbb{R}$The derivative of $f$ is defined by the following:$$f'(x) = k'( p(x) ) * p'(x)$$This equivalence is also known as the **chain rule**.Let's see that in code:<jupyter_code>def chain1d(f: Callable[[float], float], g: Callable[[float], float], x: float):
    ""
    delta_g = derivative(g, x)
    g_value = g(x)
    delta_f = derivative(f, g_value)
    return delta_f * delta_g
# chain test
def f(x): return 2 * x
def g(x): return x * x
# derivative 2 * 2x = 4x =1
def testfn(x): return f(g(x))
print("We have equal derivatives, right ? ", 
      round(derivative(testfn, x=2), 3) == round(chain1d(f=f, g=g, x=2), 3))<jupyter_output>We have equal derivatives, right ?  True<jupyter_text>Again what if our function signature is $f: \mathbb{R}^n \to \mathbb{R}^m$where the interior signatures are something like: - $p:\mathbb{R}^n \to \mathbb{R}^z$- $k:\mathbb{R}^z \to \mathbb{R}^m$Well by the hand of God, the chain rule is strictly the same, we just use the Jacobian:$f'(x) = J_k(g(x)) * J_g(x)$In code this would be:<jupyter_code>def chain(f: Callable[[np.ndarray], np.ndarray], 
                    g: Callable[[np.ndarray], np.ndarray],
                    x: np.ndarray
                   ) -> np.ndarray:
        ""
        g_val = g(x)
        g_j = jacobian(f=g, args=x)
        f_j = jacobian(f=f, args=g_val)
        return np.matmul(f_j, g_j)
# test jacoby
# test fn
def f(x: np.ndarray): return np.array([x[0]*3 + 2, x[1], x[2]*2, 
                                       x[0] + x[1]], dtype=np.float)

def g(x: np.ndarray): return np.array([x[0] / 2, x[1]*2, x[2]/2], dtype=np.float)

args = np.array([3, 1, 3], dtype=np.float)
jmat = chain(f, g, args)
print(np.around(jmat, 3))<jupyter_output>[[1.5 0.  0. ]
 [0.5 0.  0.5]
 [0.  2.  0. ]
 [0.  4.  0. ]]<jupyter_text>Computation GraphsHere is the last bit of knowledge required for understanding neural networks. It turns out, composing functions are quite cool, and that we can express almost anything in terms of composite functions. Wouldn't it be better if we can represent function composition in an intuitive way. Well computation graphs are mostly that, meaning a nice a way of expressing sequence of computations. They also allow us to know the general form of the computation before evaluation which gives you a margin for optimizing and/or serializing the computation.From the perspective of graph theory, computation graphs are almost always acyclic and directed. So basically if we have $f(x) = g(k(p(x)))$We represent it as:```   p         k          gx ---> x_p -----> x_k ----> x_g```If we take the derivative of the function $f$, the resulting computational graph would be something like:```   p         k             gx ---> x_p -------> x_k ------------> x_g                                       /       p'(x)          k'(x)         g'(x)x'_p <-------- x'_k <------- x'_g <------ ```So each node of this graph is a computation. In code that would make:<jupyter_code>class AbstractComputation:
    def __init__(self):
        self.args = None
        
    def result(self, arg: np.ndarray):
        raise NotImplemented
        
    def delta(self):
        raise NotImplemented
        
    @property
    def output(self):
        raise NotImplemented
        
    @property
    def d(self):
        raise NotImplemented
        
class Computation(AbstractComputation):
    def __init__(self, inpt=None, function=lambda x: x):
        super().__init__()
        self.args = inpt
        self.f = function
        
    def __str__(self):
        return ("Computation Node:\nInput: " 
                + str(self.args.shape) 
                + "\nOutput:" + str(self.output)
                + "\nFunction" + str(self.f)
               )
    @property
    def output(self):
        return self.compute()
    
    @property
    def d(self):
        return jacobian(f=self.f, args=self.args)
        
    def result(self, arg):
        return self.f(arg)
    
    def compute(self):
        res = self.result(arg=self.args)
        return res.copy()
# test computation node
n = Computation()
n.args = np.array([1, 3, 2], dtype=np.float)
n.f = lambda x: x ** 2
print("node output: ", n.output)  # 1 9 4
print("node derivative", np.around(n.d, 3)) # 2 6 4<jupyter_output>node output:  [1. 9. 4.]
node derivative [[2. 0. 0.]
 [0. 6. 0.]
 [0. 0. 4.]]<jupyter_text>A layer is an instance of a computational graph. In the simplest case, it is a sequence of computations. A network can be composed of more than one layers.However let's implement the layer as a computation node as well, since we ultimately we require the same set of operations from them anyway.<jupyter_code>class Layer(AbstractComputation):
    def __init__(self, flst: List[Callable[[np.ndarray], np.ndarray]], 
                 arg: np.ndarray):
        "<NAME> (layer)"
        self.args = arg
        self.computations = [Computation(function=f) for f in flst]
        self.is_computed = False
        
    def compute(self):
        previous = self.args.copy()
        current = self.computations[0]
        current.args = previous
        for n in self.computations[1:]:
            n.args = current.output
            current = n
        self.is_computed = True
        
    def compute_until(self, limit: int):
        """
        """
        if limit > len(self.computations):
            raise ValueError(
                "Limit out of bounds for available number of computations " + str(limit)
                + " limit " + str(len(self.computations))
            )
        ns = self.computations[:limit]
        previous_args = self.args.copy()
        current = ns[0]
        current.args = previous_args
        for n in ns[1:]:
            n.args = current.output
            current = n
        return ns
    
    def compute_until_with_arg(self, limit: int, args: np.ndarray):
        """
        """
        if limit > len(self.computations):
            raise ValueError(
                "Limit out of bounds for available number of computations " + str(limit)
                + " limit " + str(len(self.computations))
            )
        ns = self.computations[:limit]
        previous_args = args.copy()
        current = ns[0]
        current.args = previous_args
        for n in ns[1:]:
            n.args = current.output
            current = n
        return ns
    
    def delta_from(self, start: int):
        """
        """
        self.compute()
        if start >= len(self.computations):
            raise ValueError("Start out of bounds " + str(start) +" number of "+
                            " computations: "+ str(len(self.computations)))
        current = self.computations[start]
        current_d = current.d
        for n in self.computations[start+1:]:
            d = n.d
            current_d = np.matmul(d,current_d)
        return current_d
            
    @property
    def output(self):
        self.compute()
        return self.computations[-1].output
    
    @property
    def delta_backwards(self):
        "backward accumulation of derivative"
        self.compute()
        reversed_cs = list(reversed(self.computations))
        last = reversed_cs[0]
        ld = last.d
        for n in reversed_cs[1:]:
            d = n.d
            ld = np.matmul(ld, d)
        return ld
    
    @property
    def delta(self):
        "forward accumulation of derivative"
        return self.delta_from(start=0)
# test layer
def mx(x: np.ndarray): return x * x 
def gx(x: np.ndarray): return x + 5
layer = Layer(flst=[mx, gx], arg=np.array([1,0,3], dtype=np.float))
"""
f'(x): [
 [2. 0. 0.]
 [0. 0. 0.]
 [0. 0. 6.]
 ]
g'(x):  [
 [1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]
 ]
"""

print(layer.output)
print(np.around(layer.delta_backwards, 3))
print(np.around(layer.delta, 3))<jupyter_output>[ 6.  5. 14.]
[[2. 0. 0.]
 [0. 0. 0.]
 [0. 0. 6.]]
[[2. 0. 0.]
 [0. 0. 0.]
 [0. 0. 6.]]<jupyter_text>Let's implement a linear regression layer, just to get a feel for how to do familiar computations with computation graphs.<jupyter_code>class LinearRegressionLayer(Layer):
    def __init__(self, arg: np.ndarray):
        ""
        self.mx = np.random.randn(*arg.shape)
        self.b = np.zeros_like(self.mx)
        def m_x(x: np.ndarray): return x * self.mx
        def b_x(x: np.ndarray): return x + self.b
        def mx_b(x: np.ndarray): return b_x(m_x(x))
        super().__init__(flst=[m_x, b_x], arg=arg.copy())
        
    @property
    def predict(self):
        """
        y = mx + b = f(x)
        y = f(x)
        mx = M(x)
        +b = B(x)
        f(x) = B(M(x)) = mx + b
        """
        return self.output
arg = np.array([[6, 5], [2, 1], [9, 3]], dtype=np.float)
LRL = LinearRegressionLayer(arg=arg)
# ideal values
lrl_slope = np.array([[0.5, 0.5], [0.5, 0.5], [3, 3]], dtype=np.float)
lrl_intercept = np.array([[1,1], [1,1], [0,0]], dtype=np.float)
lrl_target = np.array([[4, 3.5], [2, 1.5], [27, 9]])
print("prediction: \n", LRL.output)
print("expected output: \n", lrl_target)
print("derivative matrix: \n", np.around(LRL.mx, 3))
print("expected derivative matrix: \n", lrl_slope)
print("layer derivative/slope: \n", np.around(LRL.delta, 3))<jupyter_output>prediction: 
 [[ 2.68317169 -5.32191027]
 [ 2.6374344   0.28760089]
 [-9.23522885  2.71788954]]
expected output: 
 [[ 4.   3.5]
 [ 2.   1.5]
 [27.   9. ]]
derivative matrix: 
 [[ 0.447 -1.064]
 [ 1.319  0.288]
 [-1.026  0.906]]
expected derivative matrix: 
 [[0.5 0.5]
 [0.5 0.5]
 [3.  3. ]]
layer derivative/slope: 
 [[ 0.447  0.     0.     0.     0.     0.   ]
 [ 0.    -1.064  0.     0.     0.     0.   ]
 [ 0.     0.     1.319  0.     0.     0.   ]
 [ 0.     0.     0.     0.288  0.     0.   ]
 [ 0.     0.     0.     0.    -1.026  0.   ]
 [ 0.     0.     0.     0.     0.     0.906]]
{'fraction_non_alphanumeric': 0.1492062088179366, 'fraction_numerical': 0.032326537832220406, 'mean_word_length': 2.46840490797546, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 69, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 17, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '39734', 'license': 'MIT', 'n_tokens_mistral': 8426, 'n_tokens_neox': 7791, 'n_words': 2525, 'path': 'nn101/01 - A Tiny Neuron is Born in a Cookie Night.ipynb', 'repo_name': 'D-K-E/nn-101', 'size': '22626'}