<jupyter_start><jupyter_text>Theano exercisesThis notebook contains Theano exercises not related to machine learning.The exercises work in the following way:- You have a cell with TODOs that raise errors with a description of what is needed.- The cell contains a description at the top.- Run the cell (Ctrl-Enter) to execute it. At first, it raises an error.- Modify the cell to implement what is asked in the error.- If you implement correctly all the todos and run the cell, it should print "Success" at the end (there is validation code in the cell). If not, try again.- If you want to check the solution, execute the cell that starts with "`%load`" after the exercise.  Exercise 1This exercise requires you to create Theano variables and perform some computation on them.<jupyter_code>import numpy as np
from theano import function
raise NotImplementedError("TODO: add any other imports you need")


def make_scalar():
    """
    Returns a new Theano scalar.
    """

    raise NotImplementedError("TODO: implement this function.")


def log(x):
    """
    Returns the logarithm of a Theano scalar x.
    """

    raise NotImplementedError("TODO: implement this function.")


def add(x, y):
    """
    Adds two theano scalars together and returns the result.
    """

    raise NotImplementedError("TODO: implement this function.")
    
# The following code will use your code and test it.
a = make_scalar()
b = make_scalar()
c = log(b)
d = add(a, c)
f = function([a, b], d)
a = np.cast[a.dtype](1.)
b = np.cast[b.dtype](2.)
actual = f(a, b)
expected = 1. + np.log(2.)
assert np.allclose(actual, expected)
print("SUCCESS!")<jupyter_output><empty_output><jupyter_text>Solution<jupyter_code>%load solutions/01_scalar_soln.py<jupyter_output><empty_output><jupyter_text>Exercise 2This exercise requires you to create Theano variable, and apply elementwise multiplication and matrix/vector dot product.<jupyter_code>import numpy as np
from theano import function
raise NotImplementedError("TODO: add any other imports you need")


def make_vector():
    """
    Returns a new Theano vector.
    """

    raise NotImplementedError("TODO: implement this function.")


def make_matrix():
    """
    Returns a new Theano matrix.
    """

    raise NotImplementedError("TODO: implement this function.")

def elemwise_mul(a, b):
    """
    a: A theano matrix
    b: A theano matrix
    Returns the elementwise product of a and b
    """

    raise NotImplementedError("TODO: implement this function.")


def matrix_vector_mul(a, b):
    """
    a: A theano matrix
    b: A theano vector
    Returns the matrix-vector product of a and b
    """

    raise NotImplementedError("TODO: implement this function.")

# The following code will use your code and test it.
a = make_vector()
b = make_vector()
c = elemwise_mul(a, b)
d = make_matrix()
e = matrix_vector_mul(d, c)

f = function([a, b, d], e)

rng = np.random.RandomState([1, 2, 3])
a_value = rng.randn(5).astype(a.dtype)
b_value = rng.rand(5).astype(b.dtype)
c_value = a_value * b_value
d_value = rng.randn(5, 5).astype(d.dtype)
expected = np.dot(d_value, c_value)

actual = f(a_value, b_value, d_value)
assert np.allclose(actual, expected)
print("SUCCESS!")<jupyter_output><empty_output><jupyter_text>Solution<jupyter_code>%load solutions/02_vector_mat_soln.py<jupyter_output><empty_output><jupyter_text>Exercise 3This exercise requires you to create Theano tensor variables, do broadcastable addition, and to compute the maximum over part of a tensor.<jupyter_code>import numpy as np
from theano import function
raise NotImplementedError("TODO: add any other imports you need")


def make_tensor(dim):
    """
    Returns a new Theano tensor with no broadcastable dimensions.
    dim: the total number of dimensions of the tensor.
    (You can use any dtype you like)
    """

    raise NotImplementedError("TODO: implement this function.")


def broadcasted_add(a, b):
    """
    a: a 3D theano tensor
    b: a 4D theano tensor
    Returns c, a 4D theano tensor, where

    c[i, j, k, l] = a[l, k, i] + b[i, j, k, l]

    for all i, j, k, l
    """

    raise NotImplementedError("TODO: implement this function.")

def partial_max(a):
    """
    a: a 4D theano tensor

    Returns b, a theano matrix, where

    b[i, j] = max_{k,l} a[i, k, l, j]

    for all i, j
    """

    raise NotImplementedError("TODO: implement this function.")

# The following code use your code and test it.
a = make_tensor(3)
b = make_tensor(4)
c = broadcasted_add(a, b)
d = partial_max(c)

f = function([a, b], d)

rng = np.random.RandomState([1, 2, 3])
a_value = rng.randn(2, 2, 2).astype(a.dtype)
b_value = rng.rand(2, 2, 2, 2).astype(b.dtype)
c_value = np.transpose(a_value, (2, 1, 0))[:, None, :, :] + b_value
expected = c_value.max(axis=1).max(axis=1)

actual = f(a_value, b_value)

assert np.allclose(actual, expected), (actual, expected)
print("SUCCESS!")<jupyter_output><empty_output><jupyter_text>Solution<jupyter_code>%load solutions/03_tensor_soln.py<jupyter_output><empty_output><jupyter_text>Exercise 4This exercise requires you to compile a Theano function and call it to execute `"x + y"`.<jupyter_code>from theano import tensor as T
raise NotImplementedError("TODO: add any other imports you need")


def evaluate(x, y, expr, x_value, y_value):
    """
    x: A theano variable
    y: A theano variable
    expr: A theano expression involving x and y
    x_value: A numpy value
    y_value: A numpy value

    Returns the value of expr when x_value is substituted for x
    and y_value is substituted for y
    """

    raise NotImplementedError("TODO: implement this function.")


# The following code use your code and test it.
x = T.iscalar()
y = T.iscalar()
z = x + y
assert evaluate(x, y, z, 1, 2) == 3
print("SUCCESS!")<jupyter_output><empty_output><jupyter_text>Solution<jupyter_code>%load solutions/04_function_soln.py<jupyter_output><empty_output><jupyter_text>Exercise 5This exercise makes you use shared variables. You must create them and update them by swapping 2 shared variables values.<jupyter_code>import numpy as np
raise NotImplementedError("TODO: add any other imports you need")


def make_shared(shape):
    """
    Returns a theano shared variable containing a tensor of the specified
    shape.
    You can use any value you want.
    """
    raise NotImplementedError("TODO: implement the function")


def exchange_shared(a, b):
    """
    a: a theano shared variable
    b: a theano shared variable
    Uses get_value and set_value to swap the values stored in a and b
    """
    raise NotImplementedError("TODO: implement the function")


def make_exchange_func(a, b):
    """
    a: a theano shared variable
    b: a theano shared variable
    Returns f
    where f is a theano function, that, when called, swaps the
    values in a and b
    f should not return anything
    """
    raise NotImplementedError("TODO: implement the function")


# The following code will use your code and test it.
a = make_shared((5, 4, 3))
assert a.get_value().shape == (5, 4, 3)
b = make_shared((5, 4, 3))
assert a.get_value().shape == (5, 4, 3)
a.set_value(np.zeros((5, 4, 3), dtype=a.dtype))
b.set_value(np.ones((5, 4, 3), dtype=b.dtype))
exchange_shared(a, b)
assert np.all(a.get_value() == 1.)
assert np.all(b.get_value() == 0.)
f = make_exchange_func(a, b)
rval = f()
assert isinstance(rval, list)
assert len(rval) == 0
assert np.all(a.get_value() == 0.)
assert np.all(b.get_value() == 1.)

print("SUCCESS!")<jupyter_output><empty_output><jupyter_text>Solution<jupyter_code>%load solutions/05_shared_soln.py<jupyter_output><empty_output><jupyter_text>Exercise 6This exercise makes you use Theano symbolic differentiation features, `grad`.<jupyter_code>from theano import tensor as T


def grad_sum(x, y, z):
    """
    x: A theano variable
    y: A theano variable
    z: A theano expression involving x and y

    Returns dz / dx + dz / dy
    """
    raise NotImplementedError("TODO: implement this function.")


# The following code will use your code and test it.
x = T.scalar()
y = T.scalar()
z = x + y
s = grad_sum(x, y, z)
assert s.eval({x: 0, y: 0}) == 2
print("SUCCESS!")<jupyter_output><empty_output><jupyter_text>Solution<jupyter_code>%load solutions/06_grad_soln.py<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1102523849776597, 'fraction_numerical': 0.009660669001328342, 'mean_word_length': 4.150497512437811, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 49, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '329516', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2817, 'n_tokens_neox': 2723, 'n_words': 945, 'path': 'intro_theano/exercises.ipynb', 'repo_name': 'aelnouby/summerschool2015', 'size': '8281'}