<jupyter_start><jupyter_text>SummaryThere are acceptable and un-acceptable ways of flattening lists of dicts, if you want to use the result in autograd.See below.  https://github.com/HIPS/autograd/blob/master/docs/tutorial.mdtldr-do-use<jupyter_code>import numpy as np
import autograd.numpy as ag_np
import autograd<jupyter_output><empty_output><jupyter_text>Unsafe version for autograd. Uses indexing<jupyter_code>def flatten(bnn_param_list):
    n_dims = ag_np.sum([[arr.size for arr in layer.values()] for layer in bnn_param_list])
    flat_vec = ag_np.zeros(n_dims)
    start = 0
    for layer in bnn_param_list:
        stop = start + layer['w'].size
        flat_vec[start:stop] = layer['w'].flatten()  # <- BAD
        start = stop
        
        stop = start + layer['b'].size
        flat_vec[start:stop] = layer['b'].flatten()  # <- BAD
        start = stop
    return flat_vec
def sum_of_params(bnn_param_list):
    return ag_np.sum(flatten(bnn_param_list))
bnn_param_list = [dict(w=np.random.randn(3), b=np.random.randn(3))]
bnn_param_list
sum_of_params(bnn_param_list)
g = autograd.grad(sum_of_params)<jupyter_output><empty_output><jupyter_text>Try to get gradient (will give error)<jupyter_code>g(bnn_param_list)<jupyter_output><empty_output><jupyter_text>SAFE version for autograd. Uses hstack rather than slice indexing<jupyter_code>def flatten__safe(bnn_param_list):
    flat_arr_list = []
    for layer in bnn_param_list:
        flat_arr_list.append(layer['w'].flatten())
        flat_arr_list.append(layer['b'].flatten())
    return ag_np.hstack(flat_arr_list)
def sum_of_params__safe(bnn_param_list):
    return ag_np.sum(flatten__safe(bnn_param_list))
g = autograd.grad(sum_of_params__safe)<jupyter_output><empty_output><jupyter_text>Try to get gradient (will work!)<jupyter_code>g(bnn_param_list)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11237169097784981, 'fraction_numerical': 0.0016207455429497568, 'mean_word_length': 5.235690235690235, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '452686', 'license': 'MIT', 'n_tokens_mistral': 705, 'n_tokens_neox': 694, 'n_words': 120, 'path': 'notebooks/safe_to_autograd_version_of_flatten_list_of_params.ipynb', 'repo_name': 'tufts-ml-courses/-comp150-bdl-19f-assignments', 'size': '1851'}