<jupyter_start><jupyter_text>Deferred Initialization:label:`sec_deferred_init`So far, it might seem that we got awaywith being sloppy in setting up our networks.Specifically, we did the following unintuitive things,which might not seem like they should work:* We defined the network architectures  without specifying the input dimensionality.* We added layers without specifying  the output dimension of the previous layer.* We even "initialized" these parameters  before providing enough information to determine  how many parameters our models should contain.You might be surprised that our code runs at all.After all, there is no way the deep learning frameworkcould tell what the input dimensionality of a network would be.The trick here is that the framework *defers initialization*,waiting until the first time we pass data through the model,to infer the sizes of each layer on the fly.Later on, when working with convolutional neural networks,this technique will become even more convenientsince the input dimensionality(i.e., the resolution of an image)will affect the dimensionalityof each subsequent layer.Hence, the ability to set parameterswithout the need to know,at the time of writing the code,what the dimensionality iscan greatly simplify the task of specifyingand subsequently modifying our models.Next, we go deeper into the mechanics of initialization. Instantiating a NetworkTo begin, let us instantiate an MLP.<jupyter_code>from mxnet import np, npx
from mxnet.gluon import nn
npx.set_np()

def get_net():
    net = nn.Sequential()
    net.add(nn.Dense(256, activation='relu'))
    net.add(nn.Dense(10))
    return net

net = get_net()<jupyter_output><empty_output><jupyter_text>At this point, the network cannot possibly knowthe dimensions of the input layer's weightsbecause the input dimension remains unknown.Consequently the framework has not yet initialized any parameters.We confirm by attempting to access the parameters below.<jupyter_code>print(net.collect_params)
print(net.collect_params())<jupyter_output><bound method Block.collect_params of Sequential(
  (0): Dense(-1 -> 256, Activation(relu))
  (1): Dense(-1 -> 10, linear)
)>
sequential0_ (
  Parameter dense0_weight (shape=(256, -1), dtype=float32)
  Parameter dense0_bias (shape=(256,), dtype=float32)
  Parameter dense1_weight (shape=(10, -1), dtype=float32)
  Parameter dense1_bias (shape=(10,), dtype=float32)
)<jupyter_text>Note that while the parameter objects exist,the input dimension to each layer is listed as -1.MXNet uses the special value -1 to indicatethat the parameter dimension remains unknown.At this point, attempts to access `net[0].weight.data()`would trigger a runtime error stating that the networkmust be initialized before the parameters can be accessed.Now let us see what happens when we attempt to initializeparameters via the `initialize` function.<jupyter_code>net.initialize()
net.collect_params()<jupyter_output><empty_output><jupyter_text>As we can see, nothing has changed.When input dimensions are unknown,calls to initialize do not truly initialize the parameters.Instead, this call registers to MXNet that we wish(and optionally, according to which distribution)to initialize the parameters. Next let us pass data through the networkto make the framework finally initialize parameters.<jupyter_code>X = np.random.uniform(size=(2, 20))
net(X)

net.collect_params()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06606255480853551, 'fraction_numerical': 0.013154048523823443, 'mean_word_length': 5.969450101832994, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '66224', 'license': 'MIT', 'n_tokens_mistral': 912, 'n_tokens_neox': 872, 'n_words': 401, 'path': 'd2l-en/mxnet/chapter_deep-learning-computation/deferred-init.ipynb', 'repo_name': 'gr8khan/d2lai', 'size': '3421'}