<commit_before>.. _getstartmodel:

===============
 定义模型
===============

TensorLayer提供两种模型定义模型，静态模型提供直观的代码风格来定义模型，而动态模型提供完全可控的前向传播。

静态模型
===============

.. code-block:: python

  import tensorflow as tf
  from tensorlayer.layers import Input, Dropout, Dense
  from tensorlayer.models import Model

  def get_model(inputs_shape):
      ni = Input(inputs_shape)
      nn = Dropout(keep=0.8)(ni)
      nn = Dense(n_units=800, act=tf.nn.relu, name="dense1")(nn)
      nn = Dropout(keep=0.8)(nn)
      nn = Dense(n_units=800, act=tf.nn.relu)(nn)
      nn = Dropout(keep=0.8)(nn)
      nn = Dense(n_units=10, act=tf.nn.relu)(nn)
      M = Model(inputs=ni, outputs=nn, name="mlp")
      return M

  MLP = get_model([None, 784])
  MLP.eval()
  outputs = MLP(data)

动态模型
=======================

动态模型时，前一层的输出尺寸，需要手动输入到下一层中，以初始化下一层的参数。

.. code-block:: python

  class CustomModel(Model):

      def __init__(self):
          super(CustomModel, self).__init__()

          self.dropout1 = Dropout(keep=0.8)
          self.dense1 = Dense(n_units=800, act=tf.nn.relu, in_channels=784)
          self.dropout2 = Dropout(keep=0.8)#(self.dense1)
          self.dense2 = Dense(n_units=800, act=tf.nn.relu, in_channels=800)
          self.dropout3 = Dropout(keep=0.8)#(self.dense2)
          self.dense3 = Dense(n_units=10, act=tf.nn.relu, in_channels=800)

      def forward(self, x, foo=False):
          z = self.dropout1(x)
          z = self.dense1(z)
          z = self.dropout2(z)
          z = self.dense2(z)
          z = self.dropout3(z)
          out = self.dense3(z)
          if foo:
              out = tf.nn.relu(out)
          return out

  MLP = CustomModel()
  MLP.eval()
  outputs = MLP(data, foo=True) # controls the forward here
  outputs = MLP(data, foo=False)

参数（层）复用
=====================

静态模型的层复用可以如下实现。

.. code-block:: python
<commit_msg>Update get_start_model.rst<commit_after>.. _getstartmodel:

===============
 定义模型
===============

TensorLayer提供两种模型定义模型，静态模型提供直观的代码风格来定义模型，而动态模型提供完全可控的前向传播。

静态模型
===============

.. code-block:: python

  import tensorflow as tf
  from tensorlayer.layers import Input, Dropout, Dense
  from tensorlayer.models import Model

  def get_model(inputs_shape):
      ni = Input(inputs_shape)
      nn = Dropout(keep=0.8)(ni)
      nn = Dense(n_units=800, act=tf.nn.relu, name="dense1")(nn)
      nn = Dropout(keep=0.8)(nn)
      nn = Dense(n_units=800, act=tf.nn.relu)(nn)
      nn = Dropout(keep=0.8)(nn)
      nn = Dense(n_units=10, act=tf.nn.relu)(nn)
      M = Model(inputs=ni, outputs=nn, name="mlp")
      return M

  MLP = get_model([None, 784])
  MLP.eval()
  outputs = MLP(data)

动态模型
=======================

动态模型时，前一层的输出尺寸，需要手动输入到下一层中，以初始化下一层的参数。

.. code-block:: python

  class CustomModel(Model):

      def __init__(self):
          super(CustomModel, self).__init__()

          self.dropout1 = Dropout(keep=0.8)
          self.dense1 = Dense(n_units=800, act=tf.nn.relu, in_channels=784)
          self.dropout2 = Dropout(keep=0.8)#(self.dense1)
          self.dense2 = Dense(n_units=800, act=tf.nn.relu, in_channels=800)
          self.dropout3 = Dropout(keep=0.8)#(self.dense2)
          self.dense3 = Dense(n_units=10, act=tf.nn.relu, in_channels=800)

      def forward(self, x, foo=False):
          z = self.dropout1(x)
          z = self.dense1(z)
          z = self.dropout2(z)
          z = self.dense2(z)
          z = self.dropout3(z)
          out = self.dense3(z)
          if foo:
              out = tf.nn.relu(out)
          return out

  MLP = CustomModel()
  MLP.eval()
  outputs = MLP(data, foo=True) # controls the forward here
  outputs = MLP(data, foo=False)

切换训练/测试模式
=============================

.. code-block:: python

  # method 1: switch before forward
  Model.train() # enable dropout, batch norm moving avg ...
  output = Model(train_data) 
  ... # training code here
  Model.eval()  # disable dropout, batch norm moving avg ...
  output = Model(test_data) 
  ... # testing code here
  
  # method 2: switch while forward
  output = Model(train_data, is_train=True)
  output = Model(test_data, is_train=False)


参数（层）复用
=====================

静态模型的层复用可以如下实现。

.. code-block:: python

{'fraction_non_alphanumeric': 0.17746950490313324, 'fraction_numerical': 0.026787849796699355, 'mean_word_length': 2.923076923076923, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3803259', 'n_tokens_mistral': 1743, 'n_tokens_neox': 1712, 'n_words': 284}