Q:

What is the replacement for tf.contrib.layers.layer_norm() in TensorFlow 2.0?

It seems that TF 2.0 does not have tf.contrib. Then, what is the replacement for tf.contrib.layers.layer_norm()? I didn't find it in tensorflow_addons too.

A:

The replacement in the core TensorFlow 2.0 API is tf.keras.lyaers.LayerNormalizaton(). 
Please see the documentation at: https://www.tensorflow.org/api_docs/python/tf/keras/layers/LayerNormalization?version=stable.
Below is a simple usage example:
import numpy as np
import tensorflow as tf

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(10, input_shape=[20], activation="relu"))
# Here is the LayerNormalization layer.
model.add(tf.keras.layers.LayerNormalization())
model.add(tf.keras.layers.Dense(1, activation="sigmoid"))
model.summary()

print(model.predict(np.ones([1, 20])))

