def _luong_dot_score(query, keys, scale):
    '\n    Implements the Luong-style dot scoring function.\n\n    This attention has two forms. The first is standard Luong attention, as described in:\n\n    Minh-Thang Luong, Hieu Pham, Christopher D. Manning.\n    "Effective Approaches to Attention-based Neural Machine Translation."\n    EMNLP 2015.  https://arxiv.org/abs/1508.04025\n\n    The second is the scaled form inspired partly by the normalized form of\n    Bahdanau attention.\n\n    To enable the second form, call this function with `scale=True`.\n\n    This implementation is derived from: `tensorflow.contrib.seq2seq.python.ops.attention_wrapper`\n\n    Arguments:\n        query (tf.Tensor):\n            Decoder cell outputs to compare to the keys (memory).\n            The shape is expected to be shape=(B, num_units) with B being the batch size\n            and `num_units` being the output size of the decoder_cell.\n\n        keys (tf.Tensor):\n            Processed memory (usually the encoder states processed by the memory_layer).\n            The shape is expected to be shape=(B, X, num_units) with B being the batch size\n            and `num_units` being the output size of the memory_layer. X may be the\n            maximal length of the encoder time domain or in the case of local attention the\n            window size.\n\n        scale (boolean):\n            Whether to apply a scale to the score function.\n\n    Returns:\n        score (tf.Tensor):\n            A tensor with shape=(B, X) containing the non-normalized score values.\n\n    Raises:\n      ValueError: If `key` and `query` depths do not match.\n\n    '
    depth = query.get_shape()[(- 1)]
    key_units = keys.get_shape()[(- 1)]
    if (depth != key_units):
        raise ValueError(("Incompatible or unknown inner dimensions between query and keys.  Query (%s) has units: %s.  Keys (%s) have units: %s.  Perhaps you need to set num_units to the keys' dimension (%s)?" % (query, depth, keys, key_units, key_units)))
    dtype = query.dtype
    query = tf.expand_dims(query, 1)
    score = tf.matmul(query, keys, transpose_b=True)
    score = tf.squeeze(score, [1])
    if scale:
        g = tf.get_variable('attention_g', dtype=dtype, initializer=tf.ones_initializer, shape=())
        score = (g * score)
    return score
Implements the Luong-style dot scoring function.

This attention has two forms. The first is standard Luong attention, as described in:

Minh-Thang Luong, Hieu Pham, Christopher D. Manning.
"Effective Approaches to Attention-based Neural Machine Translation."
EMNLP 2015.  https://arxiv.org/abs/1508.04025

The second is the scaled form inspired partly by the normalized form of
Bahdanau attention.

To enable the second form, call this function with `scale=True`.

This implementation is derived from: `tensorflow.contrib.seq2seq.python.ops.attention_wrapper`

Arguments:
    query (tf.Tensor):
        Decoder cell outputs to compare to the keys (memory).
        The shape is expected to be shape=(B, num_units) with B being the batch size
        and `num_units` being the output size of the decoder_cell.

    keys (tf.Tensor):
        Processed memory (usually the encoder states processed by the memory_layer).
        The shape is expected to be shape=(B, X, num_units) with B being the batch size
        and `num_units` being the output size of the memory_layer. X may be the
        maximal length of the encoder time domain or in the case of local attention the
        window size.

    scale (boolean):
        Whether to apply a scale to the score function.

Returns:
    score (tf.Tensor):
        A tensor with shape=(B, X) containing the non-normalized score values.

Raises:
  ValueError: If `key` and `query` depths do not match.
def _luong_dot_score(query, keys, scale):
    '\n    Implements the Luong-style dot scoring function.\n\n    This attention has two forms. The first is standard Luong attention, as described in:\n\n    Minh-Thang Luong, Hieu Pham, Christopher D. Manning.\n    "Effective Approaches to Attention-based Neural Machine Translation."\n    EMNLP 2015.  https://arxiv.org/abs/1508.04025\n\n    The second is the scaled form inspired partly by the normalized form of\n    Bahdanau attention.\n\n    To enable the second form, call this function with `scale=True`.\n\n    This implementation is derived from: `tensorflow.contrib.seq2seq.python.ops.attention_wrapper`\n\n    Arguments:\n        query (tf.Tensor):\n            Decoder cell outputs to compare to the keys (memory).\n            The shape is expected to be shape=(B, num_units) with B being the batch size\n            and `num_units` being the output size of the decoder_cell.\n\n        keys (tf.Tensor):\n            Processed memory (usually the encoder states processed by the memory_layer).\n            The shape is expected to be shape=(B, X, num_units) with B being the batch size\n            and `num_units` being the output size of the memory_layer. X may be the\n            maximal length of the encoder time domain or in the case of local attention the\n            window size.\n\n        scale (boolean):\n            Whether to apply a scale to the score function.\n\n    Returns:\n        score (tf.Tensor):\n            A tensor with shape=(B, X) containing the non-normalized score values.\n\n    Raises:\n      ValueError: If `key` and `query` depths do not match.\n\n    '
    depth = query.get_shape()[(- 1)]
    key_units = keys.get_shape()[(- 1)]
    if (depth != key_units):
        raise ValueError(("Incompatible or unknown inner dimensions between query and keys.  Query (%s) has units: %s.  Keys (%s) have units: %s.  Perhaps you need to set num_units to the keys' dimension (%s)?" % (query, depth, keys, key_units, key_units)))
    dtype = query.dtype
    query = tf.expand_dims(query, 1)
    score = tf.matmul(query, keys, transpose_b=True)
    score = tf.squeeze(score, [1])
    if scale:
        g = tf.get_variable('attention_g', dtype=dtype, initializer=tf.ones_initializer, shape=())
        score = (g * score)
    return score