parameters: (input,
                    label,
                    weight=None,
                    ignore_index=-100,
                    reduction='mean',
                    soft_label=False,
                    axis=-1,
                    use_softmax=True,
                    name=None)
docstring: r"""
      By default, this operator implements the cross entropy loss function with softmax. This function 
      combines the calculation of the softmax operation and the cross entropy loss function 
      to provide a more numerically stable computing. 
  
      This operator will calculate the cross entropy loss function without softmax when use_softmax=False.
  
      By default, this operator will calculate the mean of the result, and you can also affect 
      the default behavior by using the reduction parameter. Please refer to the part of 
      parameters for details.
  
      This operator can be used to calculate the softmax cross entropy loss with soft and hard labels.
      Where, the hard labels mean the actual label value, 0, 1, 2, etc.  And the soft labels 
      mean the probability of the actual label, 0.6, 0.8, 0.2, etc.
  
      The calculation of this operator includes the following two steps.
  
      - **1.softmax cross entropy**
  
          1. Hard label (each sample can only be assigned into one category)
  
          1.1. when use_softmax=True
  
              .. math::
                \\loss_j=-\text{logits}_{label_j}+\log\left(\sum_{i=0}^{C}\exp(\text{logits}_i)\right) , j = 1,...,N
  
              where, N is the number of samples and C is the number of categories.
  
          1.2. when use_softmax=False
  
              .. math::
                \\loss_j=-\log\left({P}_{label_j}\right) , j = 1,...,N
  
              where, N is the number of samples and C is the number of categories, P is input(the output of softmax).
  
  
          2. Soft label (each sample is assigned to multiple categories with a certain probability, and the probability sum is 1).
  
          2.1. when use_softmax=True
  
              .. math::
                \\loss_j=-\sum_{i=0}^{C}\text{label}_i\left(\text{logits}_i-\log\left(\sum_{i=0}^{C}\exp(\text{logits}_i)\right)\right) , j = 1,...,N
  
              where, N is the number of samples and C is the number of categories.
  
          2.2. when use_softmax=False
  
              .. math::
                \\loss_j=-\sum_{j=0}^{C}\left({label}_j*\log\left({P}_{label_j}\right)\right) , j = 1,...,N
  
              where, N is the number of samples and C is the number of categories, P is input(the output of softmax).
  
  
  
  
      - **2. Weight and reduction processing**
  
          1. Weight
  
              If the ``weight`` parameter is ``None`` , go to the next step directly.
  
              If the ``weight`` parameter is not ``None`` , the cross entropy of each sample is weighted by weight
              according to soft_label = False or True as follows.
  
              1.1. Hard labels (soft_label = False)
  
              .. math::
                  \\loss_j=loss_j*weight[label_j] 
  
  
              1.2. Soft labels (soft_label = True)
  
               .. math::
                  \\loss_j=loss_j*\sum_{i}\left(weight[label_i]*logits_i\right)
  
          2. reduction
  
              2.1 if the ``reduction`` parameter is ``none`` 
  
                  Return the previous result directly
  
              2.2 if the ``reduction`` parameter is ``sum`` 
  
                  Return the sum of the previous results
  
              .. math::
                 \\loss=\sum_{j}loss_j
  
              2.3 if the ``reduction`` parameter is ``mean`` , it will be processed according to 
              the ``weight`` parameter as follows. 
  
              2.3.1. If the  ``weight``  parameter is ``None`` 
  
                     Return the average value of the previous results
  
               .. math::
                  \\loss=\sum_{j}loss_j/N
  
                    where, N is the number of samples and C is the number of categories.
  
              2.3.2. If the 'weight' parameter is not 'None', the weighted average value of the previous result will be returned
  
              1. Hard labels (soft_label = False)
  
               .. math::
                  \\loss=\sum_{j}loss_j/\sum_{j}weight[label_j] 
  
              2. Soft labels (soft_label = True)
  
               .. math::
                  \\loss=\sum_{j}loss_j/\sum_{j}\left(\sum_{i}weight[label_i]\right)
   
   
      Parameters:
  
          - **input** (Tensor)
  
              Input tensor, the data type is float32, float64. Shape is
  	    :math:`[N_1, N_2, ..., N_k, C]`, where C is number of classes ,  ``k >= 1`` . 
  
              Note: 
  
                  1. when use_softmax=True, it expects unscaled logits. This operator should not be used with the 
                  output of softmax operator, which will produce incorrect results.
  
                  2. when use_softmax=False, it expects the output of softmax operator.
   
          - **label** (Tensor)
  
              1. If soft_label=False, the shape is
              :math:`[N_1, N_2, ..., N_k]` or :math:`[N_1, N_2, ..., N_k, 1]`, k >= 1.
              the data type is int32, int64, float32, float64, where each value is [0, C-1].
  
              2. If soft_label=True, the shape and data type should be same with ``input`` , 
              and the sum of the labels for each sample should be 1.
  
          - **weight** (Tensor, optional)
  
              a manual rescaling weight given to each class. 
              If given, has to be a Tensor of size C and the data type is float32, float64. 
              Default is ``'None'`` .
  
          - **ignore_index** (int64, optional)
  
              Specifies a target value that is ignored
              and does not contribute to the loss. A negative value means that no label 
              value needs to be ignored. Only valid when soft_label = False.  
              Default is ``-100`` .
  
          - **reduction** (str, optional)
  
              Indicate how to average the loss by batch_size,
              the candicates are ``'none'`` | ``'mean'`` | ``'sum'``.
              If :attr:`reduction` is ``'mean'``, the reduced mean loss is returned;
              If :attr:`size_average` is ``'sum'``, the reduced sum loss is returned.
              If :attr:`reduction` is ``'none'``, the unreduced loss is returned.
              Default is ``'mean'``.
  
          - **soft_label** (bool, optional)
  
              Indicate whether label is soft. 
              Default is ``False``.
  
          - **axis** (int, optional)
  
              The index of dimension to perform softmax calculations. 
              It should be in range :math:`[-1, rank - 1]`, where :math:`rank` is the 
              number of dimensions of input :attr:`input`. 
              Default is ``-1`` .
  
          - **use_softmax** (bool, optional)
  
              Indicate whether compute softmax before cross_entropy.
              Default is ``True``.
  
          - **name** (str, optional)
  
              The name of the operator. Default is ``None`` .
              For more information, please refer to :ref:`api_guide_Name` .
  
      Returns:
  
          Tensor. Return the softmax cross_entropy loss of ``input`` and ``label``.
          The data type is the same as input.
  
          If :attr:`reduction` is ``'mean'`` or ``'sum'`` , the dimension of return value is ``1``.
  
          If :attr:`reduction` is ``'none'``:
  
          1. If soft_label = False, the dimension of return value is the same with ``label`` . 
  
          2. if soft_label = True, the dimension of return value is :math:`[N_1, N_2, ..., N_k, 1]` . 
  
  
       Example1(hard labels):
  
          .. code-block:: python
              
              import paddle
              paddle.seed(99999)
              N=100
              C=200
              reduction='mean'
              input =  paddle.rand([N, C], dtype='float64')  
              label =  paddle.randint(0, C, shape=[N], dtype='int64')
              weight = paddle.rand([C], dtype='float64') 
              
              cross_entropy_loss = paddle.nn.loss.CrossEntropyLoss(
                  weight=weight, reduction=reduction)
              dy_ret = cross_entropy_loss(
                                         input,
                                         label)
              print(dy_ret.numpy()) #[5.41993642]
  
  
      Example2(soft labels):
  
          .. code-block:: python
              
              import paddle
              paddle.seed(99999)
              axis = -1
              ignore_index = -100
              N = 4
              C = 3
              shape = [N, C]
              reduction='mean'
              weight = None
              logits = paddle.uniform(shape, dtype='float64', min=0.1, max=1.0)
              labels = paddle.uniform(shape, dtype='float64', min=0.1, max=1.0)
              labels /= paddle.sum(labels, axis=axis, keepdim=True)
              paddle_loss_mean = paddle.nn.functional.cross_entropy(
                                                                    logits,  
                                                                    labels, 
                                                                    soft_label=True, 
                                                                    axis=axis,
                                                                    weight=weight,
                                                                    reduction=reduction)
              print(paddle_loss_mean.numpy()) #[1.12908343]
docstring_summary: r"""
      By default, this operator implements the cross entropy loss function with softmax. This function 
      combines the calculation of the softmax operation and the cross entropy loss function 
      to provide a more numerically stable computing.
docstring_tokens: r By default this operator implements the cross entropy loss function with softmax . This function combines the calculation of the softmax operation and the cross entropy loss function to provide a more numerically stable computing .
function: def cross_entropy(input,
                    label,
                    weight=None,
                    ignore_index=-100,
                    reduction='mean',
                    soft_label=False,
                    axis=-1,
                    use_softmax=True,
                    name=None):
      r"""
      By default, this operator implements the cross entropy loss function with softmax. This function 
      combines the calculation of the softmax operation and the cross entropy loss function 
      to provide a more numerically stable computing. 
  
      This operator will calculate the cross entropy loss function without softmax when use_softmax=False.
  
      By default, this operator will calculate the mean of the result, and you can also affect 
      the default behavior by using the reduction parameter. Please refer to the part of 
      parameters for details.
  
      This operator can be used to calculate the softmax cross entropy loss with soft and hard labels.
      Where, the hard labels mean the actual label value, 0, 1, 2, etc.  And the soft labels 
      mean the probability of the actual label, 0.6, 0.8, 0.2, etc.
  
      The calculation of this operator includes the following two steps.
  
      - **1.softmax cross entropy**
  
          1. Hard label (each sample can only be assigned into one category)
  
          1.1. when use_softmax=True
  
              .. math::
                \\loss_j=-\text{logits}_{label_j}+\log\left(\sum_{i=0}^{C}\exp(\text{logits}_i)\right) , j = 1,...,N
  
              where, N is the number of samples and C is the number of categories.
  
          1.2. when use_softmax=False
  
              .. math::
                \\loss_j=-\log\left({P}_{label_j}\right) , j = 1,...,N
  
              where, N is the number of samples and C is the number of categories, P is input(the output of softmax).
  
  
          2. Soft label (each sample is assigned to multiple categories with a certain probability, and the probability sum is 1).
  
          2.1. when use_softmax=True
  
              .. math::
                \\loss_j=-\sum_{i=0}^{C}\text{label}_i\left(\text{logits}_i-\log\left(\sum_{i=0}^{C}\exp(\text{logits}_i)\right)\right) , j = 1,...,N
  
              where, N is the number of samples and C is the number of categories.
  
          2.2. when use_softmax=False
  
              .. math::
                \\loss_j=-\sum_{j=0}^{C}\left({label}_j*\log\left({P}_{label_j}\right)\right) , j = 1,...,N
  
              where, N is the number of samples and C is the number of categories, P is input(the output of softmax).
  
  
  
  
      - **2. Weight and reduction processing**
  
          1. Weight
  
              If the ``weight`` parameter is ``None`` , go to the next step directly.
  
              If the ``weight`` parameter is not ``None`` , the cross entropy of each sample is weighted by weight
              according to soft_label = False or True as follows.
  
              1.1. Hard labels (soft_label = False)
  
              .. math::
                  \\loss_j=loss_j*weight[label_j] 
  
  
              1.2. Soft labels (soft_label = True)
  
               .. math::
                  \\loss_j=loss_j*\sum_{i}\left(weight[label_i]*logits_i\right)
  
          2. reduction
  
              2.1 if the ``reduction`` parameter is ``none`` 
  
                  Return the previous result directly
  
              2.2 if the ``reduction`` parameter is ``sum`` 
  
                  Return the sum of the previous results
  
              .. math::
                 \\loss=\sum_{j}loss_j
  
              2.3 if the ``reduction`` parameter is ``mean`` , it will be processed according to 
              the ``weight`` parameter as follows. 
  
              2.3.1. If the  ``weight``  parameter is ``None`` 
  
                     Return the average value of the previous results
  
               .. math::
                  \\loss=\sum_{j}loss_j/N
  
                    where, N is the number of samples and C is the number of categories.
  
              2.3.2. If the 'weight' parameter is not 'None', the weighted average value of the previous result will be returned
  
              1. Hard labels (soft_label = False)
  
               .. math::
                  \\loss=\sum_{j}loss_j/\sum_{j}weight[label_j] 
  
              2. Soft labels (soft_label = True)
  
               .. math::
                  \\loss=\sum_{j}loss_j/\sum_{j}\left(\sum_{i}weight[label_i]\right)
   
   
      Parameters:
  
          - **input** (Tensor)
  
              Input tensor, the data type is float32, float64. Shape is
  	    :math:`[N_1, N_2, ..., N_k, C]`, where C is number of classes ,  ``k >= 1`` . 
  
              Note: 
  
                  1. when use_softmax=True, it expects unscaled logits. This operator should not be used with the 
                  output of softmax operator, which will produce incorrect results.
  
                  2. when use_softmax=False, it expects the output of softmax operator.
   
          - **label** (Tensor)
  
              1. If soft_label=False, the shape is
              :math:`[N_1, N_2, ..., N_k]` or :math:`[N_1, N_2, ..., N_k, 1]`, k >= 1.
              the data type is int32, int64, float32, float64, where each value is [0, C-1].
  
              2. If soft_label=True, the shape and data type should be same with ``input`` , 
              and the sum of the labels for each sample should be 1.
  
          - **weight** (Tensor, optional)
  
              a manual rescaling weight given to each class. 
              If given, has to be a Tensor of size C and the data type is float32, float64. 
              Default is ``'None'`` .
  
          - **ignore_index** (int64, optional)
  
              Specifies a target value that is ignored
              and does not contribute to the loss. A negative value means that no label 
              value needs to be ignored. Only valid when soft_label = False.  
              Default is ``-100`` .
  
          - **reduction** (str, optional)
  
              Indicate how to average the loss by batch_size,
              the candicates are ``'none'`` | ``'mean'`` | ``'sum'``.
              If :attr:`reduction` is ``'mean'``, the reduced mean loss is returned;
              If :attr:`size_average` is ``'sum'``, the reduced sum loss is returned.
              If :attr:`reduction` is ``'none'``, the unreduced loss is returned.
              Default is ``'mean'``.
  
          - **soft_label** (bool, optional)
  
              Indicate whether label is soft. 
              Default is ``False``.
  
          - **axis** (int, optional)
  
              The index of dimension to perform softmax calculations. 
              It should be in range :math:`[-1, rank - 1]`, where :math:`rank` is the 
              number of dimensions of input :attr:`input`. 
              Default is ``-1`` .
  
          - **use_softmax** (bool, optional)
  
              Indicate whether compute softmax before cross_entropy.
              Default is ``True``.
  
          - **name** (str, optional)
  
              The name of the operator. Default is ``None`` .
              For more information, please refer to :ref:`api_guide_Name` .
  
      Returns:
  
          Tensor. Return the softmax cross_entropy loss of ``input`` and ``label``.
          The data type is the same as input.
  
          If :attr:`reduction` is ``'mean'`` or ``'sum'`` , the dimension of return value is ``1``.
  
          If :attr:`reduction` is ``'none'``:
  
          1. If soft_label = False, the dimension of return value is the same with ``label`` . 
  
          2. if soft_label = True, the dimension of return value is :math:`[N_1, N_2, ..., N_k, 1]` . 
  
  
       Example1(hard labels):
  
          .. code-block:: python
              
              import paddle
              paddle.seed(99999)
              N=100
              C=200
              reduction='mean'
              input =  paddle.rand([N, C], dtype='float64')  
              label =  paddle.randint(0, C, shape=[N], dtype='int64')
              weight = paddle.rand([C], dtype='float64') 
              
              cross_entropy_loss = paddle.nn.loss.CrossEntropyLoss(
                  weight=weight, reduction=reduction)
              dy_ret = cross_entropy_loss(
                                         input,
                                         label)
              print(dy_ret.numpy()) #[5.41993642]
  
  
      Example2(soft labels):
  
          .. code-block:: python
              
              import paddle
              paddle.seed(99999)
              axis = -1
              ignore_index = -100
              N = 4
              C = 3
              shape = [N, C]
              reduction='mean'
              weight = None
              logits = paddle.uniform(shape, dtype='float64', min=0.1, max=1.0)
              labels = paddle.uniform(shape, dtype='float64', min=0.1, max=1.0)
              labels /= paddle.sum(labels, axis=axis, keepdim=True)
              paddle_loss_mean = paddle.nn.functional.cross_entropy(
                                                                    logits,  
                                                                    labels, 
                                                                    soft_label=True, 
                                                                    axis=axis,
                                                                    weight=weight,
                                                                    reduction=reduction)
              print(paddle_loss_mean.numpy()) #[1.12908343]
  
      """
  
      if reduction not in ['sum', 'mean', 'none']:
          raise ValueError(
              "The value of 'reduction' in softmax_cross_entropy"
              "should be 'sum', 'mean' or 'none', but received %s, which is not allowed."
              % reduction)
      if ignore_index > 0 and soft_label == True:
          raise ValueError(
              "When soft_label == True, the value of 'ignore_index' in softmax_cross_entropy"
              "should be '-100', but received %s, which is not allowed." %
              ignore_index)
  
      input_dims = len(list(input.shape))
      if input_dims == 0:
          raise ValueError('The dimention of input should be larger than zero!')
  
      label_dims = len(list(label.shape))
      if input_dims - 1 != label_dims and input_dims != label_dims:
          raise ValueError(
              'Expected nput_dims - 1 = label_dims or input_dims == label_dims\
               (got nput_dims{}, label_dims{})'.format(input_dims, label_dims))
      if input_dims - 1 == label_dims:
          label = paddle.unsqueeze(label, axis=axis)
      if in_dygraph_mode():
          if soft_label == False:
              valid_label = paddle.cast(
                  label != ignore_index, dtype=label.dtype) * label
              label_min = paddle.min(valid_label)
              label_max = paddle.max(valid_label)
              if label_min < 0:
                  raise ValueError("label should not out of bound, but got{}".
                                   format(label_min))
              if label_max >= input.shape[axis]:
                  raise ValueError("label should not out of bound, but got{}".
                                   format(label_max))
          if core.is_compiled_with_npu():
              _, _, out = _C_ops.softmax_with_cross_entropy(
                  input, label, 'soft_label', soft_label, 'ignore_index',
                  ignore_index, 'numeric_stable_mode', True, 'axis', axis,
                  'use_softmax', use_softmax)
          else:
              _, out = _C_ops.softmax_with_cross_entropy(
                  input, label, 'soft_label', soft_label, 'ignore_index',
                  ignore_index, 'numeric_stable_mode', True, 'axis', axis,
                  'use_softmax', use_softmax)
  
          if weight is not None:
  
              # trans weight from class to sample, shape:N or [N,H,W] for 1d and 2d cases.
              if soft_label == True:
                  # chajchaj:
                  # weight's shape is C, where C is class num.
                  # for 1d case: label's shape is [N,C], weight_gather's shape is N.
                  # for 2d case: label's shape is [N,H,W,C], weight_gather's shape is [N,H,W].
                  weight_gather = paddle.matmul(
                      x=paddle.cast(label, weight.dtype),
                      y=weight,
                      transpose_x=False,
                      transpose_y=True)
                  out_shape = list(out.shape)
                  weight_gather_reshape = reshape(weight_gather, shape=out_shape)
                  out = paddle.cast(out, weight_gather_reshape.dtype)
  
                  out = _C_ops.elementwise_mul(out, weight_gather_reshape)
  
              else:
                  if input.shape[axis] != weight.shape[-1]:
                      raise ValueError(
                          "input's class_dimension({}) must equal to "
                          "weight's class_dimension({}) "
                          "when weight is provided" \
                              .format(input.shape[axis], weight.shape[-1]))
  
                  ignore_weight_mask = paddle.cast((label != ignore_index),
                                                   out.dtype)
                  if ignore_weight_mask.ndim > 1 and ignore_weight_mask.shape[
                          axis] == 1:
                      # TODO: Temporarily use squeeze instead of squeeze_
                      ignore_weight_mask = paddle.squeeze(ignore_weight_mask,
                                                          axis)
                  if axis != -1 and axis != valid_label.ndim - 1:
                      temp_perm = list(range(axis % valid_label.ndim)) \
                                  + list(range((axis % valid_label.ndim + 1), valid_label.ndim)) \
                                  + [axis % valid_label.ndim]
                      weight_gather = _C_ops.gather_nd(
                          weight, valid_label.transpose(temp_perm))
                  else:
                      weight_gather = _C_ops.gather_nd(weight, valid_label)
                  weight_gather = _C_ops.elementwise_mul(weight_gather,
                                                         ignore_weight_mask)
                  input_shape = list(label.shape)
                  weight_gather_reshape = reshape(
                      weight_gather, shape=input_shape)
                  out = paddle.cast(out, weight_gather_reshape.dtype)
                  out = _C_ops.elementwise_mul(out, weight_gather_reshape)
  
          if reduction == "sum":
              #   because of fluid_softmax_with_cross_entropy op's inner logic,
              #   in the out tensor of this op, the loss of sample with class_index==ignore_index is 0
              #   so, reduce_sum all directly is ok
              return _C_ops.reduce_sum(out, 'reduce_all', True)
          elif reduction == "mean":
              # 1. if weight==none,
              #     numerator: reduce_sum all loss directly is ok causeof fluid_softmax_with_cross_entropy's inner logic
              #     denominator: count sample num with class_index!=ignore_index
              # 2. else
              #     numerator: loss's weighted sum
              #     denominator: cal the sum of weight where the sample's class_index!=ignore_index
              if ignore_index != -100:
                  out_sum = _C_ops.reduce_sum(out, 'reduce_all', True)
                  # for each label[i],set 1 or 0, according to ignore_index
                  # mask[i]=0, if label[i]==ignore_index
                  # mask[i]=1, otherwise
                  mask = (label != ignore_index)
                  if weight is None:
                      mask = paddle.cast(mask, dtype=out_sum.dtype)
                      count = _C_ops.reduce_sum(mask, 'reduce_all', True)
                      ret = out_sum / (count + (count == 0.0))
                  else:
                      mask = paddle.cast(mask, weight_gather_reshape.dtype)
                      weight_ignored = _C_ops.elementwise_mul(
                          mask, weight_gather_reshape)
                      weight_sum = _C_ops.reduce_sum(weight_ignored, 'reduce_all',
                                                     True)
                      ret = out_sum / (weight_sum + (weight_sum == 0.0))
                  return ret
              elif weight is not None:
                  out_sum = _C_ops.reduce_sum(out, 'reduce_all', True)
                  total_weight = _C_ops.reduce_sum(weight_gather_reshape,
                                                   'reduce_all', True)
                  return out_sum / (total_weight + (total_weight == 0.0))
              else:
                  return _C_ops.mean(out)
  
          else:
              if input_dims - 1 == label_dims:
                  out = paddle.squeeze(out, axis=axis)
              return out
  
      fluid.data_feeder.check_variable_and_dtype(
          input, 'input', ['float32', 'float64'], 'softmax_cross_entropy')
      fluid.data_feeder.check_variable_and_dtype(
          label, 'label',
          ['uint8', 'int8', 'int16', 'int32', 'int64', 'float32', 'float64'],
          'softmax_cross_entropy')
      attrs = {
          'soft_label': soft_label,
          'ignore_index': ignore_index,
          'numeric_stable_mode': True,
          'axis': axis,
          'use_softmax': use_softmax
      }
      helper = LayerHelper('softmax_with_cross_entropy', **locals())
      softmax = helper.create_variable_for_type_inference(dtype=input.dtype)
      out = helper.create_variable_for_type_inference(dtype=input.dtype)
      helper.append_op(
          type='softmax_with_cross_entropy',
          inputs={'Logits': input,
                  'Label': label},
          outputs={'Softmax': softmax,
                   'Loss': out},
          attrs=attrs)
  
      if weight is not None:
          fluid.data_feeder.check_variable_and_dtype(
              weight, 'weight', ['float32', 'float64'], 'softmax_cross_entropy')
          weight_name = name if reduction == 'none' else None
          if soft_label == True:
              # chajchaj:
              # trans weight from class to sample, shape:N or [N,H,W] for 1d and 2d cases.
              # weight's shape is C, where C is class num.
              # for 1d case: label's shape is [N,C], weight_gather's shape is N.
              # for 2d case: label's shape is [N,H,W,C], weight_gather's shape is [N,H,W].
              weight_gather = paddle.matmul(
                  x=paddle.cast(label, weight.dtype),
                  y=weight,
                  transpose_x=False,
                  transpose_y=True)
  
              out_shape = list(out.shape)
              weight_gather_reshape = reshape(weight_gather, shape=out_shape)
              out = paddle.cast(out, weight_gather_reshape.dtype)
          else:
              if input.shape[axis] != weight.shape[-1]:
                  raise ValueError("input's class_dimension({}) must equal to "
                                   "weight's class_dimension({}) "
                                   "when weight is provided" \
                                   .format(input.shape[axis], weight.shape[-1]))
  
              valid_label = paddle.multiply(
                  paddle.cast(
                      label != ignore_index, dtype=label.dtype), label)
              ignore_weight_mask = paddle.cast((label != ignore_index),
                                               input.dtype)
              if ignore_weight_mask.ndim > 1 and ignore_weight_mask.shape[
                      axis] == 1:
                  ignore_weight_mask = paddle.squeeze(ignore_weight_mask, axis)
              if axis != -1 and axis != valid_label.ndim - 1:
                  temp_perm = list(range(axis % valid_label.ndim)) \
                              + list(range((axis % valid_label.ndim + 1), valid_label.ndim)) \
                              + [axis % valid_label.ndim]
                  weight_gather = paddle.gather_nd(
                      weight, paddle.transpose(valid_label, temp_perm))
              else:
                  weight_gather = paddle.gather_nd(weight, valid_label)
              weight_gather = paddle.multiply(weight_gather, ignore_weight_mask)
  
              input_shape = list(label.shape)
              weight_gather_reshape = reshape(weight_gather, shape=input_shape)
          out = paddle.multiply(out, weight_gather_reshape, name=weight_name)
  
      if reduction == "sum":
          return paddle.sum(out, name=name)
      elif reduction == "mean":
          if ignore_index != -100:
              out_sum = paddle.sum(out, name=name)
              # for each label[i],set 1 or 0, according to ignore_index
              # mask[i]=0, if label[i]==ignore_index
              # mask[i]=1, otherwise
              mask = (label != ignore_index)
              if (weight is None):
                  mask = paddle.cast(mask, dtype=out_sum.dtype)
                  count = paddle.sum(mask, name=name)
                  ret = out_sum / (count + (count == 0.0))
              else:
                  mask = paddle.cast(mask, weight_gather_reshape.dtype)
                  weight_ignored = paddle.multiply(mask, weight_gather_reshape)
                  weight_sum = paddle.sum(weight_ignored, name=name)
                  ret = out_sum / (weight_sum + (weight_sum == 0.0))
              return ret
          elif weight is not None:
              out_sum = paddle.sum(out, name=name)
              total_weight = paddle.sum(weight_gather_reshape)
              return out_sum / (total_weight + (total_weight == 0.0))
          else:
              return paddle.mean(out, name=name)
  
      else:
          if input_dims - 1 == label_dims:
              out = paddle.squeeze(out, axis=axis)
  
          return out
function_tokens: def cross_entropy ( input , label , weight = None , ignore_index = - 100 , reduction = 'mean' , soft_label = False , axis = - 1 , use_softmax = True , name = None ) : if reduction not in [ 'sum' , 'mean' , 'none' ] : raise ValueError ( "The value of 'reduction' in softmax_cross_entropy" "should be 'sum', 'mean' or 'none', but received %s, which is not allowed." % reduction ) if ignore_index > 0 and soft_label == True : raise ValueError ( "When soft_label == True, the value of 'ignore_index' in softmax_cross_entropy" "should be '-100', but received %s, which is not allowed." % ignore_index ) input_dims = len ( list ( input . shape ) ) if input_dims == 0 : raise ValueError ( 'The dimention of input should be larger than zero!' ) label_dims = len ( list ( label . shape ) ) if input_dims - 1 != label_dims and input_dims != label_dims : raise ValueError ( 'Expected nput_dims - 1 = label_dims or input_dims == label_dims\
               (got nput_dims{}, label_dims{})' . format ( input_dims , label_dims ) ) if input_dims - 1 == label_dims : label = paddle . unsqueeze ( label , axis = axis ) if in_dygraph_mode ( ) : if soft_label == False : valid_label = paddle . cast ( label != ignore_index , dtype = label . dtype ) * label label_min = paddle . min ( valid_label ) label_max = paddle . max ( valid_label ) if label_min < 0 : raise ValueError ( "label should not out of bound, but got{}" . format ( label_min ) ) if label_max >= input . shape [ axis ] : raise ValueError ( "label should not out of bound, but got{}" . format ( label_max ) ) if core . is_compiled_with_npu ( ) : _ , _ , out = _C_ops . softmax_with_cross_entropy ( input , label , 'soft_label' , soft_label , 'ignore_index' , ignore_index , 'numeric_stable_mode' , True , 'axis' , axis , 'use_softmax' , use_softmax ) else : _ , out = _C_ops . softmax_with_cross_entropy ( input , label , 'soft_label' , soft_label , 'ignore_index' , ignore_index , 'numeric_stable_mode' , True , 'axis' , axis , 'use_softmax' , use_softmax ) if weight is not None : # trans weight from class to sample, shape:N or [N,H,W] for 1d and 2d cases. if soft_label == True : # chajchaj: # weight's shape is C, where C is class num. # for 1d case: label's shape is [N,C], weight_gather's shape is N. # for 2d case: label's shape is [N,H,W,C], weight_gather's shape is [N,H,W]. weight_gather = paddle . matmul ( x = paddle . cast ( label , weight . dtype ) , y = weight , transpose_x = False , transpose_y = True ) out_shape = list ( out . shape ) weight_gather_reshape = reshape ( weight_gather , shape = out_shape ) out = paddle . cast ( out , weight_gather_reshape . dtype ) out = _C_ops . elementwise_mul ( out , weight_gather_reshape ) else : if input . shape [ axis ] != weight . shape [ - 1 ] : raise ValueError ( "input's class_dimension({}) must equal to " "weight's class_dimension({}) " "when weight is provided" . format ( input . shape [ axis ] , weight . shape [ - 1 ] ) ) ignore_weight_mask = paddle . cast ( ( label != ignore_index ) , out . dtype ) if ignore_weight_mask . ndim > 1 and ignore_weight_mask . shape [ axis ] == 1 : # TODO: Temporarily use squeeze instead of squeeze_ ignore_weight_mask = paddle . squeeze ( ignore_weight_mask , axis ) if axis != - 1 and axis != valid_label . ndim - 1 : temp_perm = list ( range ( axis % valid_label . ndim ) ) + list ( range ( ( axis % valid_label . ndim + 1 ) , valid_label . ndim ) ) + [ axis % valid_label . ndim ] weight_gather = _C_ops . gather_nd ( weight , valid_label . transpose ( temp_perm ) ) else : weight_gather = _C_ops . gather_nd ( weight , valid_label ) weight_gather = _C_ops . elementwise_mul ( weight_gather , ignore_weight_mask ) input_shape = list ( label . shape ) weight_gather_reshape = reshape ( weight_gather , shape = input_shape ) out = paddle . cast ( out , weight_gather_reshape . dtype ) out = _C_ops . elementwise_mul ( out , weight_gather_reshape ) if reduction == "sum" : #   because of fluid_softmax_with_cross_entropy op's inner logic, #   in the out tensor of this op, the loss of sample with class_index==ignore_index is 0 #   so, reduce_sum all directly is ok return _C_ops . reduce_sum ( out , 'reduce_all' , True ) elif reduction == "mean" : # 1. if weight==none, #     numerator: reduce_sum all loss directly is ok causeof fluid_softmax_with_cross_entropy's inner logic #     denominator: count sample num with class_index!=ignore_index # 2. else #     numerator: loss's weighted sum #     denominator: cal the sum of weight where the sample's class_index!=ignore_index if ignore_index != - 100 : out_sum = _C_ops . reduce_sum ( out , 'reduce_all' , True ) # for each label[i],set 1 or 0, according to ignore_index # mask[i]=0, if label[i]==ignore_index # mask[i]=1, otherwise mask = ( label != ignore_index ) if weight is None : mask = paddle . cast ( mask , dtype = out_sum . dtype ) count = _C_ops . reduce_sum ( mask , 'reduce_all' , True ) ret = out_sum / ( count + ( count == 0.0 ) ) else : mask = paddle . cast ( mask , weight_gather_reshape . dtype ) weight_ignored = _C_ops . elementwise_mul ( mask , weight_gather_reshape ) weight_sum = _C_ops . reduce_sum ( weight_ignored , 'reduce_all' , True ) ret = out_sum / ( weight_sum + ( weight_sum == 0.0 ) ) return ret elif weight is not None : out_sum = _C_ops . reduce_sum ( out , 'reduce_all' , True ) total_weight = _C_ops . reduce_sum ( weight_gather_reshape , 'reduce_all' , True ) return out_sum / ( total_weight + ( total_weight == 0.0 ) ) else : return _C_ops . mean ( out ) else : if input_dims - 1 == label_dims : out = paddle . squeeze ( out , axis = axis ) return out fluid . data_feeder . check_variable_and_dtype ( input , 'input' , [ 'float32' , 'float64' ] , 'softmax_cross_entropy' ) fluid . data_feeder . check_variable_and_dtype ( label , 'label' , [ 'uint8' , 'int8' , 'int16' , 'int32' , 'int64' , 'float32' , 'float64' ] , 'softmax_cross_entropy' ) attrs = { 'soft_label' : soft_label , 'ignore_index' : ignore_index , 'numeric_stable_mode' : True , 'axis' : axis , 'use_softmax' : use_softmax } helper = LayerHelper ( 'softmax_with_cross_entropy' , * * locals ( ) ) softmax = helper . create_variable_for_type_inference ( dtype = input . dtype ) out = helper . create_variable_for_type_inference ( dtype = input . dtype ) helper . append_op ( type = 'softmax_with_cross_entropy' , inputs = { 'Logits' : input , 'Label' : label } , outputs = { 'Softmax' : softmax , 'Loss' : out } , attrs = attrs ) if weight is not None : fluid . data_feeder . check_variable_and_dtype ( weight , 'weight' , [ 'float32' , 'float64' ] , 'softmax_cross_entropy' ) weight_name = name if reduction == 'none' else None if soft_label == True : # chajchaj: # trans weight from class to sample, shape:N or [N,H,W] for 1d and 2d cases. # weight's shape is C, where C is class num. # for 1d case: label's shape is [N,C], weight_gather's shape is N. # for 2d case: label's shape is [N,H,W,C], weight_gather's shape is [N,H,W]. weight_gather = paddle . matmul ( x = paddle . cast ( label , weight . dtype ) , y = weight , transpose_x = False , transpose_y = True ) out_shape = list ( out . shape ) weight_gather_reshape = reshape ( weight_gather , shape = out_shape ) out = paddle . cast ( out , weight_gather_reshape . dtype ) else : if input . shape [ axis ] != weight . shape [ - 1 ] : raise ValueError ( "input's class_dimension({}) must equal to " "weight's class_dimension({}) " "when weight is provided" . format ( input . shape [ axis ] , weight . shape [ - 1 ] ) ) valid_label = paddle . multiply ( paddle . cast ( label != ignore_index , dtype = label . dtype ) , label ) ignore_weight_mask = paddle . cast ( ( label != ignore_index ) , input . dtype ) if ignore_weight_mask . ndim > 1 and ignore_weight_mask . shape [ axis ] == 1 : ignore_weight_mask = paddle . squeeze ( ignore_weight_mask , axis ) if axis != - 1 and axis != valid_label . ndim - 1 : temp_perm = list ( range ( axis % valid_label . ndim ) ) + list ( range ( ( axis % valid_label . ndim + 1 ) , valid_label . ndim ) ) + [ axis % valid_label . ndim ] weight_gather = paddle . gather_nd ( weight , paddle . transpose ( valid_label , temp_perm ) ) else : weight_gather = paddle . gather_nd ( weight , valid_label ) weight_gather = paddle . multiply ( weight_gather , ignore_weight_mask ) input_shape = list ( label . shape ) weight_gather_reshape = reshape ( weight_gather , shape = input_shape ) out = paddle . multiply ( out , weight_gather_reshape , name = weight_name ) if reduction == "sum" : return paddle . sum ( out , name = name ) elif reduction == "mean" : if ignore_index != - 100 : out_sum = paddle . sum ( out , name = name ) # for each label[i],set 1 or 0, according to ignore_index # mask[i]=0, if label[i]==ignore_index # mask[i]=1, otherwise mask = ( label != ignore_index ) if ( weight is None ) : mask = paddle . cast ( mask , dtype = out_sum . dtype ) count = paddle . sum ( mask , name = name ) ret = out_sum / ( count + ( count == 0.0 ) ) else : mask = paddle . cast ( mask , weight_gather_reshape . dtype ) weight_ignored = paddle . multiply ( mask , weight_gather_reshape ) weight_sum = paddle . sum ( weight_ignored , name = name ) ret = out_sum / ( weight_sum + ( weight_sum == 0.0 ) ) return ret elif weight is not None : out_sum = paddle . sum ( out , name = name ) total_weight = paddle . sum ( weight_gather_reshape ) return out_sum / ( total_weight + ( total_weight == 0.0 ) ) else : return paddle . mean ( out , name = name ) else : if input_dims - 1 == label_dims : out = paddle . squeeze ( out , axis = axis ) return out