parameters: (self, loss, update_fp32_grads=True)
docstring: fp16_optimizer_obj.backward performs the following conceptual operations:
  
          fp32_loss = loss.float() (see first Note below)
  
          scaled_loss = fp32_loss*loss_scale
  
          scaled_loss.backward(), which accumulates scaled gradients into the .grad attributes of the
          fp16 model's leaves.
  
          fp16 grads are then copied to the stored fp32 params' .grad attributes (see second Note).
  
          Finally, fp32 grads are divided by loss_scale.
  
          In this way, after fp16_optimizer_obj.backward, the fp32 parameters have fresh gradients,
          and fp16_optimizer_obj.step may be called.
  
          .. note::
              Converting the loss to fp32 before applying the loss scale provides some
              additional safety against overflow if the user has supplied an fp16 value.  
              However, for maximum overflow safety, the user should
              compute the loss criterion (MSE, cross entropy, etc) in fp32 before supplying it to 
              fp16_optimizer_obj.backward.
  
          .. note::
              The gradients found in an fp16 model's leaves after a call to 
              fp16_optimizer_obj.backward should not be regarded as valid in general, 
              because it's possible 
              they have been scaled (and in the case of dynamic loss scaling, 
              the scale factor may change over time).  
              If the user wants to inspect gradients after a call to fp16_optimizer_obj.backward,  
              only the master gradients should be regarded as valid, and can be retrieved via
              :attr:`inspect_fp32_grad_data()`.
  
  
          Args:
              loss:  The loss output by the user's model.  loss may be either float or half (but see first Note above).
              update_fp32_grads (bool, optional, default=True):  Option to copy fp16 grads to fp32 grads on this call.  By setting this to False, the user can delay this copy, which is useful to eliminate redundant fp16->fp32 grad copies if fp16_optimizer_obj.backward is being called on multiple losses in one iteration.  If set to False, the user becomes responsible for calling fp16_optimizer_obj.update_fp32_grads before calling fp16_optimizer_obj.step.
  
          Example::
  
              # Ordinary operation:
              optimizer.backward(loss)
  
              # Naive operation with multiple losses (technically valid, but less efficient):
              # fp32 grads will be correct after the second call,  but 
              # the first call incurs an unnecessary fp16->fp32 grad copy.
              optimizer.backward(loss1)
              optimizer.backward(loss2)
  
              # More efficient way to handle multiple losses:
              # The fp16->fp32 grad copy is delayed until fp16 grads from all 
              # losses have been accumulated.
              optimizer.backward(loss1, update_fp32_grads=False)
              optimizer.backward(loss2, update_fp32_grads=False)
              optimizer.update_fp32_grads()
docstring_summary: fp16_optimizer_obj.backward performs the following conceptual operations:
docstring_tokens: fp16_optimizer_obj . backward performs the following conceptual operations :
function: def backward(self, loss, update_fp32_grads=True):
          """ 
          fp16_optimizer_obj.backward performs the following conceptual operations:
  
          fp32_loss = loss.float() (see first Note below)
  
          scaled_loss = fp32_loss*loss_scale
  
          scaled_loss.backward(), which accumulates scaled gradients into the .grad attributes of the
          fp16 model's leaves.
  
          fp16 grads are then copied to the stored fp32 params' .grad attributes (see second Note).
  
          Finally, fp32 grads are divided by loss_scale.
  
          In this way, after fp16_optimizer_obj.backward, the fp32 parameters have fresh gradients,
          and fp16_optimizer_obj.step may be called.
  
          .. note::
              Converting the loss to fp32 before applying the loss scale provides some
              additional safety against overflow if the user has supplied an fp16 value.  
              However, for maximum overflow safety, the user should
              compute the loss criterion (MSE, cross entropy, etc) in fp32 before supplying it to 
              fp16_optimizer_obj.backward.
  
          .. note::
              The gradients found in an fp16 model's leaves after a call to 
              fp16_optimizer_obj.backward should not be regarded as valid in general, 
              because it's possible 
              they have been scaled (and in the case of dynamic loss scaling, 
              the scale factor may change over time).  
              If the user wants to inspect gradients after a call to fp16_optimizer_obj.backward,  
              only the master gradients should be regarded as valid, and can be retrieved via
              :attr:`inspect_fp32_grad_data()`.
  
  
          Args:
              loss:  The loss output by the user's model.  loss may be either float or half (but see first Note above).
              update_fp32_grads (bool, optional, default=True):  Option to copy fp16 grads to fp32 grads on this call.  By setting this to False, the user can delay this copy, which is useful to eliminate redundant fp16->fp32 grad copies if fp16_optimizer_obj.backward is being called on multiple losses in one iteration.  If set to False, the user becomes responsible for calling fp16_optimizer_obj.update_fp32_grads before calling fp16_optimizer_obj.step.
  
          Example::
  
              # Ordinary operation:
              optimizer.backward(loss)
  
              # Naive operation with multiple losses (technically valid, but less efficient):
              # fp32 grads will be correct after the second call,  but 
              # the first call incurs an unnecessary fp16->fp32 grad copy.
              optimizer.backward(loss1)
              optimizer.backward(loss2)
  
              # More efficient way to handle multiple losses:
              # The fp16->fp32 grad copy is delayed until fp16 grads from all 
              # losses have been accumulated.
              optimizer.backward(loss1, update_fp32_grads=False)
              optimizer.backward(loss2, update_fp32_grads=False)
              optimizer.update_fp32_grads()
          """ 
          self.loss_scaler.backward(loss.float())
          if update_fp32_grads:
              self.update_fp32_grads()
function_tokens: def backward ( self , loss , update_fp32_grads = True ) : self . loss_scaler . backward ( loss . float ( ) ) if update_fp32_grads : self . update_fp32_grads ( )