parameters: (output_tensor, input_tensor, group=None, async_op=False)
docstring: Single tensor all gather. Gathers a single tensor from all ranks, and puts them in a single output tensor.
  
      Args:
          output_tensor (Tensor): Output tensor. It should contain
              correctly-sized tensors to be used for output of the collective.
          input_tensor (Tensor): Tensor to be broadcast from current process.
          group (ProcessGroup, optional): The process group to work on. If None,
              the default process group will be used.
          async_op (bool, optional): Whether this op should be an async op
  
      Returns:
          Async work handle, if async_op is set to True.
          None, if not async_op or if not part of the group
  
      Examples:
          >>> # All tensors below are of torch.int64 dtype.
          >>> # We have 2 process groups, 2 ranks.
          >>> output_tensor = torch.zeros(2, dtype=torch.int64)
          >>> output_tensor
          [tensor([0, 0])] # Rank 0 and 1
          >>> tensor = torch.arange(1, dtype=torch.int64) + 1 + rank
          >>> tensor
          tensor([1]) # Rank 0
          tensor([2]) # Rank 1
          >>> dist.all_gather_base(output_tensor, tensor)
          >>> output_tensor
          tensor([1,2]) # Rank 0
          tensor([1,2]) # Rank 1
  
      .. warning::
          `_all_gather_base` is experimental and subject to change.
          It is the caller's responsibility to ensure the output_tensor
          is correctly sized.
docstring_summary: Single tensor all gather. Gathers a single tensor from all ranks, and puts them in a single output tensor.
docstring_tokens: Single tensor all gather . Gathers a single tensor from all ranks and puts them in a single output tensor .
function: def _all_gather_base(output_tensor, input_tensor, group=None, async_op=False):
      """
      Single tensor all gather. Gathers a single tensor from all ranks, and puts them in a single output tensor.
  
      Args:
          output_tensor (Tensor): Output tensor. It should contain
              correctly-sized tensors to be used for output of the collective.
          input_tensor (Tensor): Tensor to be broadcast from current process.
          group (ProcessGroup, optional): The process group to work on. If None,
              the default process group will be used.
          async_op (bool, optional): Whether this op should be an async op
  
      Returns:
          Async work handle, if async_op is set to True.
          None, if not async_op or if not part of the group
  
      Examples:
          >>> # All tensors below are of torch.int64 dtype.
          >>> # We have 2 process groups, 2 ranks.
          >>> output_tensor = torch.zeros(2, dtype=torch.int64)
          >>> output_tensor
          [tensor([0, 0])] # Rank 0 and 1
          >>> tensor = torch.arange(1, dtype=torch.int64) + 1 + rank
          >>> tensor
          tensor([1]) # Rank 0
          tensor([2]) # Rank 1
          >>> dist.all_gather_base(output_tensor, tensor)
          >>> output_tensor
          tensor([1,2]) # Rank 0
          tensor([1,2]) # Rank 1
  
      .. warning::
          `_all_gather_base` is experimental and subject to change.
          It is the caller's responsibility to ensure the output_tensor
          is correctly sized.
  
      """
      _check_single_tensor(input_tensor, "input_tensor")
      _check_single_tensor(output_tensor, "output_tensor")
      if _rank_not_in_group(group):
          _warn_not_in_group("_all_gather_base")
          return
  
      output_tensor = (
          output_tensor
          if not output_tensor.is_complex()
          else torch.view_as_real(output_tensor)
      )
      input_tensor = (
          input_tensor
          if not input_tensor.is_complex()
          else torch.view_as_real(input_tensor)
      )
  
      if group is None:
          default_pg = _get_default_group()
          work = default_pg._allgather_base(output_tensor, input_tensor)
      else:
          work = group._allgather_base(output_tensor, input_tensor)
  
      if async_op:
          return work
      else:
          work.wait()
function_tokens: def _all_gather_base ( output_tensor , input_tensor , group = None , async_op = False ) : _check_single_tensor ( input_tensor , "input_tensor" ) _check_single_tensor ( output_tensor , "output_tensor" ) if _rank_not_in_group ( group ) : _warn_not_in_group ( "_all_gather_base" ) return output_tensor = ( output_tensor if not output_tensor . is_complex ( ) else torch . view_as_real ( output_tensor ) ) input_tensor = ( input_tensor if not input_tensor . is_complex ( ) else torch . view_as_real ( input_tensor ) ) if group is None : default_pg = _get_default_group ( ) work = default_pg . _allgather_base ( output_tensor , input_tensor ) else : work = group . _allgather_base ( output_tensor , input_tensor ) if async_op : return work else : work . wait ( )