parameters: (self, use_symbolic_shape_infer=True, **kwargs)
docstring: Convert a model to half (default) or mixed precision.
             To use mixed precision, user need specify which graph inputs, outputs, operator type or list of nodes shall keep in float32.
             By default, we use symbolic shape inference to get shape and type information. If not, ONNX shape inference will be used.
             Note that symbolic/ONNX shape inference might fail, and the conversion might not proceed without shape and type information.
  
          Args:
              use_symbolic_shape_infer (bool, optional): use symbolic shape inference instead of onnx shape inference. Defaults to True.
              keep_io_types (Union[bool, List[str]], optional): It could be boolean or a list of float32 input/output names. 
                                                                If True, model inputs/outputs should be left as float32. Defaults to False.
              op_block_list (List[str], optional): List of operator types to leave as float32.
                                                   Defaults to None, which will use `float16.DEFAULT_OP_BLOCK_LIST` as default.
              node_block_list (List[str], optional): List of node names to leave as float32. Defaults to None.
              force_fp16_initializers(bool): force converting all float initializers to float16.
                                             Default to false, which will convert only the one needed to avoid precision loss.
              min_positive_val (float, optional): minimal positive value. Defaults to 1e-7.
              max_finite_val (float, optional): maximal finite value. Defaults to 1e4.
docstring_summary: Convert a model to half (default) or mixed precision.
             To use mixed precision, user need specify which graph inputs, outputs, operator type or list of nodes shall keep in float32.
             By default, we use symbolic shape inference to get shape and type information. If not, ONNX shape inference will be used.
             Note that symbolic/ONNX shape inference might fail, and the conversion might not proceed without shape and type information.
docstring_tokens: Convert a model to half ( default ) or mixed precision . To use mixed precision user need specify which graph inputs outputs operator type or list of nodes shall keep in float32 . By default we use symbolic shape inference to get shape and type information . If not ONNX shape inference will be used . Note that symbolic / ONNX shape inference might fail and the conversion might not proceed without shape and type information .
function: def convert_float_to_float16(self, use_symbolic_shape_infer=True, **kwargs):
          """Convert a model to half (default) or mixed precision.
             To use mixed precision, user need specify which graph inputs, outputs, operator type or list of nodes shall keep in float32.
             By default, we use symbolic shape inference to get shape and type information. If not, ONNX shape inference will be used.
             Note that symbolic/ONNX shape inference might fail, and the conversion might not proceed without shape and type information.
  
          Args:
              use_symbolic_shape_infer (bool, optional): use symbolic shape inference instead of onnx shape inference. Defaults to True.
              keep_io_types (Union[bool, List[str]], optional): It could be boolean or a list of float32 input/output names. 
                                                                If True, model inputs/outputs should be left as float32. Defaults to False.
              op_block_list (List[str], optional): List of operator types to leave as float32.
                                                   Defaults to None, which will use `float16.DEFAULT_OP_BLOCK_LIST` as default.
              node_block_list (List[str], optional): List of node names to leave as float32. Defaults to None.
              force_fp16_initializers(bool): force converting all float initializers to float16.
                                             Default to false, which will convert only the one needed to avoid precision loss.
              min_positive_val (float, optional): minimal positive value. Defaults to 1e-7.
              max_finite_val (float, optional): maximal finite value. Defaults to 1e4.
          """
          if "keep_io_types" not in kwargs:
              kwargs["keep_io_types"] = True
  
          def float_to_float16_func():
              # TODO: import from onnxconverter_common when it is stable
              #try:
              #    import onnxconverter_common as oc
              #    from packaging.version import Version
              #    if Version(oc.__version__) > Version("1.9.0"):
              #        from onnxconverter_common.float16 import convert_float_to_float16
              #        return convert_float_to_float16
              #except ImportError:
              #    pass
  
              from float16 import convert_float_to_float16
              return convert_float_to_float16
  
          convert_float_to_float16 = float_to_float16_func()
  
          model = self.model
          if use_symbolic_shape_infer:
              # Use symbolic shape inference since custom operators (like Gelu, SkipLayerNormalization etc) are not recognized by onnx shape inference.
              shape_infer_helper = SymbolicShapeInferenceHelper(model)
              model = shape_infer_helper.infer_shapes(model, auto_merge=True, guess_output_rank=False)
  
          parameters = {'disable_shape_infer': use_symbolic_shape_infer}
          parameters.update({
              key: kwargs[key]
              for key in [
                  'keep_io_types', 'min_positive_val', 'max_finite_val', 'op_block_list', 'node_block_list',
                  'force_fp16_initializers'
              ] if key in kwargs
          })
  
          fp16_model = convert_float_to_float16(model, **parameters)
          self.initialize(fp16_model)
  
          # Convert_float_to_float16 might add Cast(to=10) --> Cast(to=1) when two consequent nodes are computed in FP32.
          # Below are post-processing that removes those Cast nodes.
          # Remove first Cast nodes in path like  --> Cast --> Cast -->
          nodes_to_remove = []
          for node in self.nodes():
              if node.op_type == "Cast":
                  parent = self.get_parent(node, 0)
                  if parent and parent.op_type == "Cast":
                      if self.get_children(parent) == 1:  # cannot be removed if its output is used by multiple nodes
                          self.replace_input_of_all_nodes(parent.output[0], parent.input[0])
                          nodes_to_remove.append(parent)
  
          # Remove the second cast node.
          for node in self.nodes():
              if node.op_type == "Cast" and OnnxModel.get_node_attribute(node, "to") == int(TensorProto.FLOAT) and \
                  self.get_dtype(node.input[0])  == int(TensorProto.FLOAT):
  
                  if self.find_graph_output(node.output[0]):
                      self.replace_output_of_all_nodes(node.input[0], node.output[0])
                  else:
                      self.replace_input_of_all_nodes(node.output[0], node.input[0])
                  nodes_to_remove.append(node)
  
          self.remove_nodes(nodes_to_remove)
  
          if nodes_to_remove:
              self.prune_graph()
              print(f"removed {len(nodes_to_remove)} Cast nodes from float16 model")
function_tokens: def convert_float_to_float16 ( self , use_symbolic_shape_infer = True , * * kwargs ) : if "keep_io_types" not in kwargs : kwargs [ "keep_io_types" ] = True def float_to_float16_func ( ) : # TODO: import from onnxconverter_common when it is stable #try: #    import onnxconverter_common as oc #    from packaging.version import Version #    if Version(oc.__version__) > Version("1.9.0"): #        from onnxconverter_common.float16 import convert_float_to_float16 #        return convert_float_to_float16 #except ImportError: #    pass from float16 import convert_float_to_float16 return convert_float_to_float16 convert_float_to_float16 = float_to_float16_func ( ) model = self . model if use_symbolic_shape_infer : # Use symbolic shape inference since custom operators (like Gelu, SkipLayerNormalization etc) are not recognized by onnx shape inference. shape_infer_helper = SymbolicShapeInferenceHelper ( model ) model = shape_infer_helper . infer_shapes ( model , auto_merge = True , guess_output_rank = False ) parameters = { 'disable_shape_infer' : use_symbolic_shape_infer } parameters . update ( { key : kwargs [ key ] for key in [ 'keep_io_types' , 'min_positive_val' , 'max_finite_val' , 'op_block_list' , 'node_block_list' , 'force_fp16_initializers' ] if key in kwargs } ) fp16_model = convert_float_to_float16 ( model , * * parameters ) self . initialize ( fp16_model ) # Convert_float_to_float16 might add Cast(to=10) --> Cast(to=1) when two consequent nodes are computed in FP32. # Below are post-processing that removes those Cast nodes. # Remove first Cast nodes in path like  --> Cast --> Cast --> nodes_to_remove = [ ] for node in self . nodes ( ) : if node . op_type == "Cast" : parent = self . get_parent ( node , 0 ) if parent and parent . op_type == "Cast" : if self . get_children ( parent ) == 1 : # cannot be removed if its output is used by multiple nodes self . replace_input_of_all_nodes ( parent . output [ 0 ] , parent . input [ 0 ] ) nodes_to_remove . append ( parent ) # Remove the second cast node. for node in self . nodes ( ) : if node . op_type == "Cast" and OnnxModel . get_node_attribute ( node , "to" ) == int ( TensorProto . FLOAT ) and self . get_dtype ( node . input [ 0 ] ) == int ( TensorProto . FLOAT ) : if self . find_graph_output ( node . output [ 0 ] ) : self . replace_output_of_all_nodes ( node . input [ 0 ] , node . output [ 0 ] ) else : self . replace_input_of_all_nodes ( node . output [ 0 ] , node . input [ 0 ] ) nodes_to_remove . append ( node ) self . remove_nodes ( nodes_to_remove ) if nodes_to_remove : self . prune_graph ( ) print ( f"removed {len(nodes_to_remove)} Cast nodes from float16 model" )