parameters: (program, config, default_device="cpu")
return_statement: return trainer_program
docstring: split cpu-trainer program from origin-program
      1. find heter op (located on different device)
      2. find input&output of every heter-block
      3. create cpu-trainer program, add send&recv op
docstring_summary: split cpu-trainer program from origin-program
      1. find heter op (located on different device)
      2. find input&output of every heter-block
      3. create cpu-trainer program, add send&recv op
docstring_tokens: split cpu - trainer program from origin - program 1 . find heter op ( located on different device ) 2 . find input&output of every heter - block 3 . create cpu - trainer program add send&recv op
function: def split_trainer_ops_pass(program, config, default_device="cpu"):
      """
      split cpu-trainer program from origin-program
      1. find heter op (located on different device)
      2. find input&output of every heter-block
      3. create cpu-trainer program, add send&recv op 
      """
      # Todo: support user define default_device (MrChengmo)
      default_device_ = default_device
      program, heter_ops, default_ops, program_block_ops = find_heter_ops(
          program, default_device_)
      program_block_ops = union_forward_gradient_op(program_block_ops)
  
      block_vars_detail = find_block_joints(program, program_block_ops, heter_ops)
      trainer_program = program.clone()
      create_trainer_program(trainer_program, program, config, program_block_ops,
                             block_vars_detail)
      return trainer_program
function_tokens: def split_trainer_ops_pass ( program , config , default_device = "cpu" ) : # Todo: support user define default_device (MrChengmo) default_device_ = default_device program , heter_ops , default_ops , program_block_ops = find_heter_ops ( program , default_device_ ) program_block_ops = union_forward_gradient_op ( program_block_ops ) block_vars_detail = find_block_joints ( program , program_block_ops , heter_ops ) trainer_program = program . clone ( ) create_trainer_program ( trainer_program , program , config , program_block_ops , block_vars_detail ) return trainer_program