03/06 17:37:43 - OpenCompass - INFO - Task [my_api/siqa]
/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/site-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
03/06 17:37:51 - OpenCompass - INFO - Start inferencing [my_api/siqa]
[2024-03-06 17:37:51,870] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...

  0%|          | 0/245 [00:00<?, ?it/s]
  0%|          | 0/245 [00:09<?, ?it/s]
Traceback (most recent call last):
  File "/export/home/tanwentao1/project2/opencompass/opencompass/tasks/openicl_infer.py", line 153, in <module>
    inferencer.run()
  File "/export/home/tanwentao1/project2/opencompass/opencompass/tasks/openicl_infer.py", line 81, in run
    self._inference()
  File "/export/home/tanwentao1/project2/opencompass/opencompass/tasks/openicl_infer.py", line 126, in _inference
    inferencer.inference(retriever,
  File "/export/home/tanwentao1/project2/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 146, in inference
    results = self.model.generate_from_template(
  File "/export/home/tanwentao1/project2/opencompass/opencompass/models/base.py", line 165, in generate_from_template
    return self.generate(inputs, max_out_len=max_out_len, **kwargs)
  File "/export/home/tanwentao1/project2/opencompass/opencompass/models/my_api.py", line 79, in generate
    results = list(
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/export/home/tanwentao1/project2/opencompass/opencompass/models/my_api.py", line 151, in _generate
    msg = response['choices'][0]['message']['content']
TypeError: string indices must be integers
[2024-03-06 17:38:06,766] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 269672) of binary: /export/home/tanwentao1/anaconda3/envs/opencompass/bin/python
Traceback (most recent call last):
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 347, in wrapper
    return f(*args, **kwargs)
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 812, in main
    run(args)
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 803, in run
    elastic_launch(
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 135, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/export/home/tanwentao1/anaconda3/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 268, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
/export/home/tanwentao1/project2/opencompass/opencompass/tasks/openicl_infer.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-03-06_17:38:06
  host      : v100-4-11-71-1-176
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 269672)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================