stanrom
/

ShareCaptioner

Feature Extraction

InternLMXComposer

Model card Files Files and versions Community

stanrom commited on May 19

Commit

66ceca3

•

1 Parent(s): 7bbe75c

to(torch.float16)

Files changed (1) hide show

modeling_InternLM_XComposer.py +1 -1

modeling_InternLM_XComposer.py CHANGED Viewed

@@ -66,7 +66,7 @@ class InternLMXComposerForCausalLM(PreTrainedModel):
             assert int(torch.__version__[0]) == 2
             # speed up init llm
             with torch.device('meta'):
-                self.internlm_model = InternLMForCausalLM._from_config(config)
             # self.internlm_model.to_empty(device=config.device).to(torch.float16)
             self.internlm_model.tie_weights()
             # self.internlm_model.to(config.device)

             assert int(torch.__version__[0]) == 2
             # speed up init llm
             with torch.device('meta'):
+                self.internlm_model = InternLMForCausalLM._from_config(config).to(torch.float16)
             # self.internlm_model.to_empty(device=config.device).to(torch.float16)
             self.internlm_model.tie_weights()
             # self.internlm_model.to(config.device)