zero-gpu-explorers/README · CUDA must not be initialized in the main process on Spaces with Stateless GPU environment.

Jun 11

I have tried multiple times to work around this error of "CUDA must not be initialized in the main process on Spaces with Stateless GPU environment." What I did:

carefully made sure that in the main thread, there is no GPU call before a single .to(device) call
surrounded the part needing GPU with the @spaces.GPU decorator

What makes it even harder to debug is that the log above isn't complete. Can you advise what possibly could go wrong? The code is publically available at: https://huggingface.co/spaces/jedyang97/3D-GRAND/tree/main. Thanks!

Balaji23

Jul 23

How did you resolve this, i see your space is up and running ? @jedyang97

ZennyKenny

ZeroGPU Explorers org Jul 23

How did you resolve this, i see your space is up and running ? @jedyang97

Looks like this commit resolved the issue: https://huggingface.co/spaces/jedyang97/3D-GRAND/commit/ff53d7316339e1946d89e72b8cfb7a39010a06d0

tokenizer, model, data_loader = load_model_and_dataloader(
    model_path=model_path,
    model_base=model_base,
    load_8bit=load_8bit,
    load_4bit=load_4bit,
    load_bf16=load_bf16,
    scene_to_obj_mapping=scene_to_obj_mapping,
    device_map='cpu',
)  # Huggingface Zero-GPU has to use .to(device) to set the device, otherwise it will fail