Sygil

/

Sygil-Muse

like 29

Follow

Sygil Dev 5

That code in there is what I am using currently locally and it is working perfectly, make sure you are using the non-ema vae, the ema vae is broken at the moment and we need to fix it, also, make sure the rest of the configuration you are using matches the one used for training the checkpoint, pretty much all the info on what I used for training is on the readme.md and recently I added an option to the code to save a config file next to the checkpoint so I am uploading a yaml file for the new checkpoints that I am currently training, those are a WIP tho and not ready to be used.

thank you. early today i have tried quite a few times, here is my script to run an inference, can you help me find out where is wrong?

BASE=/path/saved
python3 infer_maskgit.py \
  --resume_path=$BASE/Sygil-Muse/maskgit.39000.pt \
  --vae_path=$BASE/Sygil-Muse/vae.74500.pt \
  --num_tokens=8192 \
  --dim=128 \
  --vq_codebook_size=8192 \
  --vq_codebook_dim=4096 \
  --seq_len=1024 \
  --depth=8 \
  --batch_size=1 \
  --cond_drop_prob=0.5 \
  --cond_scale=3.0 \
  --dim_head=64 \
  --ema_beta=0.995 \
  --ff_mult=4 \
  --gradient_accumulation_steps=1 \
  --heads=8 \
  --image_size=256 \
  --logging_dir=results/logs \
  --lr=0.0001 \
  --mixed_precision=no \
  --prompt="A photo of a dog" \
  --results_dir=results \
  --seed=42 \
  --t5_name=$BASE/t5-v1_1-large \
  --timesteps=100 \
  --valid_frac=0.05 

Ah, the infer_maskgit.py script is still not finished, it's not complete, we still haven't been able to fully train a maskgit model that can be usable and generate stuff so we haven't reached the part where we need the script to generate images, the MaskGit checkpoints we have are early on the training stages, they are on the repo for those that want to continue the training or want to help with it, that's why I am sharing on the repo both the VAE and MaskGit, in case someone wants to use them or in case I can't train it myself anymore so the project doesn't die. I am currently doing some changes to the code with the help of the people form the ShoukanLabs team as we are working together to have Muse up and running as soon as we can and we are doing a big refactor of the code to make it faster and easier to read, so, once we finish the changes we are making I will try to go over the infer_maskgit.py script and try to at least have it run without any error even the MaskGit checkpoints are not usable yet.

I got that, thank you a lot. As google published styledrop, i am trying to make an implemetation and found few muse-implemetation. looking for your progress.

I'm here to help, keep an eye on the repo for when we finally get things to a usable state, you will probably know when that is when you check the readme.md file of this repo and you see a bunch of images on it or when you see more info on the readme.md, I usually try to keep it up to date.

ok. it looks code changed a lot but readme not updated, is there more detailed instruction on how-to-training?

Most of what is on the readme.md is still valid, you can just clone the repo and do python setup.py install or pip install . to install it and so it installs the dependencies and stuff needed to run the code, then you can use the train_muse_vae.py and train_muse_maskgit.py scripts as mentioned on the readme.md, what we have added is mostly QOL stuff like extra arguments to have options to customize most of the training parameters, you can just use the --help to see the list of options and read their descriptions, you can also check the training scripts for more info on things. If you want you can join our discord server and message me there on the #model-general or #general-dev channels, you can also send me a DM in discord if you want, I can help you with anything you need to get things running, it will be faster to talk there than here.

ok, joined discord, looking what i can help.