guozinan/PuLID
Updated
โข
69
tazz tone
tazztone
AI & ML interests
None yet
Recent Activity
liked
a model
10 days ago
guozinan/PuLID
New activity
20 days ago
city96/t5-v1_1-xxl-encoder-gguf:how do I use this can't load t5 gguf with clip l safetensor
liked
a model
about 1 month ago
stabilityai/stable-diffusion-3.5-large
Organizations
None yet
tazztone's activity
liked
a
model
10 days ago
how do I use this can't load t5 gguf with clip l safetensor
11
#6 opened 3 months ago
by
MANOFAi94
MANOFAi94
liked
a
model
about 1 month ago
liked
a
model
about 2 months ago
updated
2
Spaces
about 2 months ago
liked
a
Space
about 2 months ago
updated
a
Space
about 2 months ago
upvoted
an
article
about 2 months ago
Article
"Diffusers Image Fill" guide
By
โข
โข
38
Reacted to
m-ric's
post with ๐
about 2 months ago
Post
3214
๐ ๐๐ก๐ ๐๐ข๐ซ๐ฌ๐ญ ๐๐ฏ๐๐ซ ๐
๐จ๐ฎ๐ง๐๐๐ญ๐ข๐จ๐ง ๐ฐ๐๐๐ญ๐ก๐๐ซ ๐ฆ๐จ๐๐๐ฅ: ๐๐ซ๐ข๐ญ๐ก๐ฏ๐ข ๐๐ฑ๐ ๐๐ง๐๐๐ฅ๐๐ฌ ๐ฅ๐ข๐๐-๐ฌ๐๐ฏ๐ข๐ง๐ ๐ฐ๐๐๐ญ๐ก๐๐ซ ๐ฉ๐ซ๐๐๐ข๐๐ญ๐ข๐จ๐ง๐ฌ
Hurricane Katrina killed hundreds of people as it made landfall on New Orleans in 2005 - many of these deaths could have been avoided if alerts had been given one day earlier. Accurate weather forecasts are really life-saving.
๐ฅย Now, NASA and IBM just dropped a game-changing new model: the first ever foundation model for weather! This means, it's the first time we have a generalist model not restricted to one task, but able to predict 160 weather variables!
Prithvi WxC (Prithvi, โเคชเฅเคฅเฅเคตเฅโ, is the Sanskrit name for Earth) - is a 2.3 billion parameter model, with an architecture close to previous vision transformers like Hiera.
๐กย But it comes with some important tweaks: under the hood, Prithvi WxC uses a clever transformer-based architecture with 25 encoder and 5 decoder blocks. It alternates between "local" and "global" attention to capture both regional and global weather patterns.
๐๐ฒ๐ ๐ถ๐ป๐๐ถ๐ด๐ต๐๐:
๐ฎ Nails short-term forecasts - Prithvi WxC crushed it on 6-12 hour predictions, even outperforming some traditional numerical weather models
๐ Tracks hurricanes like a champ - For Hurricane Ida, it predicted the landfall location within 5 km (vs 20+ km errors from other AI models), which is a huge progress!
๐ 6x downscaling power - Can zoom in on weather data to 6x higher resolution with 4x lower error than basic methods
๐ Models elusive gravity waves - Accurately simulates these crucial but hard-to-capture atmospheric oscillations
As climate change intensifies, tools like Prithvi WxC will become more and more crucial to avoid disasters!
Announcement post ๐ https://newsroom.ibm.com/2024-09-23-ibm-and-nasa-release-open-source-ai-model-on-hugging-face-for-weather-and-climate-applications
Model on the Hub ๐ https://huggingface.co/Prithvi-WxC
Thank you @clem for highlighting it!
Hurricane Katrina killed hundreds of people as it made landfall on New Orleans in 2005 - many of these deaths could have been avoided if alerts had been given one day earlier. Accurate weather forecasts are really life-saving.
๐ฅย Now, NASA and IBM just dropped a game-changing new model: the first ever foundation model for weather! This means, it's the first time we have a generalist model not restricted to one task, but able to predict 160 weather variables!
Prithvi WxC (Prithvi, โเคชเฅเคฅเฅเคตเฅโ, is the Sanskrit name for Earth) - is a 2.3 billion parameter model, with an architecture close to previous vision transformers like Hiera.
๐กย But it comes with some important tweaks: under the hood, Prithvi WxC uses a clever transformer-based architecture with 25 encoder and 5 decoder blocks. It alternates between "local" and "global" attention to capture both regional and global weather patterns.
๐๐ฒ๐ ๐ถ๐ป๐๐ถ๐ด๐ต๐๐:
๐ฎ Nails short-term forecasts - Prithvi WxC crushed it on 6-12 hour predictions, even outperforming some traditional numerical weather models
๐ Tracks hurricanes like a champ - For Hurricane Ida, it predicted the landfall location within 5 km (vs 20+ km errors from other AI models), which is a huge progress!
๐ 6x downscaling power - Can zoom in on weather data to 6x higher resolution with 4x lower error than basic methods
๐ Models elusive gravity waves - Accurately simulates these crucial but hard-to-capture atmospheric oscillations
As climate change intensifies, tools like Prithvi WxC will become more and more crucial to avoid disasters!
Announcement post ๐ https://newsroom.ibm.com/2024-09-23-ibm-and-nasa-release-open-source-ai-model-on-hugging-face-for-weather-and-climate-applications
Model on the Hub ๐ https://huggingface.co/Prithvi-WxC
Thank you @clem for highlighting it!
How to use this in Kohya?
9
#1 opened 3 months ago
by
GeneralAwareness
GeneralAwareness
liked
3
models
2 months ago
liked
3
Spaces
2 months ago
liked
a
dataset
2 months ago
replied to
MonsterMMORPG's
post
2 months ago
do the finetunes still make other (background) people look like the trained character, as it does with LoRAs?
Reacted to
MonsterMMORPG's
post with ๐
2 months ago
Post
3304
Full Fine Tuning of FLUX yields way better results than LoRA training as expected, overfitting and bleeding reduced a lot
Configs and Full Experiments
Full configs and grid files shared here : https://www.patreon.com/posts/kohya-flux-fine-112099700
Details
I am still rigorously testing different hyperparameters and comparing impact of each one to find the best workflow
So far done 16 different full trainings and completing 8 more at the moment
I am using my poor overfit 15 images dataset for experimentation (4th image)
I have already proven that when I use a better dataset it becomes many times betters and generate expressions perfectly
Here example case : https://www.reddit.com/r/FluxAI/comments/1ffz9uc/tried_expressions_with_flux_lora_training_with_my/
Conclusions
When the results are analyzed, Fine Tuning is way lesser overfit and more generalized and better quality
In first 2 images, it is able to change hair color and add beard much better, means lesser overfit
In the third image, you will notice that the armor is much better, thus lesser overfit
I noticed that the environment and clothings are much lesser overfit and better quality
Disadvantages
Kohya still doesnโt have FP8 training, thus 24 GB GPUs gets a huge speed drop
Moreover, 48 GB GPUs has to use Fused Back Pass optimization, thus have some speed drop
16 GB GPUs gets way more aggressive speed drop due to lack of FP8
Clip-L and T5 trainings still not supported
Speeds
Rank 1 Fast Config โ uses 27.5 GB VRAM, 6.28 second / it (LoRA is 4.85 second / it)
Rank 1 Slower Config โ uses 23.1 GB VRAM, 14.12 second / it (LoRA is 4.85 second / it)
Rank 1 Slowest Config โ uses 15.5 GB VRAM, 39 second / it (LoRA is 6.05 second / it)
Final Info
Saved checkpoints are FP16 and thus 23.8 GB (no Clip-L or T5 trained)
According to the Kohya, applied optimizations doesnโt change quality so all configs are ranked as Rank 1 at the moment
I am still testing whether these optimizations make any impact on quality or not
Configs and Full Experiments
Full configs and grid files shared here : https://www.patreon.com/posts/kohya-flux-fine-112099700
Details
I am still rigorously testing different hyperparameters and comparing impact of each one to find the best workflow
So far done 16 different full trainings and completing 8 more at the moment
I am using my poor overfit 15 images dataset for experimentation (4th image)
I have already proven that when I use a better dataset it becomes many times betters and generate expressions perfectly
Here example case : https://www.reddit.com/r/FluxAI/comments/1ffz9uc/tried_expressions_with_flux_lora_training_with_my/
Conclusions
When the results are analyzed, Fine Tuning is way lesser overfit and more generalized and better quality
In first 2 images, it is able to change hair color and add beard much better, means lesser overfit
In the third image, you will notice that the armor is much better, thus lesser overfit
I noticed that the environment and clothings are much lesser overfit and better quality
Disadvantages
Kohya still doesnโt have FP8 training, thus 24 GB GPUs gets a huge speed drop
Moreover, 48 GB GPUs has to use Fused Back Pass optimization, thus have some speed drop
16 GB GPUs gets way more aggressive speed drop due to lack of FP8
Clip-L and T5 trainings still not supported
Speeds
Rank 1 Fast Config โ uses 27.5 GB VRAM, 6.28 second / it (LoRA is 4.85 second / it)
Rank 1 Slower Config โ uses 23.1 GB VRAM, 14.12 second / it (LoRA is 4.85 second / it)
Rank 1 Slowest Config โ uses 15.5 GB VRAM, 39 second / it (LoRA is 6.05 second / it)
Final Info
Saved checkpoints are FP16 and thus 23.8 GB (no Clip-L or T5 trained)
According to the Kohya, applied optimizations doesnโt change quality so all configs are ranked as Rank 1 at the moment
I am still testing whether these optimizations make any impact on quality or not
-
- 2 replies