nvidia/segformer-b0-finetuned-ade-512-512
Image Segmentation
•
Updated
•
185k
•
151
Cristóbal Alcázar
alkzar90
AI & ML interests
None yet
Recent Activity
liked
a model
about 2 months ago
nvidia/segformer-b0-finetuned-ade-512-512
updated
a dataset
about 2 months ago
alkzar90/ddpm-rl-finetuning-evals
Organizations
alkzar90's activity
liked
a
model
about 2 months ago
updated
a
dataset
about 2 months ago
liked
a
model
4 months ago
updated
6
models
4 months ago
alkzar90/ddpo-aesthetic-celebahq-256
Unconditional Image Generation
•
Updated
•
9
alkzar90/ddpo-incompressibility-celebahq-256
Unconditional Image Generation
•
Updated
•
13
alkzar90/ddpo-incompressibility-church-256
Unconditional Image Generation
•
Updated
•
4
alkzar90/ddpo-aesthetic-church-256
Unconditional Image Generation
•
Updated
•
4
alkzar90/ddpo-over50-celebahq-256
Unconditional Image Generation
•
Updated
•
18
alkzar90/ddpo-compressibility-church-256
Unconditional Image Generation
•
Updated
•
8
Reacted to
merve's
post with 🔥
4 months ago
Post
3313
Chameleon 🦎 by Meta is now available in Hugging Face transformers 😍
A vision language model that comes in 7B and 34B sizes 🤩
But what makes this model so special?
Demo: merve/chameleon-7b
Models: facebook/chameleon-668da9663f80d483b4c61f58
keep reading ⥥
Chameleon is a unique model: it attempts to scale early fusion 🤨
But what is early fusion?
Modern vision language models use a vision encoder with a projection layer to project image embeddings so it can be promptable to text decoder (LLM)
Early fusion on the other hand attempts to fuse all features together (image patches and text) by using an image tokenizer and all tokens are projected into a shared space, which enables seamless generation 😏
Authors have also introduced different architectural improvements (QK norm and revise placement of layer norms) for scalable and stable training and they were able to increase the token count (5x tokens compared to Llama 3 which is a must with early-fusion IMO)
This model is an any-to-any model thanks to early fusion: it can take image and text input and output image and text, but image generation are disabled to prevent malicious use.
One can also do text-only prompting, authors noted the model catches up with larger LLMs (like Mixtral 8x7B or larger Llama-2 70B) and also image-pair prompting with larger VLMs like IDEFICS2-80B (see paper for the benchmarks Chameleon: Mixed-Modal Early-Fusion Foundation Models (2405.09818))
Thanks for reading!
A vision language model that comes in 7B and 34B sizes 🤩
But what makes this model so special?
Demo: merve/chameleon-7b
Models: facebook/chameleon-668da9663f80d483b4c61f58
keep reading ⥥
Chameleon is a unique model: it attempts to scale early fusion 🤨
But what is early fusion?
Modern vision language models use a vision encoder with a projection layer to project image embeddings so it can be promptable to text decoder (LLM)
Early fusion on the other hand attempts to fuse all features together (image patches and text) by using an image tokenizer and all tokens are projected into a shared space, which enables seamless generation 😏
Authors have also introduced different architectural improvements (QK norm and revise placement of layer norms) for scalable and stable training and they were able to increase the token count (5x tokens compared to Llama 3 which is a must with early-fusion IMO)
This model is an any-to-any model thanks to early fusion: it can take image and text input and output image and text, but image generation are disabled to prevent malicious use.
One can also do text-only prompting, authors noted the model catches up with larger LLMs (like Mixtral 8x7B or larger Llama-2 70B) and also image-pair prompting with larger VLMs like IDEFICS2-80B (see paper for the benchmarks Chameleon: Mixed-Modal Early-Fusion Foundation Models (2405.09818))
Thanks for reading!
- 8 replies
Reacted to
merve's
post with 🔥
5 months ago
Post
3276
Just shipped: introduction to vision language models (aka image-text-to-text) https://huggingface.co/tasks/image-text-to-text
Learn about more machine learning tasks at https://huggingface.co/tasks
Learn about more machine learning tasks at https://huggingface.co/tasks
updated
a
model
6 months ago
liked
2
models
8 months ago
updated
a
Space
11 months ago
liked
a
model
about 1 year ago
upvoted
a
paper
about 1 year ago
updated
a
dataset
over 1 year ago
liked
2
models
over 1 year ago
