Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
ahmed-masryΒ 
posted an update 20 days ago
Post
1017
πŸš€ Introducing ColFlor: An Efficient, OCR-Free Vision-Language Document Retrieval Model 🌟

Earlier this year, ColPali revolutionized document retrieval by eliminating the need for error-prone OCR pipelines. Instead, it directly processes the document images. However, with its 3 billion parameters, ColPali is computationally heavy for large-scale applications.

That’s where ColFlor comes inβ€”a smaller, faster alternative! πŸŽ‰ At 17x smaller than ColPali, ColFlor offers a more efficient, OCR-free document retrieval solution, making it ideal for users with limited computing resources (GPU Poor). πŸ’‘

Key Highlights:
🧠 174M parameters (vs. 3B for ColPali)
⚑ 9.8x faster query encoding, 5.25x faster image encoding
πŸ“‰ Only 1.8% performance drop on text-rich English documents

Check out the full blog post for more insights on modeling, training, and evaluations across various document retrieval tasks! πŸš€
Also, feel free to try our demo on huggingface πŸ€—

πŸ”— Resources:
πŸ“„ Blog post: https://huggingface.co/blog/ahmed-masry/colflor
🧠 Model: ahmed-masry/ColFlor
πŸ’» Demo: ahmed-masry/ColFlor-Demo
πŸ‹οΈβ€β™‚οΈ Training code: https://github.com/AhmedMasryKU/colflor
πŸ“Š Evaluation code: https://github.com/AhmedMasryKU/vidore-benchmark-colflor
In this post