Q-Bench (ICLR24' Spotlight), Q-Bench-Pair (TPAMI), and A-Bench in HuggingFace Format. Support auto-load as `dataset = load_dataset("q-future/**-HF")`
Q-Future
community
AI & ML interests
None defined yet.
Organization Card
Our Github Page: https://github.com/Q-Future
Please use the HF versions for the benchmark datasets by Q-Future.
from datasets import load_dataset
ds = load_dataset("q-future/Q-Bench-HF") # or A-Bench-HF, Q-Bench2-HF
ds["dev"][0] # Containing images (in PIL.ImageFile), questions, and answers
Our Spaces
Great thanks to the research GPU grants!
- Q-Align (Most Powerful Visual Scorer):
- Q-Instruct (Low-level Vision-Language Assistant/Chatbot, support 1-4 images):
- Q-Bench (Benchmark for General Purpose MLLMs):
Our Mainstream Models
q-future/one-align
: AutoModel for Visual Scoring. Trained with Mixture of existing datasets: See Github for details.q-future/co-instruct
: AutoModel for Low-level Visual Dialog (Description, Comparison, Question Answering). Trained with the scaled Co-Instruct-562K dataset (will also release soon!).q-future/q-instruct-mplug-owl2-1031
: Older version of Q-Instruct, as reported by paper. Trained with released Q-Instruct-200K dataset.
Though we have other model variants released for the community to replicate our results, please use the previous ones as they are proved to have more stable performance.
models
26
q-future/Compare2Score
Feature Extraction
•
Updated
•
236
•
3
q-future/one-align
Zero-Shot Image Classification
•
Updated
•
18k
•
22
q-future/q-align-only-pair
Text Generation
•
Updated
•
4
q-future/q-align-single-pair
Text Generation
•
Updated
•
2
q-future/t2i-scorer
Text Generation
•
Updated
•
4
q-future/t2i-scorer-ft
Updated
q-future/t2i-scorer-it
Text Generation
•
Updated
•
4
q-future/q-align-sc-itu
Text Generation
•
Updated
•
2
q-future/q-align-sc-fss
Text Generation
•
Updated
•
5
q-future/q-align-ft-ytugc
Feature Extraction
•
Updated
•
2
datasets
9
q-future/A-Bench-HF
Viewer
•
Updated
•
2.86k
•
282
•
2
q-future/Q-Bench2-HF
Viewer
•
Updated
•
2k
•
104
•
1
q-future/Q-Bench-HF
Viewer
•
Updated
•
2.99k
•
170
•
1
q-future/A-Bench
Viewer
•
Updated
•
2.86k
•
41
q-future/ytugc-backup-mp4
Updated
•
54
q-future/Co-Instruct-DB
Updated
•
84
•
4
q-future/q-bench2
Viewer
•
Updated
•
2.14k
•
68
•
3
q-future/q-align-datasets
Preview
•
Updated
•
126
•
2
q-future/Q-Instruct-DB
Preview
•
Updated
•
112
•
16