update
Browse files
README.md
CHANGED
@@ -5,9 +5,9 @@ license: mit
|
|
5 |
|
6 |
# Model Summary
|
7 |
OmniParser is a general screen parsing tool, which interprets/converts UI screenshot to structured format, to improve existing LLM based UI agent.
|
8 |
-
Training Datasets include: 1) an interactable icon detection dataset, which was curated from
|
9 |
|
10 |
-
|
11 |
|
12 |
# Responsible AI Considerations
|
13 |
## Intended Use
|
|
|
5 |
|
6 |
# Model Summary
|
7 |
OmniParser is a general screen parsing tool, which interprets/converts UI screenshot to structured format, to improve existing LLM based UI agent.
|
8 |
+
Training Datasets include: 1) an interactable icon detection dataset, which was curated from popular web pages and automatically annotated to highlight clickable and actionable regions, and 2) an icon description dataset, designed to associate each UI element with its corresponding function.
|
9 |
|
10 |
+
This model hub includes a finetuned version of YOLOv8 and a finetuned BLIP-2 model on the above dataset respectively. For more details of the models used and finetuning, please refer to the [paper](https://arxiv.org/abs/2408.00203).
|
11 |
|
12 |
# Responsible AI Considerations
|
13 |
## Intended Use
|