Spaces:
Runtime error
Runtime error
File size: 1,790 Bytes
9094cc2 5111f27 675f890 9094cc2 31b9ddb 9094cc2 a92816b 9094cc2 9ef3bbd a92816b e6fac54 a92816b 969d45a a92816b e6fac54 a92816b e6fac54 85a8429 e6fac54 a92816b e6fac54 449b6ea 580b4e4 969d45a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
---
title: Model Evaluator
emoji: 📊
colorFrom: red
colorTo: red
sdk: streamlit
sdk_version: 1.10.0
app_file: app.py
---
# Model Evaluator
> Submit evaluation jobs to AutoTrain from the Hugging Face Hub
## Supported tasks
The table below shows which tasks are currently supported for evaluation in the AutoTrain backend:
| Task | Supported |
|:-----------------------------------|:---------:|
| `binary_classification` | ✅ |
| `multi_class_classification` | ✅ |
| `multi_label_classification` | ❌ |
| `entity_extraction` | ✅ |
| `extractive_question_answering` | ✅ |
| `translation` | ✅ |
| `summarization` | ✅ |
| `image_binary_classification` | ✅ |
| `image_multi_class_classification` | ✅ |
| `text_zero_shot_evaluation` | ✅ |
## Installation
To run the application locally, first clone this repository and install the dependencies as follows:
```
pip install -r requirements.txt
```
Next, copy the example file of environment variables:
```
cp .env.template .env
```
and set the `HF_TOKEN` variable with a valid API token from the `autoevaluator` user. Finally, spin up the application by running:
```
streamlit run app.py
```
## AutoTrain configuration details
Models are evaluated by AutoTrain, with the payload sent to the `AUTOTRAIN_BACKEND_API` environment variable. The current configuration for evaluation jobs running on Spaces is:
```
AUTOTRAIN_BACKEND_API=https://api.autotrain.huggingface.co
```
To evaluate models with a _local_ instance of AutoTrain, change the environment to:
```
AUTOTRAIN_BACKEND_API=http://localhost:8000
```
|