Update README.md
Browse files
README.md
CHANGED
@@ -7,9 +7,9 @@ license: apache-2.0
|
|
7 |
# FineWeb-Edu classifier
|
8 |
|
9 |
## Model summary
|
10 |
-
This is a classifier for judging the educational value of web pages. It was developed to filter and curate educational content from web datasets and was trained on
|
11 |
|
12 |
-
|
13 |
### How to use in transformers
|
14 |
To load the FineWeb-Edu classifier, use the following code:
|
15 |
|
|
|
7 |
# FineWeb-Edu classifier
|
8 |
|
9 |
## Model summary
|
10 |
+
This is a classifier for judging the educational value of web pages. It was developed to filter and curate educational content from web datasets and was trained on 450k annotations generated by [LLama3-70B-instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct) for web samples from [FineWeb](https://huggingface.co/datasets/HuggingFaceFW/fineweb) dataset.
|
11 |
|
12 |
+
We used this classifier to build [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) dataset.
|
13 |
### How to use in transformers
|
14 |
To load the FineWeb-Edu classifier, use the following code:
|
15 |
|