Update README.md
Browse files
README.md
CHANGED
@@ -56,7 +56,7 @@ We added a classification head with a single regression output to [Snowflake-arc
|
|
56 |
## Limitations
|
57 |
While the FineWeb-Edu classifier performs well in distinguishing high-quality educational content for FineWeb dataset, there are some limitations:
|
58 |
|
59 |
-
- Scope: The model's performance might
|
60 |
- Bias: The model's performance is dependent on the quality and representativeness of the training data and the LLM used for the annotation. Biases in both can affect the classifier's judgments.
|
61 |
- Context: The classifier evaluates individual web pages or extracts without considering broader context, which might impact its effectiveness in certain scenarios.
|
62 |
|
|
|
56 |
## Limitations
|
57 |
While the FineWeb-Edu classifier performs well in distinguishing high-quality educational content for FineWeb dataset, there are some limitations:
|
58 |
|
59 |
+
- Scope: The model's performance might change for other datasets, in particular for out of distribution samples. It is also focused on educational content relevant to primary and grade school levels and may not perform as well on content intended for higher education or specialized domains.
|
60 |
- Bias: The model's performance is dependent on the quality and representativeness of the training data and the LLM used for the annotation. Biases in both can affect the classifier's judgments.
|
61 |
- Context: The classifier evaluates individual web pages or extracts without considering broader context, which might impact its effectiveness in certain scenarios.
|
62 |
|