Update README.md
Browse files
README.md
CHANGED
@@ -15,7 +15,9 @@ should probably proofread and complete it, then remove this comment. -->
|
|
15 |
|
16 |
# flan-t5-large-spelling-peft
|
17 |
|
18 |
-
This model is
|
|
|
|
|
19 |
It achieves the following results on the evaluation set:
|
20 |
- Loss: 0.2537
|
21 |
- Rouge1: 95.8905
|
@@ -26,15 +28,38 @@ It achieves the following results on the evaluation set:
|
|
26 |
|
27 |
## Model description
|
28 |
|
29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
30 |
|
31 |
## Intended uses & limitations
|
32 |
|
33 |
-
|
|
|
|
|
|
|
|
|
|
|
34 |
|
35 |
## Training and evaluation data
|
36 |
|
37 |
-
|
|
|
|
|
38 |
|
39 |
## Training procedure
|
40 |
|
|
|
15 |
|
16 |
# flan-t5-large-spelling-peft
|
17 |
|
18 |
+
This model is an *experimental* peft adapter for [google/flan-t5-large](https://huggingface.co/google/flan-t5-large)
|
19 |
+
trained on the `wiki.en` dataset from [oliverguhr/spelling](https://github.com/oliverguhr/spelling).
|
20 |
+
|
21 |
It achieves the following results on the evaluation set:
|
22 |
- Loss: 0.2537
|
23 |
- Rouge1: 95.8905
|
|
|
28 |
|
29 |
## Model description
|
30 |
|
31 |
+
This an experimental model that should be capable of fixing typos and punctuation.
|
32 |
+
|
33 |
+
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, pipeline
|
34 |
+
|
35 |
+
```python
|
36 |
+
model_id = "google/flan-t5-large"
|
37 |
+
peft_model_id = "jbochi/flan-t5-large-spelling-peft"
|
38 |
+
|
39 |
+
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
|
40 |
+
model.load_adapter(peft_model_id)
|
41 |
+
|
42 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
43 |
+
|
44 |
+
pipe = pipeline("text2text-generation", model=model, tokenizer=tokenizer)
|
45 |
+
pipe("Fix spelling: This restuarant is awesome")
|
46 |
+
# [{'generated_text': 'This restaurant is awesome'}]
|
47 |
+
```
|
48 |
|
49 |
## Intended uses & limitations
|
50 |
|
51 |
+
Intented for research purposes.
|
52 |
+
|
53 |
+
- It may produce artifacts.
|
54 |
+
- Doesn't seen capable of fixing multiple errors in a single sentence.
|
55 |
+
- It doesn't support languages other than English.
|
56 |
+
- It was fine-tuned with a `max_length` of 100 tokens.
|
57 |
|
58 |
## Training and evaluation data
|
59 |
|
60 |
+
Data from [oliverguhr/spelling](https://github.com/oliverguhr/spelling), with a "Fix spelling: " prefix added to every example.
|
61 |
+
|
62 |
+
The model was only evaluated on the first 100 test examples only during training.
|
63 |
|
64 |
## Training procedure
|
65 |
|