dahara1 commited on
Commit
27a3dd3
1 Parent(s): c55417d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -5
README.md CHANGED
@@ -25,18 +25,25 @@ This page confirms the effectiveness of multilingual imatrix.
25
  ![wiki.test.raw_perplexity_score.png](wiki.test.raw_perplexity_score.png)
26
 
27
 
28
- ![wiki.test.raw_perplexity_score.png](wiki.test.raw_perplexity_score.png)
29
 
30
 
31
 
32
  #### Terminology
33
 
34
- Bartowski model
35
- Bartowski is a person who has quantized many models and contributed to the community. He creates imatrix from English-only data[calibration_datav3.txt](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
 
 
 
36
 
37
- Imatrix-jpn-test model
38
 
39
- Comparing the performance of models created with the multilingual version of Imatrix
 
 
 
 
 
40
 
41
 
42
 
 
25
  ![wiki.test.raw_perplexity_score.png](wiki.test.raw_perplexity_score.png)
26
 
27
 
28
+ ![ja-wiki.test.raw_perplexity_score.png](ja-wiki.test.raw_perplexity_score.png)
29
 
30
 
31
 
32
  #### Terminology
33
 
34
+ wiki.test.raw score
35
+ Perplexity Score measured using wiki.test.raw published by Salesforce and the llama-perplexity command with -c 512 setting.
36
+
37
+ ja-wiki.test.raw perplexity score
38
+ Perplexity score measured in the same way using Japanese data ([ja-wiki.test.raw]()) with the same file size as wiki.test.raw, extracted from a Japanese wiki.
39
 
 
40
 
41
+
42
+ Bartowski model
43
+ [Bartowski](https://huggingface.co/bartowski) is a person who has quantized many models and contributed to the community. He creates imatrix from English-only data[calibration_datav3.txt](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
44
+
45
+ Imatrix-jpn-test model
46
+ This model. Its created with the multilingual version of text[calibration_datav3_plus_jpn_v1.txt](https://huggingface.co/dahara1/imatrix-jpn-test/blob/main/calibration_datav3_plus_jpn_v1.txt).
47
 
48
 
49