yam-peleg commited on
Commit
306c9cd
โ€ข
1 Parent(s): 4275b59

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +66 -0
README.md CHANGED
@@ -1,3 +1,69 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - en
5
+ - he
6
+ library_name: transformers
7
  ---
8
+ # Hebrew-Mixtral-8x22B
9
+
10
+ Hebrew-Mixtral-8x22B is an open-source Large Language Model (LLM) pretrained in hebrew and english pretrained with 141 billion parameters, based on Mixtral-8x22B from Mistral.
11
+
12
+ It is continuesly pretrained from Mixtral-8x22B on tokens in both English and Hebrew.
13
+
14
+ The resulting model is a powerful general-purpose language model suitable for a wide range of natural language processing tasks, with a focus on Hebrew language understanding and generation.
15
+
16
+ ### Usage
17
+
18
+ Below are some code snippets on how to get quickly started with running the model.
19
+
20
+ First make sure to `pip install -U transformers`, then copy the snippet from the section that is relevant for your usecase.
21
+
22
+ ### Running on CPU
23
+
24
+ ```python
25
+ from transformers import AutoTokenizer, AutoModelForCausalLM
26
+
27
+ tokenizer = AutoTokenizer.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B")
28
+ model = AutoModelForCausalLM.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B")
29
+
30
+ input_text = "ืฉืœื•ื! ืžื” ืฉืœื•ืžืš ื”ื™ื•ื?"
31
+ input_ids = tokenizer(input_text, return_tensors="pt")
32
+
33
+ outputs = model.generate(**input_ids)
34
+ print(tokenizer.decode(outputs[0]))
35
+ ```
36
+
37
+ ### Running on GPU
38
+
39
+ ```python
40
+ from transformers import AutoTokenizer, AutoModelForCausalLM
41
+
42
+ tokenizer = AutoTokenizer.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B")
43
+ model = AutoModelForCausalLM.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B", device_map="auto")
44
+
45
+ input_text = "ืฉืœื•ื! ืžื” ืฉืœื•ืžืš ื”ื™ื•ื?"
46
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
47
+
48
+ outputs = model.generate(**input_ids)
49
+ print(tokenizer.decode(outputs[0]))
50
+ ```
51
+
52
+ ### Running with 4-Bit precision
53
+
54
+ ```python
55
+ from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
56
+
57
+ tokenizer = AutoTokenizer.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B")
58
+ model = AutoModelForCausalLM.from_pretrained("yam-peleg/Hebrew-Mixtral-8x22B", quantization_config = BitsAndBytesConfig(load_in_4bit=True))
59
+
60
+ input_text = "ืฉืœื•ื! ืžื” ืฉืœื•ืžืš ื”ื™ื•ื?"
61
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
62
+
63
+ outputs = model.generate(**input_ids)
64
+ print(tokenizer.decode(outputs[0])
65
+ ```
66
+
67
+ ### Notice
68
+
69
+ Hebrew-Mixtral-8x22B is a pretrained base model and therefore does not have any moderation mechanisms.