Florents-Tselai
commited on
Commit
•
4385359
1
Parent(s):
36e9d58
Update README.md
Browse files
README.md
CHANGED
@@ -11,37 +11,69 @@ base_model:
|
|
11 |
- ilsp/Meltemi-7B-Instruct-v1.5
|
12 |
---
|
13 |
|
14 |
-
# Meltemi
|
15 |
|
16 |
-
This
|
17 |
|
18 |
-
|
|
|
|
|
19 |
|
20 |
-
|
21 |
-
- 8192 context length
|
22 |
|
23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
|
25 |
-
|
26 |
|
27 |
-
|
28 |
|
29 |
-
|
|
|
|
|
30 |
|
31 |
```shell
|
32 |
llama-cli -m ./Meltemi-7B-Instruct-v1.5-F16.gguf -p "Ποιό είναι το νόημα της ζωής;" -n 128
|
33 |
```
|
34 |
|
35 |
-
|
36 |
|
37 |
```shell
|
38 |
llama-cli -m ./Meltemi-7B-Instruct-v1.5-F16.gguf --conv
|
39 |
```
|
40 |
|
41 |
-
|
42 |
|
43 |
```shell
|
44 |
llama-server -m ./Meltemi-7B-Instruct-v1.5-F16.gguf --port 8080
|
45 |
```
|
46 |
|
|
|
|
|
|
|
|
|
47 |
|
|
|
|
11 |
- ilsp/Meltemi-7B-Instruct-v1.5
|
12 |
---
|
13 |
|
14 |
+
# Meltemi llamafile & gguf
|
15 |
|
16 |
+
This repo contains `llamafile` and `gguf` file format models for [Meltemi 7B Instract v1.5](https://huggingface.co/ilsp/Meltemi-7B-Instruct-v1.5), the first Greek Large Language Model (LLM)
|
17 |
|
18 |
+
lamafile is a file format introduced by Mozilla Ocho on Nov 20th 2023,
|
19 |
+
and it collapses the complexity of an LLM into a single executable file.
|
20 |
+
This gives you the easiest, fastest way to use Meltemi on Linux, MacOS, Windows, FreeBSD, OpenBSD, and NetBSD systems you control on both AMD64 and ARM64.
|
21 |
|
22 |
+
It's as simple as this
|
|
|
23 |
|
24 |
+
```shell
|
25 |
+
wget https://huggingface.co/Florents-Tselai/Meltemi-llamafile/resolve/main/Meltemi-7B-Instruct-v1.5-Q8_0.llamafile
|
26 |
+
chmod +x Meltemi-7B-Instruct-v1.5-Q8_0.llamafile
|
27 |
+
```
|
28 |
+
|
29 |
+
```shell
|
30 |
+
./Meltemi-7B-Instruct-v1.5-Q8_0.llamafile
|
31 |
+
```
|
32 |
+
|
33 |
+
This will open a tab with a chatbot and completion interface in your browser.
|
34 |
+
For additional help on how it may be used, pass the `--help` flag.
|
35 |
+
The server also has an OpenAI API-compatible completions endpoint.
|
36 |
+
|
37 |
+
An advanced CLI mode is provided that's useful for shell scripting.
|
38 |
+
You can use it by passing the `--cli` flag. For additional help on how it may be used, pass the --help flag.
|
39 |
+
|
40 |
+
```shell
|
41 |
+
./Meltemi-7B-Instruct-v1.5-Q8_0.llamafile -p 'Ποιό είναι το νόημα της ζωής;'
|
42 |
+
```
|
43 |
+
|
44 |
+
To see all available options
|
45 |
+
|
46 |
+
```shell
|
47 |
+
./Meltemi-7B-Instruct-v1.5-Q8_0.llamafile --help
|
48 |
+
```
|
49 |
|
50 |
+
## gguf
|
51 |
|
52 |
+
`gguf` file formats are also available if you're working with llama.cpp [llama.cpp](https://github.com/ggerganov/llama.cpp)
|
53 |
|
54 |
+
llama.cpp offers quite a lot of options, thus refer to its documentation.
|
55 |
+
|
56 |
+
### Basic Usage
|
57 |
|
58 |
```shell
|
59 |
llama-cli -m ./Meltemi-7B-Instruct-v1.5-F16.gguf -p "Ποιό είναι το νόημα της ζωής;" -n 128
|
60 |
```
|
61 |
|
62 |
+
### Conversation Mode
|
63 |
|
64 |
```shell
|
65 |
llama-cli -m ./Meltemi-7B-Instruct-v1.5-F16.gguf --conv
|
66 |
```
|
67 |
|
68 |
+
### Web Server
|
69 |
|
70 |
```shell
|
71 |
llama-server -m ./Meltemi-7B-Instruct-v1.5-F16.gguf --port 8080
|
72 |
```
|
73 |
|
74 |
+
# Model Information
|
75 |
+
|
76 |
+
- Vocabulary extension of the Mistral 7b tokenizer with Greek tokens for lower costs and faster inference (**1.52** vs. 6.80 tokens/word for Greek)
|
77 |
+
- 8192 context length
|
78 |
|
79 |
+
For more details, please refer to the original model card [Meltemi 7B Instract v1.5](https://huggingface.co/ilsp/Meltemi-7B-Instruct-v1.5)
|