Create README.md

README.md

@@ -0,0 +1,38 @@
+---
+license: apache-2.0
+language:
+- en
+---
+
+This is GGUF model of [saucam/aqua-qwen-0.1-110B](https://huggingface.co/saucam/aqua-qwen-0.1-110B)
+
+## Usage
+
+Download the 2 files and requantize using [llama.cpp](https://github.com/ggerganov/llama.cpp). Then use the single generated file like below:
+
+```
+$ ./main -m ../aqua-qwen/ggml-model-Q4_K_M.gg
+uf -p "<|im_start|>user\nHow are you?<|im_end|>\n<|im_start|>assistant" -n 400 -e
+Log start
+main: build = 2874 (e0f55618)
+main: built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu
+main: seed  = 1715672499
+...
+...
+sampling order: 
+CFG -> Penalties -> top_k -> tfs_z -> typical_p -> top_p -> min_p -> temperature 
+generate: n_ctx = 512, n_batch = 2048, n_predict = 400, n_keep = 0
+
+
+,<|im_start|>user
+How are you?<|im_end|>
+<|im_start|>assistant
+I am an AI, I do not have feelings. How can I assist you?<|im_end|> [end of text]
+
+llama_print_timings:        load time =    4065.12 ms
+llama_print_timings:      sample time =       1.70 ms /    19 runs   (    0.09 ms per token, 11150.23 tokens per second)
+llama_print_timings: prompt eval time =    2898.40 ms /    12 tokens (  241.53 ms per token,     4.14 tokens per second)
+llama_print_timings:        eval time =  178067.55 ms /    18 runs   ( 9892.64 ms per token,     0.10 tokens per second)
+llama_print_timings:       total time =  181014.78 ms /    30 tokens
+Log end
+```