Upload folder using huggingface_hub

README.md

@@ -0,0 +1,52 @@
+---
+license: apache-2.0
+---
+
+## qwen-chat-7B-ggml
+
+This repo contains GGML format model files for qwen-chat-7B.
+
+### Example code
+
+#### Install packages
+```bash
+pip install xinference[ggml]>=0.4.3
+pip install qwen-cpp
+```
+If you want to run with GPU acceleration, refer to [installation](https://github.com/xorbitsai/inference#installation).
+
+####  Start a local instance of Xinference
+```bash
+xinference -p 9997
+```
+
+#### Launch and inference
+```python
+from xinference.client import Client
+
+client = Client("http://localhost:9997")
+model_uid = client.launch_model(
+    model_name="qwen-chat",
+    model_format="ggmlv3", 
+    model_size_in_billions=7,
+    quantization="q4_0",
+    )
+model = client.get_model(model_uid)
+
+chat_history = []
+prompt = "最大的动物是什么？"
+model.chat(
+    prompt,
+    chat_history,
+    generate_config={"max_tokens": 1024}
+)
+```
+
+### More information
+
+[Xinference](https://github.com/xorbitsai/inference) Replace OpenAI GPT with another LLM in your app 
+by changing a single line of code. Xinference gives you the freedom to use any LLM you need. 
+With Xinference, you are empowered to run inference with any open-source language models, 
+speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
+
+<i><a href="https://join.slack.com/t/xorbitsio/shared_invite/zt-1z3zsm9ep-87yI9YZ_B79HLB2ccTq4WA">👉 Join our Slack community!</a></i>

configuration.json

@@ -0,0 +1,11 @@
+{
+    "framework": "xinference",
+    "task": "code",
+    "model": {
+        "type": "qwen-chat"
+    },
+    "allow_remote": true,
+    "pipeline": {
+        "type": "text-generation-chat-pipeline"
+    }
+}

qwen7b-ggml-q4_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec3a608c9e60c1ee49d3b5a7102857d92edb87829bd4f1a31158c6e085682227
+size 4345527328