base_model:
- THUDM/glm-4-voice-9b
base_model_relation: quantized
GLM-4-Voice-9B (INT4 Quantized)
Model Overview
GLM-4-Voice is an end-to-end speech model developed by Zhipu AI. It can directly understand and generate speech in both Chinese and English, facilitating real-time voice conversations. The model can also modify voice attributes such as emotion, tone, speech rate, and dialect based on user instructions. This repository features the INT4 quantized version of GLM-4-Voice-9B. After optimization, the memory requirements are significantly reduced, requiring only 12GB of GPU memory to run smoothly. Testing has shown that this model runs well on an NVIDIA GeForce RTX 3060 with 12GB of memory.
Usage Instructions
Creating a Virtual Environment
First, ensure you are using Python 3.10, and create a virtual environment:
# Confirmed not compatible with python3.8/3.9/3.12 due to library compatibility issues
conda create -n GLM-4-Voice python=3.10
Activate the Virtual Environment and Clone the Model
After activating the virtual environment, clone the model and code:
conda activate GLM-4-Voice
git clone https://huggingface.co/cydxg/glm-4-voice-9b-int4
For users in mainland China, you can use the following command to clone:
git clone https://hf-mirror.com/cydxg/glm-4-voice-9b-int4
Install Dependencies
Navigate to the model directory and install the required dependencies:
cd glm-4-voice-9b-int4
pip install -r requirements.txt
mkdir third_party
cd third_party
git clone https://github.com/shivammehta25/Matcha-TTS Matcha-TTS
# Choose the appropriate version of torch based on your CUDA version
conda install pytorch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 pytorch-cuda=12.4 -c pytorch -c nvidia
Start the Model Service
First, start the model service:
python model_server.py
Run the Web Demo
Next, run the web demo to access the model:
python web_demo.py
You can then access the model by visiting http://localhost:8888.