Error when running gguf-imat-for-FP16.py file
When trying to run the above-mentioned file in windows 11 facing the issue.
My log -
The 'llama.cpp' repository is ready.
Downloading latest 'llama.cpp' prebuilt Windows binaries...
Download and extraction completed successfully.
Cuda resources already exist. Skipping download.
Enter the model ID to download (e.g., huggingface/transformers): Qwen/Qwen2-0.5B
Enter the name of the imatrix.txt file (default: imatrix.txt):
Remove HF model folder after converting original model to GGUF? (yes/no) (default: no):
Model repository already exists. Using existing repository.
Traceback (most recent call last):
File "D:\PROJECTS\SL_GIT_REPO\ModelConversion\llama.cpp\convert-hf-to-gguf.py", line 19, in
import numpy as np
ModuleNotFoundError: No module named 'numpy'
Original model directory 'D:\PROJECTS\SL_GIT_REPO\ModelConversion\models\Qwen2-0.5B' was not deleted. You can remove it manually.
Error occurred while moving imatrix.dat: [WinError 2] The system cannot find the file specified
Traceback (most recent call last):
File "D:\PROJECTS\SL_GIT_REPO\ModelConversion\gguf-imat-for-FP16.py", line 170, in
main()
File "D:\PROJECTS\SL_GIT_REPO\ModelConversion\gguf-imat-for-FP16.py", line 166, in main
download_model_repo()
File "D:\PROJECTS\SL_GIT_REPO\ModelConversion\gguf-imat-for-FP16.py", line 92, in download_model_repo
convert_model_to_gguf_f16(base_dir, model_dir, model_name, delete_model_dir, imatrix_file_name)
File "D:\PROJECTS\SL_GIT_REPO\ModelConversion\gguf-imat-for-FP16.py", line 121, in convert_model_to_gguf_f16
create_imatrix(base_dir, gguf_dir, gguf_model_path, model_name, imatrix_file_name)
File "D:\PROJECTS\SL_GIT_REPO\ModelConversion\gguf-imat-for-FP16.py", line 138, in create_imatrix
quantize_models(base_dir, model_name)
File "D:\PROJECTS\SL_GIT_REPO\ModelConversion\gguf-imat-for-FP16.py", line 158, in quantize_models
subprocess.run([quantize_command, "--imatrix", imatrix_path,
File "C:\Users\Rakshit\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 501, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\Rakshit\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 966, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\Rakshit\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1435, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
For starters, it looks like you don't have the numpy package installed. Also check that you are running the script through Python 3.11.
To install the package, you can try this command in the terminal: py -3.11 -m pip install numpy
For other ModuleNotFoundError errors, do the same - download the missing packages.
Thanks for your response.
I am running on Python 3.10 and I have a Numpy package also others, but also it's throwing the error. That's why I am confused.
Maybe you have these packages installed on a different version of Python?
Still, it’s better to run it on the version of Python that is recommended. Because, for example, I had a couple of cases when an error occurred due to the wrong version of Python.
Might be worth using pip install -r .\requirements.txt.
I'd recommend you create a Python 3.11 enviroment using miniconda3...
(create: conda create --name 311gguf python=3.11, restart terminal, activate: conda activate 311gguf, run the script once, let it fail, then pip install -r .\requirements.txt, try again).