Hugging Face's logo
Robert Shaw's picture
10 4 5

Robert Shaw

robertgshaw2
shubhrapandit's profile picture 21world's profile picture mgoin's profile picture
·

AI & ML interests

None yet

Organizations

Neural Magic's profile picture Revel Labs's profile picture NM Testing's profile picture

robertgshaw2's activity

New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 about 1 month ago

vllm says the requested model does not exist

2
#1 opened about 1 month ago by shivams101
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 about 1 month ago

Storage format differs from other w4a16 models

2
#2 opened about 1 month ago by timdettmers
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 about 1 month ago

Model weights are not loaded

2
#3 opened about 1 month ago by MarvelousMouse
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 3 months ago

Can not be inferenced with vllm openai server

1
#1 opened 3 months ago by jjqsdq
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-quantized.w4a16 3 months ago

Code example request with vllm

2
#1 opened 3 months ago by ShiningJazz
New activity in neuralmagic/Mistral-7B-Instruct-v0.3-GPTQ-4bit 3 months ago

4bit quantisation does not reduce vram usage.

1
#2 opened 3 months ago by fu-man
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-FP8 3 months ago

How to run Meta-Llama-3-70B-Instruct-FP8 using several devices?

5
#3 opened 3 months ago by Fertel
New activity in open-llm-leaderboard/open_llm_leaderboard 3 months ago

Reproduction

2
#792 opened 3 months ago by robertgshaw2
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 5 months ago

Fails to run with nm-vllm

1
#1 opened 5 months ago by clintonruairi
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_50 6 months ago

Update chart template
#2 opened 6 months ago by robertgshaw2