Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
neuralmagic
's Collections
FP8 LLMs for vLLM
Llama-3.2 Quantization
Llama-3.1 Quantization
INT8 LLMs for vLLM
INT4 LLMs for vLLM
Sparse Foundational Llama 2 Models
Compression Papers
DeepSparse Sparse LLMs
Sparse Finetuning MPT
Compressed LLMs from the Community
FP8 LLMs for vLLM
updated
9 days ago
Accurate FP8 quantized models by Neural Magic, ready for use with vLLM!
Upvote
53
+43
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8
Text Generation
•
Updated
Aug 22
•
1.47k
•
28
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8
Text Generation
•
Updated
Aug 23
•
12.8k
•
27
neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8
Text Generation
•
Updated
Aug 23
•
63.9k
•
28
neuralmagic/Phi-3-medium-128k-instruct-FP8
Text Generation
•
Updated
Aug 12
•
34.8k
•
5
neuralmagic/Mistral-Nemo-Instruct-2407-FP8
Text Generation
•
Updated
Jul 19
•
2.17k
•
13
neuralmagic/Meta-Llama-3-8B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
12.7k
•
18
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic
Text Generation
•
Updated
Aug 22
•
324
•
13
neuralmagic/Meta-Llama-3-70B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
5.65k
•
10
neuralmagic/Mixtral-8x7B-Instruct-v0.1-FP8
Text Generation
•
Updated
Jul 18
•
1.38k
•
2
neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV
Text Generation
•
Updated
Jun 19
•
20.4k
•
6
neuralmagic/Meta-Llama-3-70B-Instruct-FP8-KV
Text Generation
•
Updated
Jun 26
•
292
•
2
neuralmagic/Mixtral-8x22B-Instruct-v0.1-FP8
Text Generation
•
Updated
Aug 12
•
497
•
1
neuralmagic/Qwen2-72B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
1.11k
•
9
neuralmagic/Qwen2-7B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
678
•
1
neuralmagic/Qwen2-1.5B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
94
neuralmagic/Qwen2-0.5B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
412
•
2
neuralmagic/Mistral-7B-Instruct-v0.3-FP8
Text Generation
•
Updated
Jul 18
•
677
•
2
neuralmagic/Llama-2-7b-chat-hf-FP8
Text Generation
•
Updated
Jul 18
•
339
neuralmagic/Phi-3-mini-128k-instruct-FP8
Text Generation
•
Updated
Aug 12
•
325
neuralmagic/gemma-2-9b-it-FP8
Text Generation
•
Updated
Jul 18
•
1.35k
•
5
neuralmagic/Qwen2-57B-A14B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
385
•
1
neuralmagic/DeepSeek-Coder-V2-Lite-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
3.18k
•
4
neuralmagic/DeepSeek-Coder-V2-Lite-Base-FP8
Text Generation
•
Updated
Jul 18
•
113
neuralmagic/DeepSeek-Coder-V2-Base-FP8
Text Generation
•
Updated
Jul 22
•
11
neuralmagic/DeepSeek-Coder-V2-Instruct-FP8
Text Generation
•
Updated
Jul 22
•
2.99k
•
6
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8-dynamic
Text Generation
•
Updated
Aug 23
•
7.95k
•
5
neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8-dynamic
Text Generation
•
Updated
Aug 23
•
2k
•
2
neuralmagic/Meta-Llama-3.1-8B-FP8
Text Generation
•
Updated
Aug 13
•
1.67k
•
5
neuralmagic/Meta-Llama-3.1-70B-FP8
Text Generation
•
Updated
Aug 13
•
328
neuralmagic/starcoder2-15b-FP8
Text Generation
•
Updated
Aug 1
•
82
neuralmagic/starcoder2-3b-FP8
Text Generation
•
Updated
Aug 1
•
31
neuralmagic/starcoder2-7b-FP8
Text Generation
•
Updated
Aug 1
•
11
neuralmagic/Meta-Llama-3.1-405B-FP8
Text Generation
•
Updated
Aug 13
•
33
neuralmagic/gemma-2-2b-it-FP8
Updated
Aug 13
•
388
•
1
neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic
Text Generation
•
Updated
11 days ago
•
452
•
1
neuralmagic/Llama-3.2-3B-Instruct-FP8-dynamic
Text Generation
•
Updated
11 days ago
•
215
•
1
neuralmagic/Llama-3.2-3B-Instruct-FP8
Text Generation
•
Updated
10 days ago
•
3.33k
neuralmagic/Llama-3.2-1B-Instruct-FP8
Text Generation
•
Updated
9 days ago
•
234
neuralmagic/Llama-3.2-1B-FP8
Updated
10 days ago
•
159
neuralmagic/Phi-3.5-mini-instruct-FP8-KV
Text Generation
•
Updated
5 days ago
•
161
•
1
Upvote
53
+49
Share collection
View history
Collection guide
Browse collections