hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4
Text Generation
•
Updated
•
1.2k
•
36
Llama 3.1 GPTQ, AWQ, and BNB Quants
Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗
-
-
hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4
Text Generation • Updated • 28 • 5 -
hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4
Text Generation • Updated • 176 • 16 -
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4
Text Generation • Updated • 40.7k • 100 -
unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit
Text Generation • Updated • 5.04k • 31 -
hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4
Text Generation • Updated • 4.59k • 23 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4
Text Generation • Updated • 357k • 67 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4
Text Generation • Updated • 512 • 8 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4
Text Generation • Updated • 143k • 25