Hugging Face's logo

ZeroAgency
/
Zero-Mistral-24B-gguf

Text Generation
GGUF
Transformers
Russian
English
llama.cpp
mistral
chat
conversational
Model card Files Community

Model Card for Zero-Mistral

This is a GGUF version of ZeroAgency/Zero-Mistral-24B.

All quants made with llama.cpp version b5083.

Quants available:

  • BF16
  • F16
  • IQ4_NL
  • IQ4_NL_L - same as above but with --leave-output-tensors
  • IQ4_XS
  • IQ4_XS_L - same as above but with --leave-output-tensors
  • Q4_K_M
  • Q4_K_M_L - same as above but with --leave-output-tensors
  • Q6_K
  • Q6_K_L - same as above but with --leave-output-tensors
  • Q8_0 - quantized from bf16 gguf
  • Q8_0-direct - direct convertation from hf
  • Q8_0_L - quantized from bf16 but with --leave-output-tensors

image/png

Downloads last month
613
GGUF
Model size
23.6B params
Architecture
llama
Hardware compatibility
Log In to view the estimation

4-bit

6-bit

8-bit

16-bit

Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ZeroAgency/Zero-Mistral-24B-gguf

Base model

mistralai/Mistral-Small-3.1-24B-Base-2503
Finetuned
mistralai/Mistral-Small-3.1-24B-Instruct-2503
Finetuned
ZeroAgency/Zero-Mistral-24B
Quantized
(3)
this model

Dataset used to train ZeroAgency/Zero-Mistral-24B-gguf

Collection including ZeroAgency/Zero-Mistral-24B-gguf