GGUF Quants for: huihui-ai/SmallThinker-3B-Preview-abliterated
Model by: huihui-ai (thank you!)
Quants by: quantflex
Run with llama.cpp:
./llama-cli -m SmallThinker-3B-Preview-abliterated-Q5_K_M.gguf -p 'You are a helpful assistant.' --temp 0.7 --top-p 0.8 --top-k 20 --repeat-penalty 1.1 -cnv --chat-template chatml
- Downloads last month
- 73
Hardware compatibility
Log In
to view the estimation
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
32-bit
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for quantflex/SmallThinker-3B-Preview-abliterated-GGUF
Base model
Qwen/Qwen2.5-3B
Finetuned
Qwen/Qwen2.5-3B-Instruct
Finetuned
PowerInfer/SmallThinker-3B-Preview