@eaddario on Hugging Face: "Squeezing out tensor bits, part II At post time, watt-ai/watt-tool-70B…"

Hugging Face

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Back to feed

eaddario

posted an update Mar 4

Post

2073

Squeezing out tensor bits, part II

At post time, watt-ai/watt-tool-70B continues to top the Berkeley Function-Calling Leaderboard, with the 8B version occupying the 4th place. A remarkable achievement for a model of that size!

The "squeezed" version is now available at eaddario/Watt-Tool-8B-GGUF

(For context please see: https://huggingface.co/posts/eaddario/832567461491467)

UICO

Mar 6

Well done! Your technique is very impressiove! BTW，Could you provide quantization for QWQ-32B?

eaddario

Mar 6

Thank you @UICO , but at the moment rather than a technique, it's more of a mix of brutish-force, educated guesses, trial and error and the occasional luck, but will tackle QwQ 32B next as it will help me validate an idea (see my next post)

In this post

eaddario Ed Addario
UICO UICO H