Hugging Face's logo

Spaces:
HuggingFaceH4
/
zephyr-chat
Running on A100

App Files Community
31

How is the model so fast and accurate?

#17
by Saugatkafley - opened
Discussion
Saugatkafley

I am really impressed by how fast it can generate excellent answers almost instantly. What was used behind this low latency inference?

olivierdehaene

This demo uses TGI https://github.com/huggingface/text-generation-inference as a backend.

Saugatkafley

Thank you so much! @olivierdehaene . It is really fast!

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment