Use unique model_type "chatts" instead of reusing "qwen2"
#8
by
alexanderchemeris
- opened
This change is required to correctly auto-load the model configuration class and link it to the implementation in vllm library as being implemented at https://github.com/vllm-project/vllm/pull/16852
The auto-loading is based on model_type maps and requires a unique name to load specific classes.
With this change and the vllm change mentioned above, the model can be loaded without any changes by vllm both for offline and online (OpenAI API) inference.
alexanderchemeris
changed pull request status to
open
Thanks for implementing this and ChatTS support in vLLM!
xiezhe24
changed pull request status to
merged