大模型技术栈-实战与应用

训练框架
- deepspeed
- megatron-lm
- colossal-ai
- trlx
推理框架
- triton
- vllm
- text-generation-inference
- lit-llama
- lightllm
- TensorRT-LLM(原FasterTransformer)
- fastllm
- inferllm
- llama-cpp
- openPPL-LLM
压缩框架
- bitsandbytes
- auto-gptq
- deepspeed
embedding框架
- sentence-transformer
- FlagEmbedding
向量数据库向量数据库对比
- faiss
- pgvector
- milvus
- pinecone
- weaviate
- LanceDB
- Chroma
应用框架
- Auto-GPT
- langchain
- llama-index
- quivr
python前端
- streamlit
- gradio
python API工具
- FastAPI+uvicorn
- flask
- Django