README.md

metadata

title: LongCePO Chatbot (Sambanova)
emoji: 🤖
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.27.1
app_file: app.py
pinned: false

LongCePO Chatbot with Sambanova Backend

This is a simple chatbot interface demonstrating the LongCePO (Long-Context Planning and Optimization) method using a Sambanova model (Llama-4-Maverick-17B-128E-Instruct) as the backend LLM.

How it works

The LongCePO method is designed to handle long contexts (potentially millions of tokens) by:

1. Planning: Decomposing the initial query into sub-questions.
2. MapReduce: Answering each sub-question by processing chunks of the long context, summarizing relevant information, and aggregating results.

This application takes a long text context and a query based on that context. It then uses the modified longcepo plugin (originally from the optillm repository) to generate an answer using the Sambanova API.

How to use

1. (Optional) Enter a system prompt to guide the chatbot's behavior.
2. Paste the long text document into the Context box.
3. Enter your question based on the provided context into the Query box.
4. Click Submit.

The chatbot will process the request using the LongCePO pipeline and display the final answer.

Note: Processing long contexts can take some time depending on the length of the context and the complexity of the query.

API Key

This application requires a Sambanova API key to function. The key should be stored as a Hugging Face Space Secret named SAMBANOVA_API_KEY.