Spaces:
Runtime error
Runtime error
File size: 1,469 Bytes
163c208 276b821 163c208 276b821 163c208 cd635fb 163c208 276b821 48511d8 276b821 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
---
title: P2p Llm
emoji: 🚀
colorFrom: green
colorTo: purple
sdk: docker
sdk_version: 5.13.0
pinned: false
license: agpl-3.0
short_description: peer to peer LLM inference
app_file: app.py
app_port: 8080
models:
- deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
- unsloth/DeepSeek-R1-Distill-Qwen-1.5B-GGUF
---
# P2P LLM Inference Platform
A peer-to-peer platform for distributed LLM inference.
## Current Features
- **Trusted Peers Management**:
- Load peers from configuration file
- Load peers from database
- Combine both sources for peer discovery
- **Request Forwarding**:
- Distribute requests across available peers
- Handle both regular and SSE responses
## Planned Features
- **Trust Scores**:
- Track peer reliability and performance
- Weight request distribution based on scores
- **Peer Advertising**:
- Automatic peer discovery
- Peer-to-peer network formation
- **Enhanced Security**:
- Peer authentication
- Request validation
## Getting Started
1. Clone the repository
2. Configure `files/config.json` with your settings
3. Build and run with Docker:
```bash
docker-compose up --build
```
## Configuration
Edit `files/config.json` to specify:
- Database path
- Target URL
- Trusted peers (URLs and public keys)
- Trusted peers file path
## Development
```bash
# Run locally
go run main.go
# Run tests
go test ./...
```
Check out the configuration reference at <https://huggingface.co/docs/hub/spaces-config-reference>
|