Upload folder using huggingface_hub
Browse files- .container/.dockerignore +74 -0
- .container/DOCKER_README.md +298 -0
- .container/DOCKER_README_en.md +298 -0
- .container/Dockerfile +106 -0
- .container/build_docker.bat +147 -0
- .container/build_docker.sh +150 -0
- .container/check_docker.bat +62 -0
- .container/check_docker.sh +92 -0
- .container/docker-compose.yml +52 -0
- .container/run_in_docker.bat +116 -0
- .container/run_in_docker.sh +135 -0
- .gitattributes +10 -0
- .gitignore +60 -0
- .gradio/certificate.pem +31 -0
- .pre-commit-config.yaml +29 -0
- README.md +388 -9
- README_zh.md +375 -0
- assets/community.png +3 -0
- assets/community_2.png +3 -0
- assets/community_3.jpg +3 -0
- assets/community_4.jpg +3 -0
- assets/community_5.jpg +3 -0
- assets/community_6.jpg +3 -0
- assets/community_6.png +3 -0
- assets/meetup.jpg +3 -0
- assets/owl_architecture.png +3 -0
- assets/qr_code.jpg +3 -0
- licenses/LICENSE +13 -0
- licenses/license_template.txt +13 -0
- licenses/update_license.py +126 -0
- owl/.env_template +28 -0
- owl/app.py +669 -0
- owl/run.py +138 -0
- owl/run_deepseek_zh.py +141 -0
- owl/run_gaia_roleplaying.py +135 -0
- owl/run_mini.py +112 -0
- owl/run_openai_compatiable_model.py +136 -0
- owl/run_qwen_mini_zh.py +102 -0
- owl/run_qwen_zh.py +145 -0
- owl/script_adapter.py +263 -0
- owl/utils/__init__.py +27 -0
- owl/utils/common.py +36 -0
- owl/utils/document_toolkit.py +324 -0
- owl/utils/enhanced_role_playing.py +431 -0
- owl/utils/gaia.py +412 -0
- pyproject.toml +59 -0
- requirements.txt +4 -1
- run_app.py +60 -0
- run_app.py.bak +59 -0
- uv.lock +0 -0
.container/.dockerignore
ADDED
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Git
|
2 |
+
.git
|
3 |
+
.gitignore
|
4 |
+
.github
|
5 |
+
|
6 |
+
# Docker
|
7 |
+
Dockerfile
|
8 |
+
docker-compose.yml
|
9 |
+
.dockerignore
|
10 |
+
DOCKER_README.md
|
11 |
+
run_in_docker.sh
|
12 |
+
|
13 |
+
# Python
|
14 |
+
__pycache__/
|
15 |
+
*.py[cod]
|
16 |
+
*$py.class
|
17 |
+
*.so
|
18 |
+
.Python
|
19 |
+
env/
|
20 |
+
build/
|
21 |
+
develop-eggs/
|
22 |
+
dist/
|
23 |
+
downloads/
|
24 |
+
eggs/
|
25 |
+
.eggs/
|
26 |
+
lib/
|
27 |
+
lib64/
|
28 |
+
parts/
|
29 |
+
sdist/
|
30 |
+
var/
|
31 |
+
*.egg-info/
|
32 |
+
.installed.cfg
|
33 |
+
*.egg
|
34 |
+
.pytest_cache/
|
35 |
+
.coverage
|
36 |
+
htmlcov/
|
37 |
+
|
38 |
+
# 虚拟环境
|
39 |
+
venv/
|
40 |
+
ENV/
|
41 |
+
env/
|
42 |
+
.env
|
43 |
+
|
44 |
+
# IDE
|
45 |
+
.idea/
|
46 |
+
.vscode/
|
47 |
+
*.swp
|
48 |
+
*.swo
|
49 |
+
.DS_Store
|
50 |
+
|
51 |
+
# 临时文件
|
52 |
+
temp_*
|
53 |
+
*.tmp
|
54 |
+
*.log
|
55 |
+
*.bak
|
56 |
+
|
57 |
+
# 缓存
|
58 |
+
.cache/
|
59 |
+
.npm/
|
60 |
+
.yarn/
|
61 |
+
|
62 |
+
# 大型数据文件
|
63 |
+
*.csv
|
64 |
+
*.sqlite
|
65 |
+
*.db
|
66 |
+
*.hdf5
|
67 |
+
*.h5
|
68 |
+
*.parquet
|
69 |
+
*.feather
|
70 |
+
*.pkl
|
71 |
+
*.pickle
|
72 |
+
|
73 |
+
# 数据目录
|
74 |
+
data/
|
.container/DOCKER_README.md
ADDED
@@ -0,0 +1,298 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# OWL项目Docker使用指南
|
2 |
+
|
3 |
+
本文档提供了如何使用Docker运行OWL项目的详细说明。
|
4 |
+
|
5 |
+
## 前提条件
|
6 |
+
|
7 |
+
- 安装 [Docker](https://docs.docker.com/get-docker/)
|
8 |
+
- 安装 [Docker Compose](https://docs.docker.com/compose/install/) (推荐v2.x版本)
|
9 |
+
- 获取必要的API密钥(OpenAI API等)
|
10 |
+
|
11 |
+
## 技术说明
|
12 |
+
|
13 |
+
本Docker配置使用了以下技术来确保OWL项目在容器中正常运行:
|
14 |
+
|
15 |
+
- **Xvfb**:虚拟帧缓冲区,用于在无显示器的环境中模拟X服务器
|
16 |
+
- **Playwright**:用于自动化浏览器操作,配置为无头模式
|
17 |
+
- **共享内存**:增加了共享内存大小,以提高浏览器性能
|
18 |
+
- **BuildKit**:使用Docker BuildKit加速构建过程
|
19 |
+
- **缓存优化**:使用持久化卷缓存pip和Playwright依赖
|
20 |
+
- **跨平台兼容**:提供了适用于Windows和macOS/Linux的脚本
|
21 |
+
|
22 |
+
## Docker Compose版本说明
|
23 |
+
|
24 |
+
本项目使用的docker-compose.yml文件兼容Docker Compose v2.x版本。如果您使用的是较旧的Docker Compose v1.x版本,可能需要手动添加版本号:
|
25 |
+
|
26 |
+
```yaml
|
27 |
+
version: '3'
|
28 |
+
|
29 |
+
services:
|
30 |
+
# ...其余配置保持不变
|
31 |
+
```
|
32 |
+
|
33 |
+
## 快速开始
|
34 |
+
|
35 |
+
### 0. 检查环境
|
36 |
+
|
37 |
+
首先,运行检查脚本确保您的环境已准备好:
|
38 |
+
|
39 |
+
#### 在macOS/Linux上检查
|
40 |
+
|
41 |
+
```bash
|
42 |
+
# 先给脚本添加执行权限
|
43 |
+
chmod +x check_docker.sh
|
44 |
+
|
45 |
+
# 运行检查脚本
|
46 |
+
./check_docker.sh
|
47 |
+
```
|
48 |
+
|
49 |
+
#### 在Windows上检查
|
50 |
+
|
51 |
+
```cmd
|
52 |
+
check_docker.bat
|
53 |
+
```
|
54 |
+
|
55 |
+
如果检查脚本发现任何问题,请按照提示进行修复。
|
56 |
+
|
57 |
+
### 1. 配置环境变量
|
58 |
+
|
59 |
+
复制环境变量模板文件并填写必要的API密钥:
|
60 |
+
|
61 |
+
```bash
|
62 |
+
cp owl/.env_template owl/.env
|
63 |
+
```
|
64 |
+
|
65 |
+
然后编辑 `owl/.env` 文件,填写必要的API密钥,例如:
|
66 |
+
|
67 |
+
```
|
68 |
+
OPENAI_API_KEY=your_openai_api_key
|
69 |
+
GOOGLE_API_KEY=your_google_api_key
|
70 |
+
SEARCH_ENGINE_ID=your_search_engine_id
|
71 |
+
```
|
72 |
+
|
73 |
+
### 2. 快速构建Docker镜像
|
74 |
+
|
75 |
+
#### 在macOS/Linux上构建
|
76 |
+
|
77 |
+
使用提供的Shell脚本,可以加速Docker镜像的构建:
|
78 |
+
|
79 |
+
```bash
|
80 |
+
# 先给脚本添加执行权限
|
81 |
+
chmod +x build_docker.sh
|
82 |
+
|
83 |
+
# 运行构建脚本
|
84 |
+
./build_docker.sh
|
85 |
+
```
|
86 |
+
|
87 |
+
#### 在Windows上构建
|
88 |
+
|
89 |
+
使用提供的批处理文件:
|
90 |
+
|
91 |
+
```cmd
|
92 |
+
build_docker.bat
|
93 |
+
```
|
94 |
+
|
95 |
+
或者使用标准方式构建并启动:
|
96 |
+
|
97 |
+
```bash
|
98 |
+
# 使用BuildKit加速构建
|
99 |
+
set DOCKER_BUILDKIT=1
|
100 |
+
set COMPOSE_DOCKER_CLI_BUILD=1
|
101 |
+
docker-compose build --build-arg BUILDKIT_INLINE_CACHE=1
|
102 |
+
|
103 |
+
# 启动容器
|
104 |
+
docker-compose up -d
|
105 |
+
```
|
106 |
+
|
107 |
+
### 3. 交互式使用容器
|
108 |
+
|
109 |
+
容器启动后,会自动进入交互式shell环境,并显示欢迎信息和可用脚本列表:
|
110 |
+
|
111 |
+
```bash
|
112 |
+
# 进入容器(如果没有自动进入)
|
113 |
+
docker-compose exec owl bash
|
114 |
+
```
|
115 |
+
|
116 |
+
在容器内,您可以直接运行任何可用的脚本:
|
117 |
+
|
118 |
+
```bash
|
119 |
+
# 运行默认脚本
|
120 |
+
xvfb-python run.py
|
121 |
+
|
122 |
+
# 运行DeepSeek示例
|
123 |
+
xvfb-python run_deepseek_example.py
|
124 |
+
|
125 |
+
# 运行脚本并传递查询参数
|
126 |
+
xvfb-python run.py "什么是人工智能?"
|
127 |
+
```
|
128 |
+
|
129 |
+
### 4. 使用外部脚本运行查询
|
130 |
+
|
131 |
+
#### 在macOS/Linux上运行
|
132 |
+
|
133 |
+
```bash
|
134 |
+
# 先给脚本添加执行权限
|
135 |
+
chmod +x run_in_docker.sh
|
136 |
+
|
137 |
+
# 默认使用 run.py 脚本
|
138 |
+
./run_in_docker.sh "你的问题"
|
139 |
+
|
140 |
+
# 指定使用特定脚本
|
141 |
+
./run_in_docker.sh run_deepseek_example.py "你的问题"
|
142 |
+
```
|
143 |
+
|
144 |
+
#### 在Windows上运行
|
145 |
+
|
146 |
+
```cmd
|
147 |
+
REM 默认使用 run.py 脚本
|
148 |
+
run_in_docker.bat "你的问题"
|
149 |
+
|
150 |
+
REM 指定使用特定脚本
|
151 |
+
run_in_docker.bat run_deepseek_example.py "你的问题"
|
152 |
+
```
|
153 |
+
|
154 |
+
**可用脚本**:
|
155 |
+
- `run.py` - 默认脚本,使用OpenAI GPT-4o模型
|
156 |
+
- `run_deepseek_example.py` - 使用DeepSeek模型
|
157 |
+
- `run_gaia_roleplaying.py` - GAIA基准测试脚本
|
158 |
+
|
159 |
+
## 目录挂载
|
160 |
+
|
161 |
+
Docker Compose配置中已经设置了以下挂载点:
|
162 |
+
|
163 |
+
- `./owl/.env:/app/owl/.env`:挂载环境变量文件,方便修改API密钥
|
164 |
+
- `./data:/app/data`:挂载数据目录,用于存储和访问数据文件
|
165 |
+
- `playwright-cache`:持久化卷,用于缓存Playwright浏览器
|
166 |
+
- `pip-cache`:持久化卷,用于缓存pip包
|
167 |
+
|
168 |
+
## 环境变量
|
169 |
+
|
170 |
+
您可以通过以下两种方式设置环境变量:
|
171 |
+
|
172 |
+
1. 修改 `owl/.env` 文件
|
173 |
+
2. 在 `docker-compose.yml` 文件的 `environment` 部分添加环境变量
|
174 |
+
|
175 |
+
## 构建优化
|
176 |
+
|
177 |
+
本Docker配置包含多项构建优化:
|
178 |
+
|
179 |
+
1. **使用国内镜像源**:使用清华大学镜像源加速pip包下载
|
180 |
+
2. **层优化**:减少Dockerfile中的层数,提高构建效率
|
181 |
+
3. **缓存利用**:
|
182 |
+
- 启用pip缓存,避免重复下载依赖包
|
183 |
+
- 使用Docker BuildKit内联缓存
|
184 |
+
- 合理安排Dockerfile指令顺序,最大化利用缓存
|
185 |
+
4. **BuildKit**:启用Docker BuildKit加速构建
|
186 |
+
5. **持久化缓存**:
|
187 |
+
- 使用Docker卷缓存pip包(`pip-cache`)
|
188 |
+
- 使用Docker卷缓存Playwright浏览器(`playwright-cache`)
|
189 |
+
- 本地缓存目录(`.docker-cache`)
|
190 |
+
|
191 |
+
### 缓存清理
|
192 |
+
|
193 |
+
如果需要清理缓存,可以使用以下命令:
|
194 |
+
|
195 |
+
```bash
|
196 |
+
# 清理Docker构建缓存
|
197 |
+
docker builder prune
|
198 |
+
|
199 |
+
# 清理Docker卷(会删除所有未使用的卷,包括缓存卷)
|
200 |
+
docker volume prune
|
201 |
+
|
202 |
+
# 清理本��缓存目录
|
203 |
+
rm -rf .docker-cache
|
204 |
+
```
|
205 |
+
|
206 |
+
## 跨平台兼容性
|
207 |
+
|
208 |
+
本项目提供了适用于不同操作系统的脚本:
|
209 |
+
|
210 |
+
1. **检查脚本**:
|
211 |
+
- `check_docker.sh`(macOS/Linux):检查Docker环境
|
212 |
+
- `check_docker.bat`(Windows):检查Docker环境
|
213 |
+
|
214 |
+
2. **构建脚本**:
|
215 |
+
- `build_docker.sh`(macOS/Linux):构建Docker镜像
|
216 |
+
- `build_docker.bat`(Windows):构建Docker镜像
|
217 |
+
|
218 |
+
3. **运行脚本**:
|
219 |
+
- `run_in_docker.sh`(macOS/Linux):运行Docker容器中的脚本
|
220 |
+
- `run_in_docker.bat`(Windows):运行Docker容器中的脚本
|
221 |
+
|
222 |
+
这些脚本会自动检测操作系统类型,并使用适当的命令。
|
223 |
+
|
224 |
+
## 故障排除
|
225 |
+
|
226 |
+
### 容器无法启动
|
227 |
+
|
228 |
+
检查日志以获取更多信息:
|
229 |
+
|
230 |
+
```bash
|
231 |
+
docker-compose logs
|
232 |
+
```
|
233 |
+
|
234 |
+
### API密钥问题
|
235 |
+
|
236 |
+
确保您已经在 `owl/.env` 文件中正确设置了所有必要的API密钥。
|
237 |
+
|
238 |
+
### Docker Compose警告
|
239 |
+
|
240 |
+
如果您看到关于`version`属性过时的警告:
|
241 |
+
|
242 |
+
```
|
243 |
+
WARN[0000] docker-compose.yml: the attribute `version` is obsolete
|
244 |
+
```
|
245 |
+
|
246 |
+
这是因为您使用的是Docker Compose v2.x,它不再需要显式指定版本号。我们已经从配置文件中移除了这个属性,所以您不会再看到这个警告。
|
247 |
+
|
248 |
+
### 浏览器相关问题
|
249 |
+
|
250 |
+
如果遇到浏览器相关的问题,可以尝试以下解决方案:
|
251 |
+
|
252 |
+
1. 确保在Docker容器中使用`xvfb-python`命令运行Python脚本
|
253 |
+
2. 检查是否正确安装了Xvfb和相关依赖
|
254 |
+
3. 增加共享内存大小(在docker-compose.yml中已设置为2GB)
|
255 |
+
|
256 |
+
### 构建速度慢
|
257 |
+
|
258 |
+
如果构建速度慢,可以尝试以下解决方案:
|
259 |
+
|
260 |
+
1. 确保启用了Docker BuildKit(`DOCKER_BUILDKIT=1`)
|
261 |
+
2. 确保启用了pip缓存(已在docker-compose.yml中配置)
|
262 |
+
3. 使用`--build-arg BUILDKIT_INLINE_CACHE=1`参数构建(已在构建脚本中配置)
|
263 |
+
4. 如果是首次构建,下载依赖包可能需要较长时间,后续构建会更快
|
264 |
+
|
265 |
+
### Windows特有问题
|
266 |
+
|
267 |
+
如果在Windows上遇到问题:
|
268 |
+
|
269 |
+
1. 确保使用管理员权限运行命令提示符或PowerShell
|
270 |
+
2. 如果遇到路径问题,尝试使用正斜杠(/)而不是反斜杠(\)
|
271 |
+
3. 如果遇到Docker Compose命令问题,尝试使用`docker compose`(无连字符)
|
272 |
+
|
273 |
+
### 内存不足
|
274 |
+
|
275 |
+
如果遇到内存不足的问题,可以在 `docker-compose.yml` 文件中调整资源限制:
|
276 |
+
|
277 |
+
```yaml
|
278 |
+
services:
|
279 |
+
owl:
|
280 |
+
# 其他配置...
|
281 |
+
deploy:
|
282 |
+
resources:
|
283 |
+
limits:
|
284 |
+
cpus: '4' # 增加CPU核心数
|
285 |
+
memory: 8G # 增加内存限制
|
286 |
+
```
|
287 |
+
|
288 |
+
## 自定义Docker镜像
|
289 |
+
|
290 |
+
如果需要自定义Docker镜像,可以修改 `Dockerfile` 文件,然后重新构建:
|
291 |
+
|
292 |
+
```bash
|
293 |
+
# macOS/Linux
|
294 |
+
./build_docker.sh
|
295 |
+
|
296 |
+
# Windows
|
297 |
+
build_docker.bat
|
298 |
+
```
|
.container/DOCKER_README_en.md
ADDED
@@ -0,0 +1,298 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# OWL Project Docker Usage Guide
|
2 |
+
|
3 |
+
This document provides detailed instructions on how to run the OWL project using Docker.
|
4 |
+
|
5 |
+
## Prerequisites
|
6 |
+
|
7 |
+
• Install [Docker](https://docs.docker.com/get-docker/)
|
8 |
+
• Install [Docker Compose](https://docs.docker.com/compose/install/) (recommended v2.x version)
|
9 |
+
• Obtain necessary API keys (OpenAI API, etc.)
|
10 |
+
|
11 |
+
## Technical Notes
|
12 |
+
|
13 |
+
This Docker configuration uses the following technologies to ensure the OWL project runs smoothly in containers:
|
14 |
+
|
15 |
+
• **Xvfb**: Virtual framebuffer, used to simulate an X server in a headless environment
|
16 |
+
• **Playwright**: Used for browser automation, configured in headless mode
|
17 |
+
• **Shared Memory**: Increased shared memory size to improve browser performance
|
18 |
+
• **BuildKit**: Uses Docker BuildKit to accelerate the build process
|
19 |
+
• **Cache Optimization**: Uses persistent volumes to cache pip and Playwright dependencies
|
20 |
+
• **Cross-Platform Compatibility**: Provides scripts for both Windows and macOS/Linux
|
21 |
+
|
22 |
+
## Docker Compose Version Notes
|
23 |
+
|
24 |
+
The docker-compose.yml file used in this project is compatible with Docker Compose v2.x. If you are using an older Docker Compose v1.x version, you may need to manually add the version number:
|
25 |
+
|
26 |
+
```yaml
|
27 |
+
version: '3'
|
28 |
+
|
29 |
+
services:
|
30 |
+
# ...rest of the configuration remains unchanged
|
31 |
+
```
|
32 |
+
|
33 |
+
## Quick Start
|
34 |
+
|
35 |
+
### 0. Check Environment
|
36 |
+
|
37 |
+
First, run the check script to ensure your environment is ready:
|
38 |
+
|
39 |
+
#### Check on macOS/Linux
|
40 |
+
|
41 |
+
```bash
|
42 |
+
# First, add execute permissions to the script
|
43 |
+
chmod +x check_docker.sh
|
44 |
+
|
45 |
+
# Run the check script
|
46 |
+
./check_docker.sh
|
47 |
+
```
|
48 |
+
|
49 |
+
#### Check on Windows
|
50 |
+
|
51 |
+
```cmd
|
52 |
+
check_docker.bat
|
53 |
+
```
|
54 |
+
|
55 |
+
If the check script finds any issues, please follow the prompts to fix them.
|
56 |
+
|
57 |
+
### 1. Configure Environment Variables
|
58 |
+
|
59 |
+
Copy the environment variable template file and fill in the necessary API keys:
|
60 |
+
|
61 |
+
```bash
|
62 |
+
cp owl/.env_template owl/.env
|
63 |
+
```
|
64 |
+
|
65 |
+
Then edit the `owl/.env` file and fill in the necessary API keys, for example:
|
66 |
+
|
67 |
+
```
|
68 |
+
OPENAI_API_KEY=your_openai_api_key
|
69 |
+
GOOGLE_API_KEY=your_google_api_key
|
70 |
+
SEARCH_ENGINE_ID=your_search_engine_id
|
71 |
+
```
|
72 |
+
|
73 |
+
### 2. Quick Build Docker Image
|
74 |
+
|
75 |
+
#### Build on macOS/Linux
|
76 |
+
|
77 |
+
Use the provided shell script to speed up the Docker image build:
|
78 |
+
|
79 |
+
```bash
|
80 |
+
# First, add execute permissions to the script
|
81 |
+
chmod +x build_docker.sh
|
82 |
+
|
83 |
+
# Run the build script
|
84 |
+
./build_docker.sh
|
85 |
+
```
|
86 |
+
|
87 |
+
#### Build on Windows
|
88 |
+
|
89 |
+
Use the provided batch file:
|
90 |
+
|
91 |
+
```cmd
|
92 |
+
build_docker.bat
|
93 |
+
```
|
94 |
+
|
95 |
+
Or build and start using the standard method:
|
96 |
+
|
97 |
+
```bash
|
98 |
+
# Use BuildKit to accelerate the build
|
99 |
+
set DOCKER_BUILDKIT=1
|
100 |
+
set COMPOSE_DOCKER_CLI_BUILD=1
|
101 |
+
docker-compose build --build-arg BUILDKIT_INLINE_CACHE=1
|
102 |
+
|
103 |
+
# Start the container
|
104 |
+
docker-compose up -d
|
105 |
+
```
|
106 |
+
|
107 |
+
### 3. Interactive Use of the Container
|
108 |
+
|
109 |
+
After the container starts, it will automatically enter an interactive shell environment and display a welcome message and a list of available scripts:
|
110 |
+
|
111 |
+
```bash
|
112 |
+
# Enter the container (if not automatically entered)
|
113 |
+
docker-compose exec owl bash
|
114 |
+
```
|
115 |
+
|
116 |
+
Inside the container, you can directly run any available script:
|
117 |
+
|
118 |
+
```bash
|
119 |
+
# Run the default script
|
120 |
+
xvfb-python run.py
|
121 |
+
|
122 |
+
# Run the DeepSeek example
|
123 |
+
xvfb-python run_deepseek_example.py
|
124 |
+
|
125 |
+
# Run the script and pass query parameters
|
126 |
+
xvfb-python run.py "What is artificial intelligence?"
|
127 |
+
```
|
128 |
+
|
129 |
+
### 4. Run Queries Using External Scripts
|
130 |
+
|
131 |
+
#### Run on macOS/Linux
|
132 |
+
|
133 |
+
```bash
|
134 |
+
# First, add execute permissions to the script
|
135 |
+
chmod +x run_in_docker.sh
|
136 |
+
|
137 |
+
# Default to using the run.py script
|
138 |
+
./run_in_docker.sh "your question"
|
139 |
+
|
140 |
+
# Specify a particular script
|
141 |
+
./run_in_docker.sh run_deepseek_example.py "your question"
|
142 |
+
```
|
143 |
+
|
144 |
+
#### Run on Windows
|
145 |
+
|
146 |
+
```cmd
|
147 |
+
REM Default to using the run.py script
|
148 |
+
run_in_docker.bat "your question"
|
149 |
+
|
150 |
+
REM Specify a particular script
|
151 |
+
run_in_docker.bat run_deepseek_example.py "your question"
|
152 |
+
```
|
153 |
+
|
154 |
+
**Available Scripts**:
|
155 |
+
• `run.py` - Default script, uses OpenAI GPT-4o model
|
156 |
+
• `run_deepseek_example.py` - Uses the DeepSeek model
|
157 |
+
• `run_gaia_roleplaying.py` - GAIA benchmark script
|
158 |
+
|
159 |
+
## Directory Mounts
|
160 |
+
|
161 |
+
The Docker Compose configuration has set up the following mount points:
|
162 |
+
|
163 |
+
• `./owl/.env:/app/owl/.env`: Mounts the environment variable file for easy modification of API keys
|
164 |
+
• `./data:/app/data`: Mounts the data directory for storing and accessing data files
|
165 |
+
• `playwright-cache`: Persistent volume for caching Playwright browsers
|
166 |
+
• `pip-cache`: Persistent volume for caching pip packages
|
167 |
+
|
168 |
+
## Environment Variables
|
169 |
+
|
170 |
+
You can set environment variables in two ways:
|
171 |
+
|
172 |
+
1. Modify the `owl/.env` file
|
173 |
+
2. Add environment variables in the `environment` section of the `docker-compose.yml` file
|
174 |
+
|
175 |
+
## Build Optimization
|
176 |
+
|
177 |
+
This Docker configuration includes several build optimizations:
|
178 |
+
|
179 |
+
1. **Use of Domestic Mirror Sources**: Uses Tsinghua University mirror sources to accelerate pip package downloads
|
180 |
+
2. **Layer Optimization**: Reduces the number of layers in the Dockerfile to improve build efficiency
|
181 |
+
3. **Cache Utilization**:
|
182 |
+
• Enables pip caching to avoid repeated dependency downloads
|
183 |
+
• Uses Docker BuildKit inline caching
|
184 |
+
• Arranges Dockerfile instructions to maximize cache utilization
|
185 |
+
4. **BuildKit**: Enables Docker BuildKit to accelerate builds
|
186 |
+
5. **Persistent Caching**:
|
187 |
+
• Uses Docker volumes to cache pip packages (`pip-cache`)
|
188 |
+
• Uses Docker volumes to cache Playwright browsers (`playwright-cache`)
|
189 |
+
• Local cache directory (`.docker-cache`)
|
190 |
+
|
191 |
+
### Cache Cleanup
|
192 |
+
|
193 |
+
If you need to clean the cache, you can use the following commands:
|
194 |
+
|
195 |
+
```bash
|
196 |
+
# Clean Docker build cache
|
197 |
+
docker builder prune
|
198 |
+
|
199 |
+
# Clean Docker volumes (will delete all unused volumes, including cache volumes)
|
200 |
+
docker volume prune
|
201 |
+
|
202 |
+
# Clean local cache directory
|
203 |
+
rm -rf .docker-cache
|
204 |
+
```
|
205 |
+
|
206 |
+
## Cross-Platform Compatibility
|
207 |
+
|
208 |
+
This project provides scripts for different operating systems:
|
209 |
+
|
210 |
+
1. **Check Scripts**:
|
211 |
+
• `check_docker.sh` (macOS/Linux): Checks the Docker environment
|
212 |
+
• `check_docker.bat` (Windows): Checks the Docker environment
|
213 |
+
|
214 |
+
2. **Build Scripts**:
|
215 |
+
• `build_docker.sh` (macOS/Linux): Builds the Docker image
|
216 |
+
• `build_docker.bat` (Windows): Builds the Docker image
|
217 |
+
|
218 |
+
3. **Run Scripts**:
|
219 |
+
• `run_in_docker.sh` (macOS/Linux): Runs scripts in the Docker container
|
220 |
+
• `run_in_docker.bat` (Windows): Runs scripts in the Docker container
|
221 |
+
|
222 |
+
These scripts automatically detect the operating system type and use appropriate commands.
|
223 |
+
|
224 |
+
## Troubleshooting
|
225 |
+
|
226 |
+
### Container Fails to Start
|
227 |
+
|
228 |
+
Check the logs for more information:
|
229 |
+
|
230 |
+
```bash
|
231 |
+
docker-compose logs
|
232 |
+
```
|
233 |
+
|
234 |
+
### API Key Issues
|
235 |
+
|
236 |
+
Ensure that you have correctly set all necessary API keys in the `owl/.env` file.
|
237 |
+
|
238 |
+
### Docker Compose Warnings
|
239 |
+
|
240 |
+
If you see a warning about the `version` attribute being obsolete:
|
241 |
+
|
242 |
+
```
|
243 |
+
WARN[0000] docker-compose.yml: the attribute `version` is obsolete
|
244 |
+
```
|
245 |
+
|
246 |
+
This is because you are using Docker Compose v2.x, which no longer requires an explicit version number. We have removed this attribute from the configuration file, so you should no longer see this warning.
|
247 |
+
|
248 |
+
### Browser-Related Issues
|
249 |
+
|
250 |
+
If you encounter browser-related issues, try the following solutions:
|
251 |
+
|
252 |
+
1. Ensure that you are using the `xvfb-python` command to run Python scripts in the Docker container
|
253 |
+
2. Check that Xvfb and related dependencies are correctly installed
|
254 |
+
3. Increase the shared memory size (set to 2GB in docker-compose.yml)
|
255 |
+
|
256 |
+
### Slow Build Speed
|
257 |
+
|
258 |
+
If the build speed is slow, try the following solutions:
|
259 |
+
|
260 |
+
1. Ensure that Docker BuildKit is enabled (`DOCKER_BUILDKIT=1`)
|
261 |
+
2. Ensure that pip caching is enabled (configured in docker-compose.yml)
|
262 |
+
3. Use the `--build-arg BUILDKIT_INLINE_CACHE=1` parameter when building (configured in the build script)
|
263 |
+
4. If this is the first build, downloading dependencies may take some time, but subsequent builds will be faster
|
264 |
+
|
265 |
+
### Windows-Specific Issues
|
266 |
+
|
267 |
+
If you encounter issues on Windows:
|
268 |
+
|
269 |
+
1. Ensure that you are running the Command Prompt or PowerShell with administrator privileges
|
270 |
+
2. If you encounter path issues, try using forward slashes (/) instead of backslashes (\)
|
271 |
+
3. If you encounter Docker Compose command issues, try using `docker compose` (without the hyphen)
|
272 |
+
|
273 |
+
### Insufficient Memory
|
274 |
+
|
275 |
+
If you encounter insufficient memory issues, you can adjust resource limits in the `docker-compose.yml` file:
|
276 |
+
|
277 |
+
```yaml
|
278 |
+
services:
|
279 |
+
owl:
|
280 |
+
# Other configurations...
|
281 |
+
deploy:
|
282 |
+
resources:
|
283 |
+
limits:
|
284 |
+
cpus: '4' # Increase CPU cores
|
285 |
+
memory: 8G # Increase memory limit
|
286 |
+
```
|
287 |
+
|
288 |
+
## Custom Docker Image
|
289 |
+
|
290 |
+
If you need to customize the Docker image, modify the `Dockerfile` file and then rebuild:
|
291 |
+
|
292 |
+
```bash
|
293 |
+
# macOS/Linux
|
294 |
+
./build_docker.sh
|
295 |
+
|
296 |
+
# Windows
|
297 |
+
build_docker.bat
|
298 |
+
```
|
.container/Dockerfile
ADDED
@@ -0,0 +1,106 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# 使用ARG定义可配置的构建参数 | Using ARG to define configurable build parameters
|
2 |
+
ARG PYTHON_VERSION=3.10
|
3 |
+
ARG PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple
|
4 |
+
ARG PLAYWRIGHT_DOWNLOAD_HOST=https://npmmirror.com/mirrors/playwright
|
5 |
+
|
6 |
+
# 第一阶段:构建依赖 | Stage 1: Build dependencies
|
7 |
+
FROM python:${PYTHON_VERSION}-slim AS builder
|
8 |
+
|
9 |
+
# 设置工作目录 | Set working directory
|
10 |
+
WORKDIR /build
|
11 |
+
|
12 |
+
# 设置pip镜像源以加速下载 | Set pip mirror to accelerate downloads
|
13 |
+
ARG PIP_INDEX_URL
|
14 |
+
RUN pip config set global.index-url ${PIP_INDEX_URL}
|
15 |
+
|
16 |
+
# 安装构建依赖 | Install build dependencies
|
17 |
+
RUN apt-get update && apt-get install -y --no-install-recommends \
|
18 |
+
build-essential \
|
19 |
+
&& apt-get clean \
|
20 |
+
&& rm -rf /var/lib/apt/lists/*
|
21 |
+
|
22 |
+
# 复制并安装requirements.txt | Copy and install requirements.txt
|
23 |
+
COPY requirements.txt .
|
24 |
+
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
|
25 |
+
|
26 |
+
# 第二阶段:运行时环境 | Stage 2: Runtime environment
|
27 |
+
FROM python:${PYTHON_VERSION}-slim
|
28 |
+
|
29 |
+
# 添加构建信息标签 | Add build information labels
|
30 |
+
ARG BUILD_DATE
|
31 |
+
ARG VERSION
|
32 |
+
LABEL org.opencontainers.image.created="${BUILD_DATE}" \
|
33 |
+
org.opencontainers.image.version="${VERSION}" \
|
34 |
+
org.opencontainers.image.title="OWL Project" \
|
35 |
+
org.opencontainers.image.description="OWL Project Docker Image" \
|
36 |
+
org.opencontainers.image.source="https://github.com/yourusername/owl"
|
37 |
+
|
38 |
+
# 设置工作目录 | Set working directory
|
39 |
+
WORKDIR /app
|
40 |
+
|
41 |
+
# 设置pip镜像源以加速下载 | Set pip mirror to accelerate downloads
|
42 |
+
ARG PIP_INDEX_URL
|
43 |
+
RUN pip config set global.index-url ${PIP_INDEX_URL}
|
44 |
+
|
45 |
+
# 从builder阶段复制已安装的Python包 | Copy installed Python packages from builder stage
|
46 |
+
COPY --from=builder /install /usr/local
|
47 |
+
|
48 |
+
# 优化apt安装,减少层数 | Optimize apt installation, reduce layers
|
49 |
+
RUN apt-get update && apt-get install -y --no-install-recommends \
|
50 |
+
curl \
|
51 |
+
git \
|
52 |
+
ffmpeg \
|
53 |
+
libsm6 \
|
54 |
+
libxext6 \
|
55 |
+
# 添加xvfb和相关依赖 | Add xvfb and related dependencies
|
56 |
+
xvfb \
|
57 |
+
xauth \
|
58 |
+
x11-utils \
|
59 |
+
&& apt-get clean \
|
60 |
+
&& rm -rf /var/lib/apt/lists/*
|
61 |
+
|
62 |
+
# 安装 Playwright 依赖(使用国内镜像源) | Install Playwright dependencies (using Chinese mirror)
|
63 |
+
ENV PLAYWRIGHT_BROWSERS_PATH=/root/.cache/ms-playwright
|
64 |
+
ARG PLAYWRIGHT_DOWNLOAD_HOST
|
65 |
+
ENV PLAYWRIGHT_DOWNLOAD_HOST=${PLAYWRIGHT_DOWNLOAD_HOST}
|
66 |
+
RUN pip install --no-cache-dir playwright && \
|
67 |
+
playwright install --with-deps chromium
|
68 |
+
|
69 |
+
# 创建非root用户 | Create non-root user
|
70 |
+
RUN groupadd -r owl && useradd -r -g owl -m owl
|
71 |
+
|
72 |
+
# 复制项目文件 | Copy project files
|
73 |
+
COPY owl/ ./owl/
|
74 |
+
COPY licenses/ ./licenses/
|
75 |
+
COPY assets/ ./assets/
|
76 |
+
COPY README.md .
|
77 |
+
COPY README_zh.md .
|
78 |
+
|
79 |
+
# 设置环境变量文件 | Set environment variables file
|
80 |
+
COPY owl/.env_template ./owl/.env
|
81 |
+
|
82 |
+
# 创建启动脚本 | Create startup script
|
83 |
+
RUN echo '#!/bin/bash\nxvfb-run --auto-servernum --server-args="-screen 0 1280x960x24" python "$@"' > /usr/local/bin/xvfb-python && \
|
84 |
+
chmod +x /usr/local/bin/xvfb-python
|
85 |
+
|
86 |
+
# 创建欢迎脚本 | Create welcome script
|
87 |
+
RUN echo '#!/bin/bash\necho "欢迎使用OWL项目Docker环境!"\necho "Welcome to OWL Project Docker environment!"\necho ""\necho "可用的脚本 | Available scripts:"\nls -1 *.py | grep -v "__" | sed "s/^/- /"\necho ""\necho "运行示例 | Run examples:"\necho " xvfb-python run.py # 运行默认脚本 | Run default script"\necho " xvfb-python run_deepseek_example.py # 运行DeepSeek示例 | Run DeepSeek example"\necho ""\necho "或者使用自定义查询 | Or use custom query:"\necho " xvfb-python run.py \"你的问题 | Your question\""\necho ""' > /usr/local/bin/owl-welcome && \
|
88 |
+
chmod +x /usr/local/bin/owl-welcome
|
89 |
+
|
90 |
+
# 设置工作目录 | Set working directory
|
91 |
+
WORKDIR /app/owl
|
92 |
+
|
93 |
+
# 设置适当的权限 | Set appropriate permissions
|
94 |
+
RUN chown -R owl:owl /app
|
95 |
+
RUN mkdir -p /root/.cache && chown -R owl:owl /root/.cache
|
96 |
+
|
97 |
+
# 切换到非root用户 | Switch to non-root user
|
98 |
+
# 注意:如果需要访问/dev/shm,可能仍需要root用户 | Note: If you need to access /dev/shm, you may still need root user
|
99 |
+
# USER owl
|
100 |
+
|
101 |
+
# 添加健康检查 | Add health check
|
102 |
+
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
103 |
+
CMD python -c "import sys; sys.exit(0 if __import__('os').path.exists('/app/owl') else 1)"
|
104 |
+
|
105 |
+
# 容器启动命令 | Container startup command
|
106 |
+
CMD ["/bin/bash", "-c", "owl-welcome && /bin/bash"]
|
.container/build_docker.bat
ADDED
@@ -0,0 +1,147 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
@echo off
|
2 |
+
setlocal enabledelayedexpansion
|
3 |
+
|
4 |
+
echo 在Windows上构建Docker镜像... | Building Docker image on Windows...
|
5 |
+
|
6 |
+
REM 设置配置变量 | Set configuration variables
|
7 |
+
set CACHE_DIR=.docker-cache\pip
|
8 |
+
set BUILD_ARGS=--build-arg BUILDKIT_INLINE_CACHE=1
|
9 |
+
set COMPOSE_FILE=docker-compose.yml
|
10 |
+
|
11 |
+
REM 解析命令行参数 | Parse command line arguments
|
12 |
+
set CLEAN_CACHE=0
|
13 |
+
set REBUILD=0
|
14 |
+
set SERVICE=
|
15 |
+
|
16 |
+
:parse_args
|
17 |
+
if "%~1"=="" goto :end_parse_args
|
18 |
+
if /i "%~1"=="--clean" (
|
19 |
+
set CLEAN_CACHE=1
|
20 |
+
shift
|
21 |
+
goto :parse_args
|
22 |
+
)
|
23 |
+
if /i "%~1"=="--rebuild" (
|
24 |
+
set REBUILD=1
|
25 |
+
shift
|
26 |
+
goto :parse_args
|
27 |
+
)
|
28 |
+
if /i "%~1"=="--service" (
|
29 |
+
set SERVICE=%~2
|
30 |
+
shift
|
31 |
+
shift
|
32 |
+
goto :parse_args
|
33 |
+
)
|
34 |
+
if /i "%~1"=="--help" (
|
35 |
+
echo 用法 | Usage: build_docker.bat [选项 | options]
|
36 |
+
echo 选项 | Options:
|
37 |
+
echo --clean 清理缓存目录 | Clean cache directory
|
38 |
+
echo --rebuild 强制重新构建镜像 | Force rebuild image
|
39 |
+
echo --service 指定要构建的服务名称 | Specify service name to build
|
40 |
+
echo --help 显示此帮助信息 | Show this help message
|
41 |
+
exit /b 0
|
42 |
+
)
|
43 |
+
shift
|
44 |
+
goto :parse_args
|
45 |
+
:end_parse_args
|
46 |
+
|
47 |
+
REM 检查Docker是否安装 | Check if Docker is installed
|
48 |
+
where docker >nul 2>nul
|
49 |
+
if %ERRORLEVEL% NEQ 0 (
|
50 |
+
echo 错误 | Error: Docker未安装 | Docker not installed
|
51 |
+
echo 请先安装Docker Desktop | Please install Docker Desktop first: https://docs.docker.com/desktop/install/windows-install/
|
52 |
+
pause
|
53 |
+
exit /b 1
|
54 |
+
)
|
55 |
+
|
56 |
+
REM 检查Docker是否运行 | Check if Docker is running
|
57 |
+
docker info >nul 2>nul
|
58 |
+
if %ERRORLEVEL% NEQ 0 (
|
59 |
+
echo 错误 | Error: Docker未运行 | Docker not running
|
60 |
+
echo 请启动Docker Desktop应用程序 | Please start Docker Desktop application
|
61 |
+
pause
|
62 |
+
exit /b 1
|
63 |
+
)
|
64 |
+
|
65 |
+
REM 检查docker-compose.yml文件是否存在 | Check if docker-compose.yml file exists
|
66 |
+
if not exist "%COMPOSE_FILE%" (
|
67 |
+
echo 错误 | Error: 未找到%COMPOSE_FILE%文件 | %COMPOSE_FILE% file not found
|
68 |
+
echo 请确保在正确的目录中运行此脚本 | Please make sure you are running this script in the correct directory
|
69 |
+
pause
|
70 |
+
exit /b 1
|
71 |
+
)
|
72 |
+
|
73 |
+
REM 检查Docker Compose命令 | Check Docker Compose command
|
74 |
+
where docker-compose >nul 2>nul
|
75 |
+
if %ERRORLEVEL% EQU 0 (
|
76 |
+
set COMPOSE_CMD=docker-compose
|
77 |
+
) else (
|
78 |
+
echo 尝试使用新的docker compose命令... | Trying to use new docker compose command...
|
79 |
+
docker compose version >nul 2>nul
|
80 |
+
if %ERRORLEVEL% EQU 0 (
|
81 |
+
set COMPOSE_CMD=docker compose
|
82 |
+
) else (
|
83 |
+
echo 错误 | Error: 未找到Docker Compose命令 | Docker Compose command not found
|
84 |
+
echo 请确保Docker Desktop已正确安装 | Please make sure Docker Desktop is properly installed
|
85 |
+
pause
|
86 |
+
exit /b 1
|
87 |
+
)
|
88 |
+
)
|
89 |
+
|
90 |
+
REM 设置Docker BuildKit环境变量 | Set Docker BuildKit environment variables
|
91 |
+
set DOCKER_BUILDKIT=1
|
92 |
+
set COMPOSE_DOCKER_CLI_BUILD=1
|
93 |
+
|
94 |
+
echo 启用Docker BuildKit加速构建... | Enabling Docker BuildKit to accelerate build...
|
95 |
+
|
96 |
+
REM 清理缓存(如果指定) | Clean cache (if specified)
|
97 |
+
if %CLEAN_CACHE% EQU 1 (
|
98 |
+
echo 清理缓存目录... | Cleaning cache directory...
|
99 |
+
if exist "%CACHE_DIR%" rmdir /s /q "%CACHE_DIR%"
|
100 |
+
)
|
101 |
+
|
102 |
+
REM 创建缓存目录 | Create cache directory
|
103 |
+
if not exist "%CACHE_DIR%" (
|
104 |
+
echo 创建缓存目录... | Creating cache directory...
|
105 |
+
mkdir "%CACHE_DIR%"
|
106 |
+
)
|
107 |
+
|
108 |
+
REM 添加构建时间标记 | Add build time tag
|
109 |
+
for /f "tokens=2 delims==" %%a in ('wmic OS Get localdatetime /value') do set "dt=%%a"
|
110 |
+
set "YEAR=%dt:~0,4%"
|
111 |
+
set "MONTH=%dt:~4,2%"
|
112 |
+
set "DAY=%dt:~6,2%"
|
113 |
+
set "HOUR=%dt:~8,2%"
|
114 |
+
set "MINUTE=%dt:~10,2%"
|
115 |
+
set "BUILD_TIME=%YEAR%%MONTH%%DAY%_%HOUR%%MINUTE%"
|
116 |
+
set "BUILD_ARGS=%BUILD_ARGS% --build-arg BUILD_TIME=%BUILD_TIME%"
|
117 |
+
|
118 |
+
REM 构建Docker镜像 | Build Docker image
|
119 |
+
echo 开始构建Docker镜像... | Starting to build Docker image...
|
120 |
+
|
121 |
+
if "%SERVICE%"=="" (
|
122 |
+
if %REBUILD% EQU 1 (
|
123 |
+
echo 强制重新构建所有服务... | Force rebuilding all services...
|
124 |
+
%COMPOSE_CMD% build --no-cache %BUILD_ARGS%
|
125 |
+
) else (
|
126 |
+
%COMPOSE_CMD% build %BUILD_ARGS%
|
127 |
+
)
|
128 |
+
) else (
|
129 |
+
if %REBUILD% EQU 1 (
|
130 |
+
echo 强制重新构建服务 %SERVICE%... | Force rebuilding service %SERVICE%...
|
131 |
+
%COMPOSE_CMD% build --no-cache %BUILD_ARGS% %SERVICE%
|
132 |
+
) else (
|
133 |
+
echo 构建服务 %SERVICE%... | Building service %SERVICE%...
|
134 |
+
%COMPOSE_CMD% build %BUILD_ARGS% %SERVICE%
|
135 |
+
)
|
136 |
+
)
|
137 |
+
|
138 |
+
if %ERRORLEVEL% EQU 0 (
|
139 |
+
echo Docker镜像构建成功! | Docker image build successful!
|
140 |
+
echo 构建时间 | Build time: %BUILD_TIME%
|
141 |
+
echo 可以使用以下命令启动容器: | You can use the following command to start the container:
|
142 |
+
echo %COMPOSE_CMD% up -d
|
143 |
+
) else (
|
144 |
+
echo Docker镜像构建失败,请检查错误信息。 | Docker image build failed, please check error messages.
|
145 |
+
)
|
146 |
+
|
147 |
+
pause
|
.container/build_docker.sh
ADDED
@@ -0,0 +1,150 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
#!/bin/bash
|
2 |
+
|
3 |
+
# 设置配置变量 | Set configuration variables
|
4 |
+
CACHE_DIR=".docker-cache/pip"
|
5 |
+
BUILD_ARGS="--build-arg BUILDKIT_INLINE_CACHE=1"
|
6 |
+
COMPOSE_FILE="docker-compose.yml"
|
7 |
+
CLEAN_CACHE=0
|
8 |
+
REBUILD=0
|
9 |
+
SERVICE=""
|
10 |
+
|
11 |
+
# 解析命令行参数 | Parse command line arguments
|
12 |
+
while [[ $# -gt 0 ]]; do
|
13 |
+
case "$1" in
|
14 |
+
--clean)
|
15 |
+
CLEAN_CACHE=1
|
16 |
+
shift
|
17 |
+
;;
|
18 |
+
--rebuild)
|
19 |
+
REBUILD=1
|
20 |
+
shift
|
21 |
+
;;
|
22 |
+
--service)
|
23 |
+
SERVICE="$2"
|
24 |
+
shift 2
|
25 |
+
;;
|
26 |
+
--help)
|
27 |
+
echo "用法 | Usage: ./build_docker.sh [选项 | options]"
|
28 |
+
echo "选项 | Options:"
|
29 |
+
echo " --clean 清理缓存目录 | Clean cache directory"
|
30 |
+
echo " --rebuild 强制重新构建镜像 | Force rebuild image"
|
31 |
+
echo " --service 指定要构建的服务名称 | Specify service name to build"
|
32 |
+
echo " --help 显示此帮助信息 | Show this help message"
|
33 |
+
exit 0
|
34 |
+
;;
|
35 |
+
*)
|
36 |
+
echo "未知选项 | Unknown option: $1"
|
37 |
+
echo "使用 --help 查看帮助 | Use --help to see help"
|
38 |
+
exit 1
|
39 |
+
;;
|
40 |
+
esac
|
41 |
+
done
|
42 |
+
|
43 |
+
# 检测操作系统类型 | Detect operating system type
|
44 |
+
OS_TYPE=$(uname -s)
|
45 |
+
echo "检测到操作系统 | Detected OS: $OS_TYPE"
|
46 |
+
|
47 |
+
# 检查Docker是否安装 | Check if Docker is installed
|
48 |
+
if ! command -v docker &> /dev/null; then
|
49 |
+
echo "错误 | Error: Docker未安装 | Docker not installed"
|
50 |
+
echo "请先安装Docker | Please install Docker first: https://docs.docker.com/get-docker/"
|
51 |
+
exit 1
|
52 |
+
fi
|
53 |
+
|
54 |
+
# 检查Docker是否运行 | Check if Docker is running
|
55 |
+
if ! docker info &> /dev/null; then
|
56 |
+
echo "错误 | Error: Docker未运行 | Docker not running"
|
57 |
+
echo "请启动Docker服务 | Please start Docker service"
|
58 |
+
exit 1
|
59 |
+
fi
|
60 |
+
|
61 |
+
# 检查docker-compose.yml文件是否存在 | Check if docker-compose.yml file exists
|
62 |
+
if [ ! -f "$COMPOSE_FILE" ]; then
|
63 |
+
echo "错误 | Error: 未找到$COMPOSE_FILE文件 | $COMPOSE_FILE file not found"
|
64 |
+
echo "请确保在正确的目录中运行此脚本 | Please make sure you are running this script in the correct directory"
|
65 |
+
exit 1
|
66 |
+
fi
|
67 |
+
|
68 |
+
# 设置Docker BuildKit环境变量 | Set Docker BuildKit environment variables
|
69 |
+
export DOCKER_BUILDKIT=1
|
70 |
+
export COMPOSE_DOCKER_CLI_BUILD=1
|
71 |
+
|
72 |
+
echo "启用Docker BuildKit加速构建... | Enabling Docker BuildKit to accelerate build..."
|
73 |
+
|
74 |
+
# 清理缓存(如果指定) | Clean cache (if specified)
|
75 |
+
if [ $CLEAN_CACHE -eq 1 ]; then
|
76 |
+
echo "清理缓存目录... | Cleaning cache directory..."
|
77 |
+
rm -rf "$CACHE_DIR"
|
78 |
+
fi
|
79 |
+
|
80 |
+
# 创建缓存目录 | Create cache directory
|
81 |
+
mkdir -p "$CACHE_DIR"
|
82 |
+
|
83 |
+
# 添加构建时间标记 | Add build time tag
|
84 |
+
BUILD_TIME=$(date +"%Y%m%d_%H%M%S")
|
85 |
+
BUILD_ARGS="$BUILD_ARGS --build-arg BUILD_TIME=$BUILD_TIME"
|
86 |
+
|
87 |
+
# 获取脚本所在目录 | Get script directory
|
88 |
+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
89 |
+
# 获取项目根目录(脚本所在目录的父目录) | Get project root directory (parent directory of script directory)
|
90 |
+
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
91 |
+
|
92 |
+
echo "脚本目录 | Script directory: $SCRIPT_DIR"
|
93 |
+
echo "项目根目录 | Project root directory: $PROJECT_ROOT"
|
94 |
+
|
95 |
+
# 切换到项目根目录 | Change to project root directory
|
96 |
+
cd "$PROJECT_ROOT"
|
97 |
+
|
98 |
+
# 检查Docker Compose命令 | Check Docker Compose command
|
99 |
+
if command -v docker-compose &> /dev/null; then
|
100 |
+
COMPOSE_CMD="docker-compose"
|
101 |
+
echo "使用 docker-compose 命令 | Using docker-compose command"
|
102 |
+
elif docker compose version &> /dev/null; then
|
103 |
+
COMPOSE_CMD="docker compose"
|
104 |
+
echo "使用 docker compose 命令 | Using docker compose command"
|
105 |
+
else
|
106 |
+
echo "错误 | Error: 未找到Docker Compose命令 | Docker Compose command not found"
|
107 |
+
echo "请安装Docker Compose | Please install Docker Compose: https://docs.docker.com/compose/install/"
|
108 |
+
exit 1
|
109 |
+
fi
|
110 |
+
|
111 |
+
# 检测CPU核心数,用于并行构建 | Detect CPU cores for parallel build
|
112 |
+
CPU_CORES=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 2)
|
113 |
+
if [ $CPU_CORES -gt 2 ]; then
|
114 |
+
PARALLEL_FLAG="--parallel"
|
115 |
+
echo "检测到${CPU_CORES}个CPU核心,启用并行构建... | Detected ${CPU_CORES} CPU cores, enabling parallel build..."
|
116 |
+
else
|
117 |
+
PARALLEL_FLAG=""
|
118 |
+
fi
|
119 |
+
|
120 |
+
# 构建命令基础部分 | Base part of build command
|
121 |
+
BUILD_CMD="$COMPOSE_CMD -f \"$SCRIPT_DIR/docker-compose.yml\" build $PARALLEL_FLAG --build-arg BUILDKIT_INLINE_CACHE=1"
|
122 |
+
|
123 |
+
# 根据操作系统类型执行不同的命令 | Execute different commands based on OS type
|
124 |
+
if [[ "$OS_TYPE" == "Darwin" ]]; then
|
125 |
+
# macOS
|
126 |
+
echo "在macOS上构建Docker镜像... | Building Docker image on macOS..."
|
127 |
+
eval $BUILD_CMD
|
128 |
+
elif [[ "$OS_TYPE" == "Linux" ]]; then
|
129 |
+
# Linux
|
130 |
+
echo "在Linux上构建Docker镜像... | Building Docker image on Linux..."
|
131 |
+
eval $BUILD_CMD
|
132 |
+
elif [[ "$OS_TYPE" == MINGW* ]] || [[ "$OS_TYPE" == CYGWIN* ]] || [[ "$OS_TYPE" == MSYS* ]]; then
|
133 |
+
# Windows
|
134 |
+
echo "在Windows上构建Docker镜像... | Building Docker image on Windows..."
|
135 |
+
eval $BUILD_CMD
|
136 |
+
else
|
137 |
+
echo "未知操作系统,尝试使用标准命令构建... | Unknown OS, trying to build with standard command..."
|
138 |
+
eval $BUILD_CMD
|
139 |
+
fi
|
140 |
+
|
141 |
+
# 检查构建结果 | Check build result
|
142 |
+
if [ $? -eq 0 ]; then
|
143 |
+
echo "Docker镜像构建成功! | Docker image build successful!"
|
144 |
+
echo "构建时间 | Build time: $BUILD_TIME"
|
145 |
+
echo "可以使用以下命令启动容器: | You can use the following command to start the container:"
|
146 |
+
echo "$COMPOSE_CMD -f \"$SCRIPT_DIR/docker-compose.yml\" up -d"
|
147 |
+
else
|
148 |
+
echo "Docker镜像构建失败,请检查错误信息。 | Docker image build failed, please check error messages."
|
149 |
+
exit 1
|
150 |
+
fi
|
.container/check_docker.bat
ADDED
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
@echo off
|
2 |
+
echo 检查Docker环境... | Checking Docker environment...
|
3 |
+
|
4 |
+
REM 检查Docker是否安装 | Check if Docker is installed
|
5 |
+
where docker >nul 2>nul
|
6 |
+
if %ERRORLEVEL% NEQ 0 (
|
7 |
+
echo 错误 | Error: Docker未安装 | Docker not installed
|
8 |
+
echo 在Windows上安装Docker的方法 | How to install Docker on Windows:
|
9 |
+
echo 1. 访问 | Visit https://docs.docker.com/desktop/install/windows-install/ 下载Docker Desktop | to download Docker Desktop
|
10 |
+
echo 2. 安装并启动Docker Desktop | Install and start Docker Desktop
|
11 |
+
pause
|
12 |
+
exit /b 1
|
13 |
+
)
|
14 |
+
|
15 |
+
echo Docker已安装 | Docker is installed
|
16 |
+
|
17 |
+
REM 检查Docker Compose是否安装 | Check if Docker Compose is installed
|
18 |
+
where docker-compose >nul 2>nul
|
19 |
+
if %ERRORLEVEL% NEQ 0 (
|
20 |
+
echo 警告 | Warning: Docker-Compose未找到,尝试使用新的docker compose命令 | Docker-Compose not found, trying to use new docker compose command
|
21 |
+
docker compose version >nul 2>nul
|
22 |
+
if %ERRORLEVEL% NEQ 0 (
|
23 |
+
echo 错误 | Error: Docker Compose未安装 | Docker Compose not installed
|
24 |
+
echo Docker Desktop for Windows应该已包含Docker Compose | Docker Desktop for Windows should already include Docker Compose
|
25 |
+
echo 请确保Docker Desktop已正确安装 | Please make sure Docker Desktop is properly installed
|
26 |
+
pause
|
27 |
+
exit /b 1
|
28 |
+
) else (
|
29 |
+
echo 使用新的docker compose命令 | Using new docker compose command
|
30 |
+
set COMPOSE_CMD=docker compose
|
31 |
+
)
|
32 |
+
) else (
|
33 |
+
echo Docker-Compose已安装 | Docker-Compose is installed
|
34 |
+
set COMPOSE_CMD=docker-compose
|
35 |
+
)
|
36 |
+
|
37 |
+
REM 检查Docker是否正在运行 | Check if Docker is running
|
38 |
+
docker info >nul 2>nul
|
39 |
+
if %ERRORLEVEL% NEQ 0 (
|
40 |
+
echo 错误 | Error: Docker未运行 | Docker not running
|
41 |
+
echo 请启动Docker Desktop应用程序 | Please start Docker Desktop application
|
42 |
+
pause
|
43 |
+
exit /b 1
|
44 |
+
)
|
45 |
+
|
46 |
+
echo Docker正在运行 | Docker is running
|
47 |
+
|
48 |
+
REM 检查是否有.env文件 | Check if .env file exists
|
49 |
+
if not exist "owl\.env" (
|
50 |
+
echo 警告 | Warning: 未找到owl\.env文件 | owl\.env file not found
|
51 |
+
echo 请运行以下命令创建环境变量文件 | Please run the following command to create environment variable file:
|
52 |
+
echo copy owl\.env_template owl\.env
|
53 |
+
echo 然后编辑owl\.env文件,填写必要的API密钥 | Then edit owl\.env file and fill in necessary API keys
|
54 |
+
) else (
|
55 |
+
echo 环境变量文件已存在 | Environment variable file exists
|
56 |
+
)
|
57 |
+
|
58 |
+
echo 所有检查完成,您的系统已准备好构建和运行OWL项目的Docker容器 | All checks completed, your system is ready to build and run OWL project Docker container
|
59 |
+
echo 请运行以下命令构建Docker镜像 | Please run the following command to build Docker image:
|
60 |
+
echo %COMPOSE_CMD% build
|
61 |
+
|
62 |
+
pause
|
.container/check_docker.sh
ADDED
@@ -0,0 +1,92 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
#!/bin/bash
|
2 |
+
|
3 |
+
# 检测操作系统类型 | Detect operating system type
|
4 |
+
OS_TYPE=$(uname -s)
|
5 |
+
echo "检测到操作系统 | Detected OS: $OS_TYPE"
|
6 |
+
|
7 |
+
# 检查Docker是否安装 | Check if Docker is installed
|
8 |
+
if ! command -v docker &> /dev/null; then
|
9 |
+
echo "错误 | Error: Docker未安装 | Docker not installed"
|
10 |
+
|
11 |
+
if [[ "$OS_TYPE" == "Darwin" ]]; then
|
12 |
+
echo "在macOS上安装Docker的方法 | How to install Docker on macOS:"
|
13 |
+
echo "1. 访问 | Visit https://docs.docker.com/desktop/install/mac-install/ 下载Docker Desktop | to download Docker Desktop"
|
14 |
+
echo "2. 安装并启动Docker Desktop | Install and start Docker Desktop"
|
15 |
+
elif [[ "$OS_TYPE" == "Linux" ]]; then
|
16 |
+
echo "在Linux上安装Docker的方法 | How to install Docker on Linux:"
|
17 |
+
echo "1. 运行以下命令 | Run the following commands:"
|
18 |
+
echo " sudo apt-get update"
|
19 |
+
echo " sudo apt-get install docker.io docker-compose"
|
20 |
+
echo "2. 启动Docker服务 | Start Docker service:"
|
21 |
+
echo " sudo systemctl start docker"
|
22 |
+
echo " sudo systemctl enable docker"
|
23 |
+
elif [[ "$OS_TYPE" == MINGW* ]] || [[ "$OS_TYPE" == CYGWIN* ]] || [[ "$OS_TYPE" == MSYS* ]]; then
|
24 |
+
echo "在Windows上安装Docker的方法 | How to install Docker on Windows:"
|
25 |
+
echo "1. 访问 | Visit https://docs.docker.com/desktop/install/windows-install/ 下载Docker Desktop | to download Docker Desktop"
|
26 |
+
echo "2. 安装并启动Docker Desktop | Install and start Docker Desktop"
|
27 |
+
fi
|
28 |
+
|
29 |
+
exit 1
|
30 |
+
fi
|
31 |
+
|
32 |
+
echo "Docker已安装 | Docker is installed"
|
33 |
+
|
34 |
+
# 检查Docker Compose是否安装 | Check if Docker Compose is installed
|
35 |
+
if ! command -v docker-compose &> /dev/null; then
|
36 |
+
echo "错误 | Error: Docker Compose未安装 | Docker Compose not installed"
|
37 |
+
|
38 |
+
if [[ "$OS_TYPE" == "Darwin" ]]; then
|
39 |
+
echo "Docker Desktop for Mac已包含Docker Compose | Docker Desktop for Mac already includes Docker Compose"
|
40 |
+
elif [[ "$OS_TYPE" == "Linux" ]]; then
|
41 |
+
echo "在Linux上安装Docker Compose的方法 | How to install Docker Compose on Linux:"
|
42 |
+
echo "1. 运行以下命令 | Run the following command:"
|
43 |
+
echo " sudo apt-get install docker-compose"
|
44 |
+
elif [[ "$OS_TYPE" == MINGW* ]] || [[ "$OS_TYPE" == CYGWIN* ]] || [[ "$OS_TYPE" == MSYS* ]]; then
|
45 |
+
echo "Docker Desktop for Windows已包含Docker Compose | Docker Desktop for Windows already includes Docker Compose"
|
46 |
+
fi
|
47 |
+
|
48 |
+
exit 1
|
49 |
+
fi
|
50 |
+
|
51 |
+
echo "Docker Compose已安装 | Docker Compose is installed"
|
52 |
+
|
53 |
+
# 检查Docker是否正在运行 | Check if Docker is running
|
54 |
+
if ! docker info &> /dev/null; then
|
55 |
+
echo "错误 | Error: Docker未运行 | Docker not running"
|
56 |
+
|
57 |
+
if [[ "$OS_TYPE" == "Darwin" ]]; then
|
58 |
+
echo "请启动Docker Desktop应用程序 | Please start Docker Desktop application"
|
59 |
+
elif [[ "$OS_TYPE" == "Linux" ]]; then
|
60 |
+
echo "请运行以下命令启动Docker服务 | Please run the following command to start Docker service:"
|
61 |
+
echo "sudo systemctl start docker"
|
62 |
+
elif [[ "$OS_TYPE" == MINGW* ]] || [[ "$OS_TYPE" == CYGWIN* ]] || [[ "$OS_TYPE" == MSYS* ]]; then
|
63 |
+
echo "请启动Docker Desktop应用程序 | Please start Docker Desktop application"
|
64 |
+
fi
|
65 |
+
|
66 |
+
exit 1
|
67 |
+
fi
|
68 |
+
|
69 |
+
echo "Docker正在运行 | Docker is running"
|
70 |
+
|
71 |
+
# 检查是否有足够的磁盘空间 | Check if there is enough disk space
|
72 |
+
FREE_SPACE=$(df -h . | awk 'NR==2 {print $4}')
|
73 |
+
echo "可用磁盘空间 | Available disk space: $FREE_SPACE"
|
74 |
+
|
75 |
+
# 检查是否有.env文件 | Check if .env file exists
|
76 |
+
if [ ! -f "owl/.env" ]; then
|
77 |
+
echo "警告 | Warning: 未找到owl/.env文件 | owl/.env file not found"
|
78 |
+
echo "请运行以下命令创建环境变量文件 | Please run the following command to create environment variable file:"
|
79 |
+
echo "cp owl/.env_template owl/.env"
|
80 |
+
echo "然后编辑owl/.env文件,填写必要的API密钥 | Then edit owl/.env file and fill in necessary API keys"
|
81 |
+
else
|
82 |
+
echo "环境变量文件已存在 | Environment variable file exists"
|
83 |
+
fi
|
84 |
+
|
85 |
+
echo "所有检查完成,您的系统已准备好构建和运行OWL项目的Docker容器 | All checks completed, your system is ready to build and run OWL project Docker container"
|
86 |
+
echo "请运行以下命令构建Docker镜像 | Please run the following command to build Docker image:"
|
87 |
+
|
88 |
+
if [[ "$OS_TYPE" == MINGW* ]] || [[ "$OS_TYPE" == CYGWIN* ]] || [[ "$OS_TYPE" == MSYS* ]]; then
|
89 |
+
echo "build_docker.bat"
|
90 |
+
else
|
91 |
+
echo "./build_docker.sh"
|
92 |
+
fi
|
.container/docker-compose.yml
ADDED
@@ -0,0 +1,52 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
services:
|
2 |
+
owl:
|
3 |
+
build:
|
4 |
+
context: ..
|
5 |
+
dockerfile: .container/Dockerfile
|
6 |
+
args:
|
7 |
+
# 构建参数 | Build arguments
|
8 |
+
BUILDKIT_INLINE_CACHE: 1
|
9 |
+
# 使用BuildKit加速构建 | Use BuildKit to accelerate build
|
10 |
+
cache_from:
|
11 |
+
- python:3.10-slim
|
12 |
+
volumes:
|
13 |
+
# 挂载.env文件,方便配置API密钥 | Mount .env file for easy API key configuration
|
14 |
+
- ./owl/.env:/app/owl/.env
|
15 |
+
# 可选:挂载数据目录 | Optional: Mount data directory
|
16 |
+
- ./data:/app/data
|
17 |
+
# 挂载缓存目录,避免重复下载 | Mount cache directories to avoid repeated downloads
|
18 |
+
- playwright-cache:/root/.cache/ms-playwright
|
19 |
+
- pip-cache:/root/.pip/cache
|
20 |
+
environment:
|
21 |
+
# 可以在这里设置环境变量,覆盖.env文件中的设置 | Set environment variables here to override settings in .env file
|
22 |
+
- OPENAI_API_KEY=${OPENAI_API_KEY}
|
23 |
+
# 添加显示相关的环境变量 | Add display-related environment variables
|
24 |
+
- DISPLAY=:99
|
25 |
+
- PLAYWRIGHT_BROWSERS_PATH=/root/.cache/ms-playwright
|
26 |
+
# 设置Python不生成.pyc文件,减少磁盘IO | Set Python to not generate .pyc files, reduce disk IO
|
27 |
+
- PYTHONDONTWRITEBYTECODE=1
|
28 |
+
# 设置Python不缓冲输出,方便查看日志 | Set Python to not buffer output for easier log viewing
|
29 |
+
- PYTHONUNBUFFERED=1
|
30 |
+
# 设置终端颜色 | Set terminal color
|
31 |
+
- TERM=xterm-256color
|
32 |
+
# 启用pip缓存 | Enable pip cache
|
33 |
+
- PIP_CACHE_DIR=/root/.pip/cache
|
34 |
+
ports:
|
35 |
+
# 如果项目有Web界面,可以映射端口 | If the project has a web interface, map ports
|
36 |
+
- "8000:8000"
|
37 |
+
# 使用交互模式运行容器 | Run container in interactive mode
|
38 |
+
stdin_open: true
|
39 |
+
tty: true
|
40 |
+
# 添加共享内存大小,提高浏览器性能 | Add shared memory size to improve browser performance
|
41 |
+
shm_size: 2gb
|
42 |
+
# 设置资源限制 | Set resource limits
|
43 |
+
deploy:
|
44 |
+
resources:
|
45 |
+
limits:
|
46 |
+
cpus: '2'
|
47 |
+
memory: 4G
|
48 |
+
|
49 |
+
# 定义持久化卷,用于缓存 | Define persistent volumes for caching
|
50 |
+
volumes:
|
51 |
+
playwright-cache:
|
52 |
+
pip-cache:
|
.container/run_in_docker.bat
ADDED
@@ -0,0 +1,116 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
@echo off
|
2 |
+
setlocal enabledelayedexpansion
|
3 |
+
|
4 |
+
REM 定义配置变量 | Define configuration variables
|
5 |
+
set SERVICE_NAME=owl
|
6 |
+
set PYTHON_CMD=xvfb-python
|
7 |
+
set MAX_WAIT_SECONDS=60
|
8 |
+
set CHECK_INTERVAL_SECONDS=2
|
9 |
+
|
10 |
+
REM 检查参数 | Check parameters
|
11 |
+
if "%~1"=="" (
|
12 |
+
echo 用法 | Usage: run_in_docker.bat [脚本名称 | script name] "你的问题 | your question"
|
13 |
+
echo 例如 | Example: run_in_docker.bat run.py "什么是人工智能? | What is artificial intelligence?"
|
14 |
+
echo 或者 | Or: run_in_docker.bat run_deepseek_example.py "什么是人工智能? | What is artificial intelligence?"
|
15 |
+
echo 如果不指定脚本名称,默认使用 run.py | If script name is not specified, run.py will be used by default
|
16 |
+
exit /b 1
|
17 |
+
)
|
18 |
+
|
19 |
+
REM 判断第一个参数是否是脚本名称 | Determine if the first parameter is a script name
|
20 |
+
set SCRIPT_NAME=%~1
|
21 |
+
set QUERY=%~2
|
22 |
+
|
23 |
+
if "!SCRIPT_NAME:~-3!"==".py" (
|
24 |
+
REM 如果提供了第二个参数,则为查询内容 | If a second parameter is provided, it's the query content
|
25 |
+
if "!QUERY!"=="" (
|
26 |
+
echo 请提供查询参数,例如 | Please provide query parameter, e.g.: run_in_docker.bat !SCRIPT_NAME! "你的问题 | your question"
|
27 |
+
exit /b 1
|
28 |
+
)
|
29 |
+
) else (
|
30 |
+
REM 如果第一个参数不是脚本名称,则默认使用 run.py | If the first parameter is not a script name, use run.py by default
|
31 |
+
set QUERY=!SCRIPT_NAME!
|
32 |
+
set SCRIPT_NAME=run.py
|
33 |
+
)
|
34 |
+
|
35 |
+
REM 检查脚本是否存在 | Check if the script exists
|
36 |
+
if not exist "owl\!SCRIPT_NAME!" (
|
37 |
+
echo 错误 | Error: 脚本 | Script 'owl\!SCRIPT_NAME!' 不存在 | does not exist
|
38 |
+
echo 可用的脚本有 | Available scripts:
|
39 |
+
dir /b owl\*.py | findstr /v "__"
|
40 |
+
exit /b 1
|
41 |
+
)
|
42 |
+
|
43 |
+
echo 使用脚本 | Using script: !SCRIPT_NAME!
|
44 |
+
echo 查询内容 | Query content: !QUERY!
|
45 |
+
|
46 |
+
REM 从docker-compose.yml获取服务名称(如果文件存在) | Get service name from docker-compose.yml (if file exists)
|
47 |
+
if exist ".container\docker-compose.yml" (
|
48 |
+
for /f "tokens=*" %%a in ('findstr /r "^ [a-zA-Z0-9_-]*:" .container\docker-compose.yml') do (
|
49 |
+
set line=%%a
|
50 |
+
set service=!line:~2,-1!
|
51 |
+
if not "!service!"=="" (
|
52 |
+
REM 使用第一个找到的服务名称 | Use the first service name found
|
53 |
+
set SERVICE_NAME=!service!
|
54 |
+
echo 从docker-compose.yml检测到服务名称 | Detected service name from docker-compose.yml: !SERVICE_NAME!
|
55 |
+
goto :found_service
|
56 |
+
)
|
57 |
+
)
|
58 |
+
)
|
59 |
+
:found_service
|
60 |
+
|
61 |
+
REM 确保Docker容器正在运行 | Ensure Docker container is running
|
62 |
+
docker-compose ps | findstr "!SERVICE_NAME!.*Up" > nul
|
63 |
+
if errorlevel 1 (
|
64 |
+
echo 启动Docker容器... | Starting Docker container...
|
65 |
+
docker-compose up -d
|
66 |
+
|
67 |
+
REM 使用循环检查容器是否就绪 | Use loop to check if container is ready
|
68 |
+
echo 等待容器启动... | Waiting for container to start...
|
69 |
+
set /a total_wait=0
|
70 |
+
|
71 |
+
:wait_loop
|
72 |
+
timeout /t !CHECK_INTERVAL_SECONDS! /nobreak > nul
|
73 |
+
set /a total_wait+=!CHECK_INTERVAL_SECONDS!
|
74 |
+
|
75 |
+
docker-compose ps | findstr "!SERVICE_NAME!.*Up" > nul
|
76 |
+
if errorlevel 1 (
|
77 |
+
if !total_wait! LSS !MAX_WAIT_SECONDS! (
|
78 |
+
echo 容器尚未就绪,已等待!total_wait!秒,继续等待... | Container not ready yet, waited for !total_wait! seconds, continuing to wait...
|
79 |
+
goto :wait_loop
|
80 |
+
) else (
|
81 |
+
echo 错误 | Error:容器启动超时,已等待!MAX_WAIT_SECONDS!秒 | Container startup timeout, waited for !MAX_WAIT_SECONDS! seconds
|
82 |
+
echo 请检查Docker容器状态 | Please check Docker container status:docker-compose ps
|
83 |
+
exit /b 1
|
84 |
+
)
|
85 |
+
) else (
|
86 |
+
echo 容器已就绪,共等待了!total_wait!秒 | Container is ready, waited for !total_wait! seconds in total
|
87 |
+
)
|
88 |
+
)
|
89 |
+
|
90 |
+
REM 检查容器中是否存在xvfb-python命令 | Check if xvfb-python command exists in container
|
91 |
+
echo 检查容器中的命令... | Checking commands in container...
|
92 |
+
docker-compose exec -T !SERVICE_NAME! which !PYTHON_CMD! > nul 2>&1
|
93 |
+
if errorlevel 1 (
|
94 |
+
echo 警告 | Warning:容器中未找到!PYTHON_CMD!命令,尝试使用python替代 | !PYTHON_CMD! command not found in container, trying to use python instead
|
95 |
+
set PYTHON_CMD=python
|
96 |
+
|
97 |
+
REM 检查python命令是否存在 | Check if python command exists
|
98 |
+
docker-compose exec -T !SERVICE_NAME! which python > nul 2>&1
|
99 |
+
if errorlevel 1 (
|
100 |
+
echo 错误 | Error:容器中未找到python命令 | python command not found in container
|
101 |
+
echo 请检查容器配置 | Please check container configuration
|
102 |
+
exit /b 1
|
103 |
+
)
|
104 |
+
)
|
105 |
+
|
106 |
+
REM 在容器中运行指定的脚本,传递查询参数 | Run the specified script in container, passing query parameter
|
107 |
+
echo 在Docker容器中使用!PYTHON_CMD!运行脚本... | Running script in Docker container using !PYTHON_CMD!...
|
108 |
+
docker-compose exec -T !SERVICE_NAME! !PYTHON_CMD! !SCRIPT_NAME! "!QUERY!"
|
109 |
+
|
110 |
+
if errorlevel 0 (
|
111 |
+
echo 查询完成! | Query completed!
|
112 |
+
) else (
|
113 |
+
echo 查询执行失败,请检查错误信息。 | Query execution failed, please check error messages.
|
114 |
+
)
|
115 |
+
|
116 |
+
pause
|
.container/run_in_docker.sh
ADDED
@@ -0,0 +1,135 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
#!/bin/bash
|
2 |
+
|
3 |
+
# 定义配置变量 | Define configuration variables
|
4 |
+
SERVICE_NAME="owl"
|
5 |
+
PYTHON_CMD="xvfb-python"
|
6 |
+
MAX_WAIT_SECONDS=60
|
7 |
+
CHECK_INTERVAL_SECONDS=2
|
8 |
+
|
9 |
+
# 检测操作系统类型 | Detect operating system type
|
10 |
+
OS_TYPE=$(uname -s)
|
11 |
+
echo "检测到操作系统 | Detected operating system: $OS_TYPE"
|
12 |
+
|
13 |
+
# 检查是否提供了查询参数 | Check if query parameters are provided
|
14 |
+
if [ $# -lt 1 ]; then
|
15 |
+
echo "用法 | Usage: ./run_in_docker.sh [脚本名称 | script name] '你的问题 | your question'"
|
16 |
+
echo "例如 | Example: ./run_in_docker.sh run.py '什么是人工智能? | What is artificial intelligence?'"
|
17 |
+
echo "或者 | Or: ./run_in_docker.sh run_deepseek_example.py '什么是人工智能? | What is artificial intelligence?'"
|
18 |
+
echo "如果不指定脚本名称,默认使用 run.py | If script name is not specified, run.py will be used by default"
|
19 |
+
exit 1
|
20 |
+
fi
|
21 |
+
|
22 |
+
# 判断第一个参数是否是脚本名称 | Determine if the first parameter is a script name
|
23 |
+
if [[ $1 == *.py ]]; then
|
24 |
+
SCRIPT_NAME="$1"
|
25 |
+
# 如果提供了第二个参数,则为查询内容 | If a second parameter is provided, it's the query content
|
26 |
+
if [ $# -ge 2 ]; then
|
27 |
+
QUERY="$2"
|
28 |
+
else
|
29 |
+
echo "请提供查询参数,例如 | Please provide query parameter, e.g.: ./run_in_docker.sh $SCRIPT_NAME '你的问题 | your question'"
|
30 |
+
exit 1
|
31 |
+
fi
|
32 |
+
else
|
33 |
+
# 如果第一个参数不是脚本名称,则默认使用 run.py | If the first parameter is not a script name, use run.py by default
|
34 |
+
SCRIPT_NAME="run.py"
|
35 |
+
QUERY="$1"
|
36 |
+
fi
|
37 |
+
|
38 |
+
# 检查脚本是否存在 | Check if the script exists
|
39 |
+
if [ ! -f "owl/$SCRIPT_NAME" ]; then
|
40 |
+
echo "错误 | Error: 脚本 | Script 'owl/$SCRIPT_NAME' 不存在 | does not exist"
|
41 |
+
echo "可用的脚本有 | Available scripts:"
|
42 |
+
if [[ "$OS_TYPE" == MINGW* ]] || [[ "$OS_TYPE" == CYGWIN* ]] || [[ "$OS_TYPE" == MSYS* ]]; then
|
43 |
+
find owl -name "*.py" | grep -v "__" | sed 's/\\/\//g'
|
44 |
+
else
|
45 |
+
ls -1 owl/*.py | grep -v "__"
|
46 |
+
fi
|
47 |
+
exit 1
|
48 |
+
fi
|
49 |
+
|
50 |
+
echo "使用脚本 | Using script: $SCRIPT_NAME"
|
51 |
+
echo "查询内容 | Query content: $QUERY"
|
52 |
+
|
53 |
+
# 从docker-compose.yml获取服务名称(如果文件存在) | Get service name from docker-compose.yml (if file exists)
|
54 |
+
if [ -f ".container/docker-compose.yml" ]; then
|
55 |
+
DETECTED_SERVICE=$(grep -E "^ [a-zA-Z0-9_-]*:" .container/docker-compose.yml | head -1 | sed 's/^ \(.*\):.*/\1/')
|
56 |
+
if [ ! -z "$DETECTED_SERVICE" ]; then
|
57 |
+
SERVICE_NAME="$DETECTED_SERVICE"
|
58 |
+
echo "从docker-compose.yml检测到服务名称 | Detected service name from docker-compose.yml: $SERVICE_NAME"
|
59 |
+
fi
|
60 |
+
fi
|
61 |
+
|
62 |
+
# 检查Docker Compose命令 | Check Docker Compose command
|
63 |
+
if command -v docker-compose &> /dev/null; then
|
64 |
+
COMPOSE_CMD="docker-compose"
|
65 |
+
elif docker compose version &> /dev/null; then
|
66 |
+
COMPOSE_CMD="docker compose"
|
67 |
+
else
|
68 |
+
echo "错误 | Error: 未找到Docker Compose命令 | Docker Compose command not found"
|
69 |
+
exit 1
|
70 |
+
fi
|
71 |
+
|
72 |
+
# 确保Docker容器正在运行 | Ensure Docker container is running
|
73 |
+
CONTAINER_RUNNING=$($COMPOSE_CMD ps | grep -c "$SERVICE_NAME.*Up" || true)
|
74 |
+
if [ "$CONTAINER_RUNNING" -eq 0 ]; then
|
75 |
+
echo "启动Docker容器... | Starting Docker container..."
|
76 |
+
$COMPOSE_CMD up -d
|
77 |
+
|
78 |
+
# 使用循环检查容器是否就绪 | Use loop to check if container is ready
|
79 |
+
echo "等待容器启动... | Waiting for container to start..."
|
80 |
+
TOTAL_WAIT=0
|
81 |
+
|
82 |
+
while [ $TOTAL_WAIT -lt $MAX_WAIT_SECONDS ]; do
|
83 |
+
sleep $CHECK_INTERVAL_SECONDS
|
84 |
+
TOTAL_WAIT=$((TOTAL_WAIT + CHECK_INTERVAL_SECONDS))
|
85 |
+
|
86 |
+
CONTAINER_RUNNING=$($COMPOSE_CMD ps | grep -c "$SERVICE_NAME.*Up" || true)
|
87 |
+
if [ "$CONTAINER_RUNNING" -gt 0 ]; then
|
88 |
+
echo "容器已就绪,共等待了 $TOTAL_WAIT 秒 | Container is ready, waited for $TOTAL_WAIT seconds in total"
|
89 |
+
break
|
90 |
+
else
|
91 |
+
echo "容器尚未就绪,已等待 $TOTAL_WAIT 秒,继续等待... | Container not ready yet, waited for $TOTAL_WAIT seconds, continuing to wait..."
|
92 |
+
fi
|
93 |
+
done
|
94 |
+
|
95 |
+
if [ "$CONTAINER_RUNNING" -eq 0 ]; then
|
96 |
+
echo "错误 | Error:容器启动超时,已等待 $MAX_WAIT_SECONDS 秒 | Container startup timeout, waited for $MAX_WAIT_SECONDS seconds"
|
97 |
+
echo "请检查Docker容器状态 | Please check Docker container status:$COMPOSE_CMD ps"
|
98 |
+
exit 1
|
99 |
+
fi
|
100 |
+
fi
|
101 |
+
|
102 |
+
# 检查容器中是否存在指定的Python命令 | Check if specified Python command exists in container
|
103 |
+
echo "检查容器中的命令... | Checking commands in container..."
|
104 |
+
if ! $COMPOSE_CMD exec -T $SERVICE_NAME which $PYTHON_CMD &> /dev/null; then
|
105 |
+
echo "警告 | Warning:容器中未找到 $PYTHON_CMD 命令,尝试使用python替代 | $PYTHON_CMD command not found in container, trying to use python instead"
|
106 |
+
PYTHON_CMD="python"
|
107 |
+
|
108 |
+
# 检查python命令是否存在 | Check if python command exists
|
109 |
+
if ! $COMPOSE_CMD exec -T $SERVICE_NAME which python &> /dev/null; then
|
110 |
+
echo "错误 | Error:容器中未找到python命令 | python command not found in container"
|
111 |
+
echo "请检查容器配置 | Please check container configuration"
|
112 |
+
exit 1
|
113 |
+
fi
|
114 |
+
fi
|
115 |
+
|
116 |
+
# 在容器中运行指定的脚本,传递查询参数 | Run the specified script in container, passing query parameter
|
117 |
+
echo "在Docker容器中使用 $PYTHON_CMD 运行脚本... | Running script in Docker container using $PYTHON_CMD..."
|
118 |
+
|
119 |
+
# 根据操作系统类型执行不同的命令 | Execute different commands based on operating system type
|
120 |
+
if [[ "$OS_TYPE" == MINGW* ]] || [[ "$OS_TYPE" == CYGWIN* ]] || [[ "$OS_TYPE" == MSYS* ]]; then
|
121 |
+
# Windows可能需要特殊处理引号 | Windows may need special handling for quotes
|
122 |
+
winpty $COMPOSE_CMD exec -T $SERVICE_NAME $PYTHON_CMD $SCRIPT_NAME "$QUERY"
|
123 |
+
RESULT=$?
|
124 |
+
else
|
125 |
+
# macOS 或 Linux | macOS or Linux
|
126 |
+
$COMPOSE_CMD exec -T $SERVICE_NAME $PYTHON_CMD $SCRIPT_NAME "$QUERY"
|
127 |
+
RESULT=$?
|
128 |
+
fi
|
129 |
+
|
130 |
+
# 检查命令执行结果 | Check command execution result
|
131 |
+
if [ $RESULT -eq 0 ]; then
|
132 |
+
echo "查询完成! | Query completed!"
|
133 |
+
else
|
134 |
+
echo "查询执行失败,请检查错误信息。 | Query execution failed, please check error messages."
|
135 |
+
fi
|
.gitattributes
CHANGED
@@ -33,3 +33,13 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
36 |
+
assets/community.png filter=lfs diff=lfs merge=lfs -text
|
37 |
+
assets/community_2.png filter=lfs diff=lfs merge=lfs -text
|
38 |
+
assets/community_3.jpg filter=lfs diff=lfs merge=lfs -text
|
39 |
+
assets/community_4.jpg filter=lfs diff=lfs merge=lfs -text
|
40 |
+
assets/community_5.jpg filter=lfs diff=lfs merge=lfs -text
|
41 |
+
assets/community_6.jpg filter=lfs diff=lfs merge=lfs -text
|
42 |
+
assets/community_6.png filter=lfs diff=lfs merge=lfs -text
|
43 |
+
assets/meetup.jpg filter=lfs diff=lfs merge=lfs -text
|
44 |
+
assets/owl_architecture.png filter=lfs diff=lfs merge=lfs -text
|
45 |
+
assets/qr_code.jpg filter=lfs diff=lfs merge=lfs -text
|
.gitignore
ADDED
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Python
|
2 |
+
__pycache__/
|
3 |
+
**/__pycache__/
|
4 |
+
*/__pycache__/*
|
5 |
+
*.py[cod]
|
6 |
+
*$py.class
|
7 |
+
*.so
|
8 |
+
.Python
|
9 |
+
build/
|
10 |
+
develop-eggs/
|
11 |
+
.dist
|
12 |
+
downloads/
|
13 |
+
eggs/
|
14 |
+
.eggs/
|
15 |
+
lib/
|
16 |
+
lib64/
|
17 |
+
parts/
|
18 |
+
sdist/
|
19 |
+
var/
|
20 |
+
wheels/
|
21 |
+
*.egg-info/
|
22 |
+
.installed.cfg
|
23 |
+
*.egg
|
24 |
+
|
25 |
+
# Virtual Environment
|
26 |
+
venv/
|
27 |
+
env/
|
28 |
+
ENV/
|
29 |
+
.env
|
30 |
+
|
31 |
+
# IDE
|
32 |
+
.idea/
|
33 |
+
.vscode/
|
34 |
+
*.swp
|
35 |
+
*.swo
|
36 |
+
.DS_Store
|
37 |
+
|
38 |
+
# Project specific
|
39 |
+
owl/data
|
40 |
+
owl/tmp
|
41 |
+
owl/.env
|
42 |
+
owl/utils/__pycache__/
|
43 |
+
|
44 |
+
# Logs
|
45 |
+
*.log
|
46 |
+
logs/
|
47 |
+
log/
|
48 |
+
|
49 |
+
# Coverage reports
|
50 |
+
htmlcov/
|
51 |
+
.tox/
|
52 |
+
.coverage
|
53 |
+
.coverage.*
|
54 |
+
.cache
|
55 |
+
coverage.xml
|
56 |
+
*.cover
|
57 |
+
|
58 |
+
owl/camel/types/__pycache__/
|
59 |
+
owl/camel/__pycache__/
|
60 |
+
owl/camel/utils/__pycache_/
|
.gradio/certificate.pem
ADDED
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
-----BEGIN CERTIFICATE-----
|
2 |
+
MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
|
3 |
+
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
|
4 |
+
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4
|
5 |
+
WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu
|
6 |
+
ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY
|
7 |
+
MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc
|
8 |
+
h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+
|
9 |
+
0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U
|
10 |
+
A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW
|
11 |
+
T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH
|
12 |
+
B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC
|
13 |
+
B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv
|
14 |
+
KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn
|
15 |
+
OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn
|
16 |
+
jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw
|
17 |
+
qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI
|
18 |
+
rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV
|
19 |
+
HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq
|
20 |
+
hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL
|
21 |
+
ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ
|
22 |
+
3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK
|
23 |
+
NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5
|
24 |
+
ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur
|
25 |
+
TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC
|
26 |
+
jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc
|
27 |
+
oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq
|
28 |
+
4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA
|
29 |
+
mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d
|
30 |
+
emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=
|
31 |
+
-----END CERTIFICATE-----
|
.pre-commit-config.yaml
ADDED
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
repos:
|
2 |
+
- repo: https://github.com/astral-sh/ruff-pre-commit
|
3 |
+
rev: 'v0.7.4'
|
4 |
+
hooks:
|
5 |
+
- id: ruff
|
6 |
+
args: [--fix, --exit-non-zero-on-fix, --show-fixes]
|
7 |
+
exclude: ^docs/cookbooks/ # Ignore files under docs/cookbooks
|
8 |
+
- id: ruff-format
|
9 |
+
exclude: ^docs/cookbooks/ # Ignore files under docs/cookbooks
|
10 |
+
|
11 |
+
- repo: local
|
12 |
+
hooks:
|
13 |
+
- id: mypy
|
14 |
+
name: Check mypy
|
15 |
+
entry: mypy --namespace-packages -p owl
|
16 |
+
language: python
|
17 |
+
types: [python]
|
18 |
+
pass_filenames: false
|
19 |
+
require_serial: true
|
20 |
+
exclude: ^docs/cookbooks/ # Ignore files under docs/cookbooks
|
21 |
+
|
22 |
+
- repo: local
|
23 |
+
hooks:
|
24 |
+
- id: check-license
|
25 |
+
name: Check License
|
26 |
+
entry: python licenses/update_license.py . licenses/license_template.txt
|
27 |
+
language: system
|
28 |
+
types: [python]
|
29 |
+
exclude: ^docs/cookbooks/ # Ignore files under docs/cookbooks
|
README.md
CHANGED
@@ -1,13 +1,392 @@
|
|
1 |
---
|
2 |
-
title:
|
3 |
-
|
4 |
-
colorFrom: yellow
|
5 |
-
colorTo: purple
|
6 |
sdk: gradio
|
7 |
-
sdk_version: 5.
|
8 |
-
app_file: app.py
|
9 |
-
pinned: false
|
10 |
-
short_description: owl
|
11 |
---
|
|
|
|
|
|
|
12 |
|
13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
title: owl
|
3 |
+
app_file: run_app.py
|
|
|
|
|
4 |
sdk: gradio
|
5 |
+
sdk_version: 5.23.1
|
|
|
|
|
|
|
6 |
---
|
7 |
+
<h1 align="center">
|
8 |
+
🦉 OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation
|
9 |
+
</h1>
|
10 |
|
11 |
+
|
12 |
+
<div align="center">
|
13 |
+
|
14 |
+
[![Documentation][docs-image]][docs-url]
|
15 |
+
[![Discord][discord-image]][discord-url]
|
16 |
+
[![X][x-image]][x-url]
|
17 |
+
[![Reddit][reddit-image]][reddit-url]
|
18 |
+
[![Wechat][wechat-image]][wechat-url]
|
19 |
+
[![Wechat][owl-image]][owl-url]
|
20 |
+
[![Hugging Face][huggingface-image]][huggingface-url]
|
21 |
+
[![Star][star-image]][star-url]
|
22 |
+
[![Package License][package-license-image]][package-license-url]
|
23 |
+
|
24 |
+
|
25 |
+
</div>
|
26 |
+
|
27 |
+
|
28 |
+
<hr>
|
29 |
+
|
30 |
+
<div align="center">
|
31 |
+
<h4 align="center">
|
32 |
+
|
33 |
+
[中文阅读](https://github.com/camel-ai/owl/tree/main/README_zh.md) |
|
34 |
+
[Community](https://github.com/camel-ai/owl#community) |
|
35 |
+
[Installation](#️-installation) |
|
36 |
+
[Examples](https://github.com/camel-ai/owl/tree/main/owl) |
|
37 |
+
[Paper](https://arxiv.org/abs/2303.17760) |
|
38 |
+
[Citation](https://github.com/camel-ai/owl#citation) |
|
39 |
+
[Contributing](https://github.com/camel-ai/owl/graphs/contributors) |
|
40 |
+
[CAMEL-AI](https://www.camel-ai.org/)
|
41 |
+
|
42 |
+
</h4>
|
43 |
+
|
44 |
+
<div align="center" style="background-color: #f0f7ff; padding: 10px; border-radius: 5px; margin: 15px 0;">
|
45 |
+
<h3 style="color: #1e88e5; margin: 0;">
|
46 |
+
🏆 OWL achieves <span style="color: #d81b60; font-weight: bold; font-size: 1.2em;">58.18</span> average score on GAIA benchmark and ranks <span style="color: #d81b60; font-weight: bold; font-size: 1.2em;">🏅️ #1</span> among open-source frameworks! 🏆
|
47 |
+
</h3>
|
48 |
+
</div>
|
49 |
+
|
50 |
+
<div align="center">
|
51 |
+
|
52 |
+
🦉 OWL is a cutting-edge framework for multi-agent collaboration that pushes the boundaries of task automation, built on top of the [CAMEL-AI Framework](https://github.com/camel-ai/camel).
|
53 |
+
|
54 |
+
<!-- OWL achieves **58.18** average score on [GAIA](https://huggingface.co/spaces/gaia-benchmark/leaderboard) benchmark and ranks 🏅️ #1 among open-source frameworks. -->
|
55 |
+
|
56 |
+
Our vision is to revolutionize how AI agents collaborate to solve real-world tasks. By leveraging dynamic agent interactions, OWL enables more natural, efficient, and robust task automation across diverse domains.
|
57 |
+
|
58 |
+
</div>
|
59 |
+
|
60 |
+

|
61 |
+
|
62 |
+
<br>
|
63 |
+
|
64 |
+
|
65 |
+
</div>
|
66 |
+
|
67 |
+
<!-- # Key Features -->
|
68 |
+
# 📋 Table of Contents
|
69 |
+
|
70 |
+
- [📋 Table of Contents](#-table-of-contents)
|
71 |
+
- [🔥 News](#-news)
|
72 |
+
- [🎬 Demo Video](#-demo-video)
|
73 |
+
- [✨️ Core Features](#-core-features)
|
74 |
+
- [🛠️ Installation](#️-installation)
|
75 |
+
- [**Clone the Github repository**](#clone-the-github-repository)
|
76 |
+
- [**Set up Environment**](#set-up-environment)
|
77 |
+
- [**Install Dependencies**](#install-dependencies)
|
78 |
+
- [**Setup Environment Variables**](#setup-environment-variables)
|
79 |
+
- [**Running with Docker**](#running-with-docker)
|
80 |
+
|
81 |
+
- [🚀 Quick Start](#-quick-start)
|
82 |
+
- [🌐 Web Interface](#-web-interface)
|
83 |
+
- [🧪 Experiments](#-experiments)
|
84 |
+
- [⏱️ Future Plans](#️-future-plans)
|
85 |
+
- [📄 License](#-license)
|
86 |
+
- [🖊️ Cite](#️-cite)
|
87 |
+
- [🔥 Community](#-community)
|
88 |
+
- [❓ FAQ](#-faq)
|
89 |
+
- [⭐ Star History](#-star-history)
|
90 |
+
|
91 |
+
|
92 |
+
# 🔥 News
|
93 |
+
|
94 |
+
- **[2025.03.07]**: We open-source the codebase of 🦉 OWL project.
|
95 |
+
|
96 |
+
# 🎬 Demo Video
|
97 |
+
|
98 |
+
https://private-user-images.githubusercontent.com/55657767/420211368-f29f477d-7eef-46da-8d7a-8f3bcf506da2.mp4
|
99 |
+
|
100 |
+
https://private-user-images.githubusercontent.com/55657767/420212194-e813fc05-136a-485f-8df3-f10d9b4e63ec.mp4
|
101 |
+
|
102 |
+
# ✨️ Core Features
|
103 |
+
|
104 |
+
- **Real-time Information Retrieval**: Leverage Wikipedia, Google Search, and other online sources for up-to-date information.
|
105 |
+
- **Multimodal Processing**: Support for handling internet or local videos, images, and audio data.
|
106 |
+
- **Browser Automation**: Utilize the Playwright framework for simulating browser interactions, including scrolling, clicking, input handling, downloading, navigation, and more.
|
107 |
+
- **Document Parsing**: Extract content from Word, Excel, PDF, and PowerPoint files, converting them into text or Markdown format.
|
108 |
+
- **Code Execution**: Write and execute Python code using interpreter.
|
109 |
+
- **Built-in Toolkits**: Access to a comprehensive set of built-in toolkits including ArxivToolkit, AudioAnalysisToolkit, CodeExecutionToolkit, DalleToolkit, DataCommonsToolkit, ExcelToolkit, GitHubToolkit, GoogleMapsToolkit, GoogleScholarToolkit, ImageAnalysisToolkit, MathToolkit, NetworkXToolkit, NotionToolkit, OpenAPIToolkit, RedditToolkit, SearchToolkit, SemanticScholarToolkit, SymPyToolkit, VideoAnalysisToolkit, WeatherToolkit, WebToolkit, and many more for specialized tasks.
|
110 |
+
|
111 |
+
# 🛠️ Installation
|
112 |
+
|
113 |
+
## Option 1: Using uv (Recommended)
|
114 |
+
|
115 |
+
```bash
|
116 |
+
# Clone github repo
|
117 |
+
git clone https://github.com/camel-ai/owl.git
|
118 |
+
|
119 |
+
# Change directory into project directory
|
120 |
+
cd owl
|
121 |
+
|
122 |
+
# Install uv if you don't have it already
|
123 |
+
pip install uv
|
124 |
+
|
125 |
+
# Create a virtual environment and install dependencies
|
126 |
+
# We support using Python 3.10, 3.11, 3.12
|
127 |
+
uv venv .venv --python=3.10
|
128 |
+
|
129 |
+
# Activate the virtual environment
|
130 |
+
# For macOS/Linux
|
131 |
+
source .venv/bin/activate
|
132 |
+
# For Windows
|
133 |
+
.venv\Scripts\activate
|
134 |
+
|
135 |
+
# Install CAMEL with all dependencies
|
136 |
+
uv pip install -e .
|
137 |
+
|
138 |
+
# Exit the virtual environment when done
|
139 |
+
deactivate
|
140 |
+
```
|
141 |
+
|
142 |
+
## Option 2: Using venv and pip
|
143 |
+
|
144 |
+
```bash
|
145 |
+
# Clone github repo
|
146 |
+
git clone https://github.com/camel-ai/owl.git
|
147 |
+
|
148 |
+
# Change directory into project directory
|
149 |
+
cd owl
|
150 |
+
|
151 |
+
# Create a virtual environment
|
152 |
+
# For Python 3.10 (also works with 3.11, 3.12)
|
153 |
+
python3.10 -m venv .venv
|
154 |
+
|
155 |
+
# Activate the virtual environment
|
156 |
+
# For macOS/Linux
|
157 |
+
source .venv/bin/activate
|
158 |
+
# For Windows
|
159 |
+
.venv\Scripts\activate
|
160 |
+
|
161 |
+
# Install from requirements.txt
|
162 |
+
pip install -r requirements.txt
|
163 |
+
```
|
164 |
+
|
165 |
+
## Option 3: Using conda
|
166 |
+
|
167 |
+
```bash
|
168 |
+
# Clone github repo
|
169 |
+
git clone https://github.com/camel-ai/owl.git
|
170 |
+
|
171 |
+
# Change directory into project directory
|
172 |
+
cd owl
|
173 |
+
|
174 |
+
# Create a conda environment
|
175 |
+
conda create -n owl python=3.10
|
176 |
+
|
177 |
+
# Activate the conda environment
|
178 |
+
conda activate owl
|
179 |
+
|
180 |
+
# Option 1: Install as a package (recommended)
|
181 |
+
pip install -e .
|
182 |
+
|
183 |
+
# Option 2: Install from requirements.txt
|
184 |
+
pip install -r requirements.txt
|
185 |
+
|
186 |
+
# Exit the conda environment when done
|
187 |
+
conda deactivate
|
188 |
+
```
|
189 |
+
|
190 |
+
## **Setup Environment Variables**
|
191 |
+
|
192 |
+
In the `owl/.env_template` file, you will find all the necessary API keys along with the websites where you can register for each service. To use these API services, follow these steps:
|
193 |
+
|
194 |
+
1. *Copy and Rename*: Duplicate the `.env_template` file and rename the copy to `.env`.
|
195 |
+
```bash
|
196 |
+
cp owl/.env_template .env
|
197 |
+
```
|
198 |
+
2. *Fill in Your Keys*: Open the `.env` file and insert your API keys in the corresponding fields. (For the minimal example (`run_mini.py`), you only need to configure the LLM API key (e.g., OPENAI_API_KEY).)
|
199 |
+
3. *For using more other models*: please refer to our CAMEL models docs:https://docs.camel-ai.org/key_modules/models.html#supported-model-platforms-in-camel
|
200 |
+
|
201 |
+
|
202 |
+
> **Note**: For optimal performance, we strongly recommend using OpenAI models. Our experiments show that other models may result in significantly lower performance on complex tasks and benchmarks.
|
203 |
+
|
204 |
+
## **Running with Docker**
|
205 |
+
|
206 |
+
```bash
|
207 |
+
# Clone the repository
|
208 |
+
git clone https://github.com/camel-ai/owl.git
|
209 |
+
cd owl
|
210 |
+
|
211 |
+
# Configure environment variables
|
212 |
+
cp owl/.env_template owl/.env
|
213 |
+
# Edit the .env file and fill in your API keys
|
214 |
+
|
215 |
+
|
216 |
+
# Option 1: Using docker-compose directly
|
217 |
+
cd .container
|
218 |
+
docker-compose up -d
|
219 |
+
# Run OWL inside the container
|
220 |
+
docker-compose exec owl bash -c "xvfb-python run.py"
|
221 |
+
|
222 |
+
# Option 2: Build and run using the provided scripts
|
223 |
+
cd .container
|
224 |
+
chmod +x build_docker.sh
|
225 |
+
./build_docker.sh
|
226 |
+
# Run OWL inside the container
|
227 |
+
./run_in_docker.sh "your question"
|
228 |
+
```
|
229 |
+
|
230 |
+
For more detailed Docker usage instructions, including cross-platform support, optimized configurations, and troubleshooting, please refer to [DOCKER_README.md](.container/DOCKER_README_en.md).
|
231 |
+
|
232 |
+
# 🚀 Quick Start
|
233 |
+
|
234 |
+
|
235 |
+
|
236 |
+
Run the following demo case:
|
237 |
+
|
238 |
+
```bash
|
239 |
+
python owl/run.py
|
240 |
+
```
|
241 |
+
|
242 |
+
## Running with Different Models
|
243 |
+
|
244 |
+
OWL supports various LLM backends. You can use the following scripts to run with different models:
|
245 |
+
|
246 |
+
```bash
|
247 |
+
# Run with Qwen model
|
248 |
+
python owl/run_qwen.py
|
249 |
+
|
250 |
+
# Run with Deepseek model
|
251 |
+
python owl/run_deepseek.py
|
252 |
+
|
253 |
+
# Run with other OpenAI-compatible models
|
254 |
+
python owl/run_openai_compatiable_model.py
|
255 |
+
```
|
256 |
+
|
257 |
+
For a simpler version that only requires an LLM API key, you can try our minimal example:
|
258 |
+
|
259 |
+
```bash
|
260 |
+
python owl/run_mini.py
|
261 |
+
```
|
262 |
+
|
263 |
+
You can run OWL agent with your own task by modifying the `run.py` script:
|
264 |
+
|
265 |
+
```python
|
266 |
+
# Define your own task
|
267 |
+
question = "Task description here."
|
268 |
+
|
269 |
+
society = construct_society(question)
|
270 |
+
answer, chat_history, token_count = run_society(society)
|
271 |
+
|
272 |
+
print(f"\033[94mAnswer: {answer}\033[0m")
|
273 |
+
```
|
274 |
+
|
275 |
+
For uploading files, simply provide the file path along with your question:
|
276 |
+
|
277 |
+
```python
|
278 |
+
# Task with a local file (e.g., file path: `tmp/example.docx`)
|
279 |
+
question = "What is in the given DOCX file? Here is the file path: tmp/example.docx"
|
280 |
+
|
281 |
+
society = construct_society(question)
|
282 |
+
answer, chat_history, token_count = run_society(society)
|
283 |
+
print(f"\033[94mAnswer: {answer}\033[0m")
|
284 |
+
```
|
285 |
+
|
286 |
+
OWL will then automatically invoke document-related tools to process the file and extract the answer.
|
287 |
+
|
288 |
+
|
289 |
+
Example tasks you can try:
|
290 |
+
- "Find the latest stock price for Apple Inc."
|
291 |
+
- "Analyze the sentiment of recent tweets about climate change"
|
292 |
+
- "Help me debug this Python code: [your code here]"
|
293 |
+
- "Summarize the main points from this research paper: [paper URL]"
|
294 |
+
|
295 |
+
# 🌐 Web Interface
|
296 |
+
|
297 |
+
OWL now includes a web-based user interface that makes it easier to interact with the system. To start the web interface, run:
|
298 |
+
|
299 |
+
```bash
|
300 |
+
python run_app.py
|
301 |
+
```
|
302 |
+
|
303 |
+
The web interface provides the following features:
|
304 |
+
|
305 |
+
- **Easy Model Selection**: Choose between different models (OpenAI, Qwen, DeepSeek, etc.)
|
306 |
+
- **Environment Variable Management**: Configure your API keys and other settings directly from the UI
|
307 |
+
- **Interactive Chat Interface**: Communicate with OWL agents through a user-friendly interface
|
308 |
+
- **Task History**: View the history and results of your interactions
|
309 |
+
|
310 |
+
The web interface is built using Gradio and runs locally on your machine. No data is sent to external servers beyond what's required for the model API calls you configure.
|
311 |
+
|
312 |
+
# 🧪 Experiments
|
313 |
+
|
314 |
+
To reproduce OWL's GAIA benchmark score of 58.18:
|
315 |
+
|
316 |
+
1. Switch to the `gaia58.18` branch:
|
317 |
+
```bash
|
318 |
+
git checkout gaia58.18
|
319 |
+
```
|
320 |
+
|
321 |
+
1. Run the evaluation script:
|
322 |
+
```bash
|
323 |
+
python run_gaia_roleplaying.py
|
324 |
+
```
|
325 |
+
|
326 |
+
# ⏱️ Future Plans
|
327 |
+
|
328 |
+
- [ ] Write a technical blog post detailing our exploration and insights in multi-agent collaboration in real-world tasks.
|
329 |
+
- [ ] Enhance the toolkit ecosystem with more specialized tools for domain-specific tasks.
|
330 |
+
- [ ] Develop more sophisticated agent interaction patterns and communication protocols
|
331 |
+
|
332 |
+
|
333 |
+
# 📄 License
|
334 |
+
|
335 |
+
The source code is licensed under Apache 2.0.
|
336 |
+
|
337 |
+
# 🖊️ Cite
|
338 |
+
|
339 |
+
If you find this repo useful, please cite:
|
340 |
+
|
341 |
+
|
342 |
+
```
|
343 |
+
@misc{owl2025,
|
344 |
+
title = {OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation},
|
345 |
+
author = {{CAMEL-AI.org}},
|
346 |
+
howpublished = {\url{https://github.com/camel-ai/owl}},
|
347 |
+
note = {Accessed: 2025-03-07},
|
348 |
+
year = {2025}
|
349 |
+
}
|
350 |
+
```
|
351 |
+
|
352 |
+
# 🔥 Community
|
353 |
+
Join us for further discussions!
|
354 |
+
<!--  -->
|
355 |
+

|
356 |
+
<!--  -->
|
357 |
+
|
358 |
+
# ❓ FAQ
|
359 |
+
|
360 |
+
**Q: Why don't I see Chrome running locally after starting the example script?**
|
361 |
+
|
362 |
+
A: If OWL determines that a task can be completed using non-browser tools (such as search or code execution), the browser will not be launched. The browser window will only appear when OWL determines that browser-based interaction is necessary.
|
363 |
+
|
364 |
+
# ⭐ Star History
|
365 |
+
|
366 |
+
[](https://star-history.com/#camel-ai/owl&Date)
|
367 |
+
|
368 |
+
|
369 |
+
|
370 |
+
[docs-image]: https://img.shields.io/badge/Documentation-EB3ECC
|
371 |
+
[docs-url]: https://camel-ai.github.io/camel/index.html
|
372 |
+
[star-image]: https://img.shields.io/github/stars/camel-ai/owl?label=stars&logo=github&color=brightgreen
|
373 |
+
[star-url]: https://github.com/camel-ai/owl/stargazers
|
374 |
+
[package-license-image]: https://img.shields.io/badge/License-Apache_2.0-blue.svg
|
375 |
+
[package-license-url]: https://github.com/camel-ai/owl/blob/main/licenses/LICENSE
|
376 |
+
|
377 |
+
[colab-url]: https://colab.research.google.com/drive/1AzP33O8rnMW__7ocWJhVBXjKziJXPtim?usp=sharing
|
378 |
+
[colab-image]: https://colab.research.google.com/assets/colab-badge.svg
|
379 |
+
[huggingface-url]: https://huggingface.co/camel-ai
|
380 |
+
[huggingface-image]: https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-CAMEL--AI-ffc107?color=ffc107&logoColor=white
|
381 |
+
[discord-url]: https://discord.camel-ai.org/
|
382 |
+
[discord-image]: https://img.shields.io/discord/1082486657678311454?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb
|
383 |
+
[wechat-url]: https://ghli.org/camel/wechat.png
|
384 |
+
[wechat-image]: https://img.shields.io/badge/WeChat-CamelAIOrg-brightgreen?logo=wechat&logoColor=white
|
385 |
+
[x-url]: https://x.com/CamelAIOrg
|
386 |
+
[x-image]: https://img.shields.io/twitter/follow/CamelAIOrg?style=social
|
387 |
+
[twitter-image]: https://img.shields.io/twitter/follow/CamelAIOrg?style=social&color=brightgreen&logo=twitter
|
388 |
+
[reddit-url]: https://www.reddit.com/r/CamelAI/
|
389 |
+
[reddit-image]: https://img.shields.io/reddit/subreddit-subscribers/CamelAI?style=plastic&logo=reddit&label=r%2FCAMEL&labelColor=white
|
390 |
+
[ambassador-url]: https://www.camel-ai.org/community
|
391 |
+
[owl-url]: ./assets/qr_code.jpg
|
392 |
+
[owl-image]: https://img.shields.io/badge/WeChat-OWLProject-brightgreen?logo=wechat&logoColor=white
|
README_zh.md
ADDED
@@ -0,0 +1,375 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
<h1 align="center">
|
2 |
+
🦉 OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation
|
3 |
+
🦉 OWL: 优化劳动力学习的通用智能体,用于处理现实世界的自动化任务
|
4 |
+
</h1>
|
5 |
+
|
6 |
+
|
7 |
+
<div align="center">
|
8 |
+
|
9 |
+
[![文档][docs-image]][docs-url]
|
10 |
+
[![Discord][discord-image]][discord-url]
|
11 |
+
[![X][x-image]][x-url]
|
12 |
+
[![Reddit][reddit-image]][reddit-url]
|
13 |
+
[![微信][wechat-image]][wechat-url]
|
14 |
+
[![微信][owl-image]][owl-url]
|
15 |
+
[![Hugging Face][huggingface-image]][huggingface-url]
|
16 |
+
[![Star][star-image]][star-url]
|
17 |
+
[![软件许可证][package-license-image]][package-license-url]
|
18 |
+
|
19 |
+
|
20 |
+
</div>
|
21 |
+
|
22 |
+
|
23 |
+
<hr>
|
24 |
+
|
25 |
+
<div align="center">
|
26 |
+
<h4 align="center">
|
27 |
+
|
28 |
+
[English README](https://github.com/camel-ai/owl/tree/main) |
|
29 |
+
[社区](https://github.com/camel-ai/camel#community) |
|
30 |
+
[安装](#️-installation) |
|
31 |
+
[示例](https://github.com/camel-ai/owl/tree/main/owl) |
|
32 |
+
[论文](https://arxiv.org/abs/2303.17760) |
|
33 |
+
[引用](#-community) |
|
34 |
+
[贡献](https://github.com/camel-ai/owl/graphs/contributors) |
|
35 |
+
[CAMEL-AI](https://www.camel-ai.org/)
|
36 |
+
|
37 |
+
</h4>
|
38 |
+
|
39 |
+
<div align="center" style="background-color: #f0f7ff; padding: 10px; border-radius: 5px; margin: 15px 0;">
|
40 |
+
<h3 style="color: #1e88e5; margin: 0;">
|
41 |
+
🏆 OWL 在 GAIA 基准测试中取得 <span style="color: #d81b60; font-weight: bold; font-size: 1.2em;">58.18</span> 平均分,在开源框架中排名 <span style="color: #d81b60; font-weight: bold; font-size: 1.2em;">🏅️ #1</span>! 🏆
|
42 |
+
</h3>
|
43 |
+
</div>
|
44 |
+
|
45 |
+
<div align="center">
|
46 |
+
|
47 |
+
🦉 OWL 是一个前沿的多智能体协作框架,推动任务自动化的边界,构建在 [CAMEL-AI Framework](https://github.com/camel-ai/camel)。
|
48 |
+
|
49 |
+
我们的愿景是彻底变革 AI 智能体协作解决现实任务的方式。通过利用动态智能体交互,OWL 实现了跨多领域更自然、高效且稳健的任务自动化。
|
50 |
+
|
51 |
+
</div>
|
52 |
+
|
53 |
+

|
54 |
+
|
55 |
+
|
56 |
+
|
57 |
+
<br>
|
58 |
+
|
59 |
+
|
60 |
+
</div>
|
61 |
+
|
62 |
+
<!-- # Key Features -->
|
63 |
+
# 📋 目录
|
64 |
+
|
65 |
+
- [📋 目录](#-目录)
|
66 |
+
- [🔥 新闻](#-新闻)
|
67 |
+
- [🎬 演示视频](#-演示视频)
|
68 |
+
- [✨️ 核心功能](#-核心功能)
|
69 |
+
- [🛠️ 安装](#️-安装)
|
70 |
+
- [**选项1:使用 uv(推荐)**](#选项1使用-uv推荐)
|
71 |
+
- [**选项2:使用 venv 和 pip**](#选项2使用-venv-和-pip)
|
72 |
+
- [**选项3:使用 conda**](#选项3使用-conda)
|
73 |
+
- [**设置环境变量**](#设置环境变量)
|
74 |
+
- [**使用Docker运行**](#使用docker运行)
|
75 |
+
- [🚀 快速开始](#-快速开始)
|
76 |
+
- [🌐 网页界面](#-网页界面)
|
77 |
+
- [🧪 实验](#-实验)
|
78 |
+
- [⏱️ 未来计划](#️-未来计划)
|
79 |
+
- [📄 许可证](#-许可证)
|
80 |
+
- [🖊️ 引用](#️-引用)
|
81 |
+
- [🔥 社区](#-社区)
|
82 |
+
- [❓ 常见问题](#-常见问题)
|
83 |
+
|
84 |
+
|
85 |
+
# 🔥 新闻
|
86 |
+
|
87 |
+
- **[2025.03.07]**: 我们开源了 🦉 OWL 项目的代码库。
|
88 |
+
|
89 |
+
# 🎬 演示视频
|
90 |
+
|
91 |
+
https://private-user-images.githubusercontent.com/55657767/420211368-f29f477d-7eef-46da-8d7a-8f3bcf506da2.mp4
|
92 |
+
|
93 |
+
https://private-user-images.githubusercontent.com/55657767/420212194-e813fc05-136a-485f-8df3-f10d9b4e63ec.mp4
|
94 |
+
|
95 |
+
# ✨️ 核心功能
|
96 |
+
|
97 |
+
- **在线搜索**:使用维基百科、谷歌搜索等,进行实时信息检索
|
98 |
+
- **多模态处理**:支持互联网或本地视频、图片、语音处理
|
99 |
+
- **浏览器操作**:借助Playwright框架开发浏览器模拟交互,支持页面滚动、点击、输入、下载、历史回退等功能
|
100 |
+
- **文件解析**:word、excel、PDF、PowerPoint信息提取,内容转文本/Markdown
|
101 |
+
- **代码执行**:编写python代码,并使用解释器运行
|
102 |
+
- **丰富工具包**:提供丰富的工具包,包括ArxivToolkit(学术论文检索)、AudioAnalysisToolkit(音频分析)、CodeExecutionToolkit(代码执行)、DalleToolkit(图像生成)、DataCommonsToolkit(数据共享)、ExcelToolkit(Excel处理)、GitHubToolkit(GitHub交互)、GoogleMapsToolkit(地图服务)、GoogleScholarToolkit(学术搜索)、ImageAnalysisToolkit(图像分析)、MathToolkit(数学计算)、NetworkXToolkit(图形分析)、NotionToolkit(Notion交互)、OpenAPIToolkit(API操作)、RedditToolkit(Reddit交互)、SearchToolkit(搜索服务)、SemanticScholarToolkit(语义学术搜索)、SymPyToolkit(符号计算)、VideoAnalysisToolkit(视频分析)、WeatherToolkit(天气查询)、WebToolkit(网页交互)等多种专业工具,满足各类特定任务需求。
|
103 |
+
|
104 |
+
# 🛠️ 安装
|
105 |
+
|
106 |
+
## 选项1:使用 uv(推荐)
|
107 |
+
|
108 |
+
```bash
|
109 |
+
# 克隆 GitHub 仓库
|
110 |
+
git clone https://github.com/camel-ai/owl.git
|
111 |
+
|
112 |
+
# 进入项目目录
|
113 |
+
cd owl
|
114 |
+
|
115 |
+
# 如果你还没有安装 uv,请先安装
|
116 |
+
pip install uv
|
117 |
+
|
118 |
+
# 创建虚拟环境并安装依赖
|
119 |
+
# 我们支持使用 Python 3.10、3.11、3.12
|
120 |
+
uv venv .venv --python=3.10
|
121 |
+
|
122 |
+
# 激活虚拟环境
|
123 |
+
# 对于 macOS/Linux
|
124 |
+
source .venv/bin/activate
|
125 |
+
# 对于 Windows
|
126 |
+
.venv\Scripts\activate
|
127 |
+
|
128 |
+
# 安装 CAMEL 及其所有依赖
|
129 |
+
uv pip install -e .
|
130 |
+
|
131 |
+
# 完成后退出虚拟环境
|
132 |
+
deactivate
|
133 |
+
```
|
134 |
+
|
135 |
+
## 选项2:使用 venv 和 pip
|
136 |
+
|
137 |
+
```bash
|
138 |
+
# 克隆 GitHub 仓库
|
139 |
+
git clone https://github.com/camel-ai/owl.git
|
140 |
+
|
141 |
+
# 进入项目目录
|
142 |
+
cd owl
|
143 |
+
|
144 |
+
# 创建虚拟环境
|
145 |
+
# 对于 Python 3.10(也适用于 3.11、3.12)
|
146 |
+
python3.10 -m venv .venv
|
147 |
+
|
148 |
+
# 激活虚拟环境
|
149 |
+
# 对于 macOS/Linux
|
150 |
+
source .venv/bin/activate
|
151 |
+
# 对于 Windows
|
152 |
+
.venv\Scripts\activate
|
153 |
+
|
154 |
+
# 从 requirements.txt 安装
|
155 |
+
pip install -r requirements.txt
|
156 |
+
```
|
157 |
+
|
158 |
+
## 选项3:使用 conda
|
159 |
+
|
160 |
+
```bash
|
161 |
+
# 克隆 GitHub 仓库
|
162 |
+
git clone https://github.com/camel-ai/owl.git
|
163 |
+
|
164 |
+
# 进入项目目录
|
165 |
+
cd owl
|
166 |
+
|
167 |
+
# 创建 conda 环境
|
168 |
+
conda create -n owl python=3.10
|
169 |
+
|
170 |
+
# 激活 conda 环境
|
171 |
+
conda activate owl
|
172 |
+
|
173 |
+
# 选项1:作为包安装(推荐)
|
174 |
+
pip install -e .
|
175 |
+
|
176 |
+
# 选项2:从 requirements.txt 安装
|
177 |
+
pip install -r requirements.txt
|
178 |
+
|
179 |
+
# 完成后退出 conda 环境
|
180 |
+
conda deactivate
|
181 |
+
```
|
182 |
+
|
183 |
+
## **设置环境变量**
|
184 |
+
|
185 |
+
在 `owl/.env_template` 文件中,你可以找到所有必要的 API 密钥以及各服务的注册网址。要使用这些 API 服务,请按照以下步骤操作:
|
186 |
+
|
187 |
+
1. *复制并重命名*: 复制 `.env_template` 文件,并将副本重命名为 `.env`。
|
188 |
+
2. *填写你的密钥*: 打开 `.env` 文件,在相应字段中填入你的 API 密钥。
|
189 |
+
3. *如需使用更多其他模型*:请参考我们CAMEL的models文档:https://docs.camel-ai.org/key_modules/models.html#supported-model-platforms-in-camel
|
190 |
+
|
191 |
+
> **注意**:为获得最佳性能,我们强烈建议使用 OpenAI 模型。我们通过测试发现,其他模型在处理复杂任务和基准测试时可能会导致性能显著降低。
|
192 |
+
|
193 |
+
## **使用Docker运行**
|
194 |
+
|
195 |
+
如果您希望使用Docker运行OWL项目,我们提供了完整的Docker支持:
|
196 |
+
|
197 |
+
```bash
|
198 |
+
# 克隆仓库
|
199 |
+
git clone https://github.com/camel-ai/owl.git
|
200 |
+
cd owl
|
201 |
+
|
202 |
+
# 配置环境变量
|
203 |
+
cp owl/.env_template owl/.env
|
204 |
+
# 编辑.env文件,填入您的API密钥
|
205 |
+
|
206 |
+
# 选项1:直接使用docker-compose
|
207 |
+
cd .container
|
208 |
+
docker-compose up -d
|
209 |
+
# 在容器中运行OWL
|
210 |
+
docker-compose exec owl bash -c "xvfb-python run.py"
|
211 |
+
|
212 |
+
# 选项2:使用提供的脚本构建和运行
|
213 |
+
cd .container
|
214 |
+
chmod +x build_docker.sh
|
215 |
+
./build_docker.sh
|
216 |
+
# 在容器中运行OWL
|
217 |
+
./run_in_docker.sh "您的问题"
|
218 |
+
```
|
219 |
+
|
220 |
+
更多详细的Docker使用说明,包括跨平台支持、优化配置和故障排除,请参阅 [DOCKER_README.md](.container/DOCKER_README.md)
|
221 |
+
|
222 |
+
# 🚀 快速开始
|
223 |
+
|
224 |
+
运行以下示例:
|
225 |
+
|
226 |
+
```bash
|
227 |
+
python owl/run.py
|
228 |
+
```
|
229 |
+
|
230 |
+
我们还提供了一个最小化示例,只需配置LLM的API密钥即可运行:
|
231 |
+
|
232 |
+
```bash
|
233 |
+
python owl/run_mini.py
|
234 |
+
```
|
235 |
+
|
236 |
+
## 使用不同的模型
|
237 |
+
|
238 |
+
OWL 支持多种 LLM 后端。您可以使用以下脚本来运行不同的模型:
|
239 |
+
|
240 |
+
```bash
|
241 |
+
# 使用 Qwen 模型运行
|
242 |
+
python owl/run_qwen.py
|
243 |
+
|
244 |
+
# 使用 Deepseek 模型运行
|
245 |
+
python owl/run_deepseek.py
|
246 |
+
|
247 |
+
# 使用其他 OpenAI 兼容模型运行
|
248 |
+
python owl/run_openai_compatiable_model.py
|
249 |
+
```
|
250 |
+
|
251 |
+
你可以通过修改 `run.py` 脚本来运行自己的任务:
|
252 |
+
|
253 |
+
```python
|
254 |
+
# Define your own task
|
255 |
+
question = "Task description here."
|
256 |
+
|
257 |
+
society = construct_society(question)
|
258 |
+
answer, chat_history, token_count = run_society(society)
|
259 |
+
|
260 |
+
print(f"\033[94mAnswer: {answer}\033[0m")
|
261 |
+
```
|
262 |
+
|
263 |
+
上传文件时,只需提供文件路径和问题:
|
264 |
+
|
265 |
+
```python
|
266 |
+
# 处理本地文件(例如,文件路径为 `tmp/example.docx`)
|
267 |
+
question = "给定的 DOCX 文件中有什么内容?文件路径如下:tmp/example.docx"
|
268 |
+
|
269 |
+
society = construct_society(question)
|
270 |
+
answer, chat_history, token_count = run_society(society)
|
271 |
+
|
272 |
+
print(f"答案:{answer}")
|
273 |
+
```
|
274 |
+
|
275 |
+
OWL 将自动调用与文档相关的工具来处理文件并提取答案。
|
276 |
+
|
277 |
+
你可以尝试以下示例任务:
|
278 |
+
- "查询苹果公司的最新股票价格"
|
279 |
+
- "分析关于气候变化的最新推文情绪"
|
280 |
+
- "帮我调试这段 Python 代码:[在此粘贴你的代码]"
|
281 |
+
- "总结这篇研究论文的主要观点:[论文URL]"
|
282 |
+
|
283 |
+
# 🌐 网页界面
|
284 |
+
|
285 |
+
OWL 现在包含一个基于网页的用户界面,使与系统交互变得更加容易。要启动网页界面,请运行:
|
286 |
+
|
287 |
+
```bash
|
288 |
+
python run_app.py
|
289 |
+
```
|
290 |
+
|
291 |
+
网页界面提供以下功能:
|
292 |
+
|
293 |
+
- **便捷的模型选择**:选择不同的模型(OpenAI、Qwen、DeepSeek等)
|
294 |
+
- **环境变量管理**:直接从界面配置API密钥和其他设置
|
295 |
+
- **交互式聊天界面**:通过用户友好的界面与OWL智能体交流
|
296 |
+
- **任务历史**:查看交互的历史记录和结果
|
297 |
+
|
298 |
+
网页界面使用Gradio构建,在您的本地机器上运行。除了您配置的模型API调用所需的数据外,不会向外部服务器发送任何数据。
|
299 |
+
|
300 |
+
# 🧪 实验
|
301 |
+
|
302 |
+
我们提供了一个脚本用于复现 GAIA 上的实验结果。
|
303 |
+
要复现我们在 GAIA 基准测试中获得的 58.18 分:
|
304 |
+
|
305 |
+
1. 切换到 `gaia58.18` 分支:
|
306 |
+
```bash
|
307 |
+
git checkout gaia58.18
|
308 |
+
```
|
309 |
+
|
310 |
+
2. 运行评估脚本:
|
311 |
+
```bash
|
312 |
+
python run_gaia_roleplaying.py
|
313 |
+
```
|
314 |
+
|
315 |
+
# ⏱️ 未来计划
|
316 |
+
|
317 |
+
- [ ] 撰写一篇技术博客,详细介绍我们在现实任务中多智能体协作方面的探索与见解。
|
318 |
+
- [ ] 通过引入更多针对特定领域任务的专业工具,进一步完善工具生态系统。
|
319 |
+
- [ ] 开发更复杂的智能体交互模式和通信协议
|
320 |
+
|
321 |
+
|
322 |
+
# 📄 许可证
|
323 |
+
|
324 |
+
源代码采用 Apache 2.0 许可证。
|
325 |
+
|
326 |
+
# 🖊️ 引用
|
327 |
+
|
328 |
+
如果你觉得这个仓库对你有帮助,请引用:
|
329 |
+
|
330 |
+
|
331 |
+
```
|
332 |
+
@misc{owl2025,
|
333 |
+
title = {OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation},
|
334 |
+
author = {{CAMEL-AI.org}},
|
335 |
+
howpublished = {\url{https://github.com/camel-ai/owl}},
|
336 |
+
note = {Accessed: 2025-03-07},
|
337 |
+
year = {2025}
|
338 |
+
}
|
339 |
+
```
|
340 |
+
|
341 |
+
# 🔥 社区
|
342 |
+
加入我们,参与更多讨论!
|
343 |
+
<!--  -->
|
344 |
+

|
345 |
+
<!--  -->
|
346 |
+
|
347 |
+
# ❓ 常见问题
|
348 |
+
|
349 |
+
**Q: 为什么启动示例脚本后,我没有看到本地运行Chrome浏览器?**
|
350 |
+
|
351 |
+
A: 当OWL判断某个任务可以使用非浏览器工具(如搜索、代码分析等)完成时,浏览器就不会启动。只有在判断需要使用浏览器工具的时候,本地才会弹出浏览器窗口,并进行浏览器模拟交互。
|
352 |
+
|
353 |
+
[docs-image]: https://img.shields.io/badge/Documentation-EB3ECC
|
354 |
+
[docs-url]: https://camel-ai.github.io/camel/index.html
|
355 |
+
[star-image]: https://img.shields.io/github/stars/camel-ai/owl?label=stars&logo=github&color=brightgreen
|
356 |
+
[star-url]: https://github.com/camel-ai/owl/stargazers
|
357 |
+
[package-license-image]: https://img.shields.io/badge/License-Apache_2.0-blue.svg
|
358 |
+
[package-license-url]: https://github.com/camel-ai/owl/blob/main/licenses/LICENSE
|
359 |
+
|
360 |
+
[colab-url]: https://colab.research.google.com/drive/1AzP33O8rnMW__7ocWJhVBXjKziJXPtim?usp=sharing
|
361 |
+
[colab-image]: https://colab.research.google.com/assets/colab-badge.svg
|
362 |
+
[huggingface-url]: https://huggingface.co/camel-ai
|
363 |
+
[huggingface-image]: https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-CAMEL--AI-ffc107?color=ffc107&logoColor=white
|
364 |
+
[discord-url]: https://discord.camel-ai.org/
|
365 |
+
[discord-image]: https://img.shields.io/discord/1082486657678311454?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb
|
366 |
+
[wechat-url]: https://ghli.org/camel/wechat.png
|
367 |
+
[wechat-image]: https://img.shields.io/badge/WeChat-CamelAIOrg-brightgreen?logo=wechat&logoColor=white
|
368 |
+
[x-url]: https://x.com/CamelAIOrg
|
369 |
+
[x-image]: https://img.shields.io/twitter/follow/CamelAIOrg?style=social
|
370 |
+
[twitter-image]: https://img.shields.io/twitter/follow/CamelAIOrg?style=social&color=brightgreen&logo=twitter
|
371 |
+
[reddit-url]: https://www.reddit.com/r/CamelAI/
|
372 |
+
[reddit-image]: https://img.shields.io/reddit/subreddit-subscribers/CamelAI?style=plastic&logo=reddit&label=r%2FCAMEL&labelColor=white
|
373 |
+
[ambassador-url]: https://www.camel-ai.org/community
|
374 |
+
[owl-url]: ./assets/qr_code.jpg
|
375 |
+
[owl-image]: https://img.shields.io/badge/WeChat-OWLProject-brightgreen?logo=wechat&logoColor=white
|
assets/community.png
ADDED
![]() |
Git LFS Details
|
assets/community_2.png
ADDED
![]() |
Git LFS Details
|
assets/community_3.jpg
ADDED
![]() |
Git LFS Details
|
assets/community_4.jpg
ADDED
![]() |
Git LFS Details
|
assets/community_5.jpg
ADDED
![]() |
Git LFS Details
|
assets/community_6.jpg
ADDED
![]() |
Git LFS Details
|
assets/community_6.png
ADDED
![]() |
Git LFS Details
|
assets/meetup.jpg
ADDED
![]() |
Git LFS Details
|
assets/owl_architecture.png
ADDED
![]() |
Git LFS Details
|
assets/qr_code.jpg
ADDED
![]() |
Git LFS Details
|
licenses/LICENSE
ADDED
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
licenses/license_template.txt
ADDED
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
licenses/update_license.py
ADDED
@@ -0,0 +1,126 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
import os
|
15 |
+
import re
|
16 |
+
import sys
|
17 |
+
from pathlib import Path
|
18 |
+
from typing import List
|
19 |
+
|
20 |
+
|
21 |
+
# The license template file is hard-coded with specific start and end lines
|
22 |
+
def fine_license_start_line(lines: List[str], start_with: str) -> int:
|
23 |
+
for i in range(len(lines)):
|
24 |
+
if lines[i].startswith(start_with):
|
25 |
+
return i
|
26 |
+
return None
|
27 |
+
|
28 |
+
|
29 |
+
def find_license_end_line(lines: List[str], start_with: str) -> int:
|
30 |
+
for i in range(len(lines) - 1, -1, -1):
|
31 |
+
if lines[i].startswith(start_with):
|
32 |
+
return i
|
33 |
+
return None
|
34 |
+
|
35 |
+
|
36 |
+
def update_license_in_file(
|
37 |
+
file_path: str,
|
38 |
+
license_template_path: str,
|
39 |
+
start_line_start_with: str,
|
40 |
+
end_line_start_with: str,
|
41 |
+
) -> bool:
|
42 |
+
with open(file_path, "r", encoding="utf-8") as f: # for windows compatibility
|
43 |
+
content = f.read()
|
44 |
+
|
45 |
+
with open(license_template_path, "r", encoding="utf-8") as f:
|
46 |
+
new_license = f.read().strip()
|
47 |
+
|
48 |
+
maybe_existing_licenses = re.findall(
|
49 |
+
r"^#.*?(?=\n)", content, re.MULTILINE | re.DOTALL
|
50 |
+
)
|
51 |
+
start_index = fine_license_start_line(
|
52 |
+
maybe_existing_licenses, start_line_start_with
|
53 |
+
)
|
54 |
+
end_index = find_license_end_line(maybe_existing_licenses, end_line_start_with)
|
55 |
+
if start_index is not None and end_index is not None:
|
56 |
+
maybe_existing_licenses = maybe_existing_licenses[start_index : end_index + 1]
|
57 |
+
else:
|
58 |
+
maybe_existing_licenses = None
|
59 |
+
if maybe_existing_licenses:
|
60 |
+
maybe_old_licenses = "\n".join(maybe_existing_licenses)
|
61 |
+
if maybe_old_licenses.strip() != new_license.strip():
|
62 |
+
replaced_content = content.replace(maybe_old_licenses, new_license)
|
63 |
+
with open(file_path, "w") as f:
|
64 |
+
f.write(replaced_content)
|
65 |
+
print(f"Replaced license in {file_path}")
|
66 |
+
return True
|
67 |
+
else:
|
68 |
+
return False
|
69 |
+
else:
|
70 |
+
with open(file_path, "w") as f:
|
71 |
+
f.write(new_license + "\n" + content)
|
72 |
+
print(f"Added license to {file_path}")
|
73 |
+
return True
|
74 |
+
|
75 |
+
|
76 |
+
def update_license_in_directory(
|
77 |
+
directory_path: str,
|
78 |
+
license_template_path: str,
|
79 |
+
start_line_start_with: str,
|
80 |
+
end_line_start_with: str,
|
81 |
+
) -> None:
|
82 |
+
# Check if directory exists
|
83 |
+
if not os.path.isdir(directory_path):
|
84 |
+
raise NotADirectoryError(f"{directory_path} is not a directory")
|
85 |
+
# Check if license template exists
|
86 |
+
if not os.path.isfile(license_template_path):
|
87 |
+
raise FileNotFoundError(f"{license_template_path} not found")
|
88 |
+
|
89 |
+
file_count = 0
|
90 |
+
for py_files in Path(directory_path).rglob("*.py"):
|
91 |
+
if py_files.name.startswith("."):
|
92 |
+
continue
|
93 |
+
if any(part.startswith(".") for part in py_files.parts):
|
94 |
+
continue
|
95 |
+
if update_license_in_file(
|
96 |
+
py_files,
|
97 |
+
license_template_path,
|
98 |
+
start_line_start_with,
|
99 |
+
end_line_start_with,
|
100 |
+
):
|
101 |
+
file_count += 1
|
102 |
+
|
103 |
+
print(f"License updated in {file_count} files")
|
104 |
+
|
105 |
+
|
106 |
+
if __name__ == "__main__":
|
107 |
+
if len(sys.argv) < 3:
|
108 |
+
print(
|
109 |
+
"Usage from command line: "
|
110 |
+
"python update_license.py <directory_path> <license_template_path>"
|
111 |
+
"No valid input arguments found, please enter manually."
|
112 |
+
)
|
113 |
+
directory_path = input("Enter directory path: ")
|
114 |
+
license_template_path = input("Enter license template path: ")
|
115 |
+
else:
|
116 |
+
directory_path = sys.argv[1]
|
117 |
+
license_template_path = sys.argv[2]
|
118 |
+
|
119 |
+
start_line_start_with = "# ========= Copyright"
|
120 |
+
end_line_start_with = "# ========= Copyright"
|
121 |
+
update_license_in_directory(
|
122 |
+
directory_path,
|
123 |
+
license_template_path,
|
124 |
+
start_line_start_with,
|
125 |
+
end_line_start_with,
|
126 |
+
)
|
owl/.env_template
ADDED
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# MODEL & API (See https://github.com/camel-ai/camel/blob/master/camel/types/enums.py)
|
2 |
+
|
3 |
+
# OPENAI API
|
4 |
+
OPENAI_API_KEY = ""
|
5 |
+
# OPENAI_API_BASE_URL = ""
|
6 |
+
|
7 |
+
# Qwen API (https://help.aliyun.com/zh/model-studio/developer-reference/get-api-key)
|
8 |
+
# QWEN_API_KEY=""
|
9 |
+
|
10 |
+
# DeepSeek API (https://platform.deepseek.com/api_keys)
|
11 |
+
# DEEPSEEK_API_KEY=""
|
12 |
+
|
13 |
+
#===========================================
|
14 |
+
# Tools & Services API
|
15 |
+
#===========================================
|
16 |
+
|
17 |
+
# Google Search API (https://developers.google.com/custom-search/v1/overview)
|
18 |
+
GOOGLE_API_KEY=""
|
19 |
+
SEARCH_ENGINE_ID=""
|
20 |
+
|
21 |
+
# Hugging Face API (https://huggingface.co/join)
|
22 |
+
HF_TOKEN=""
|
23 |
+
|
24 |
+
# Chunkr API (https://chunkr.ai/)
|
25 |
+
CHUNKR_API_KEY=""
|
26 |
+
|
27 |
+
# Firecrawl API (https://www.firecrawl.dev/)
|
28 |
+
FIRECRAWL_API_KEY=""
|
owl/app.py
ADDED
@@ -0,0 +1,669 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
import os
|
15 |
+
import sys
|
16 |
+
import gradio as gr
|
17 |
+
import subprocess
|
18 |
+
import threading
|
19 |
+
import time
|
20 |
+
from datetime import datetime
|
21 |
+
import queue
|
22 |
+
from pathlib import Path
|
23 |
+
import json
|
24 |
+
import signal
|
25 |
+
import dotenv
|
26 |
+
|
27 |
+
# 设置日志队列
|
28 |
+
log_queue = queue.Queue()
|
29 |
+
|
30 |
+
# 当前运行的进程
|
31 |
+
current_process = None
|
32 |
+
process_lock = threading.Lock()
|
33 |
+
|
34 |
+
# 脚本选项
|
35 |
+
SCRIPTS = {
|
36 |
+
"Qwen Mini (中文)": "run_qwen_mini_zh.py",
|
37 |
+
"Qwen (中文)": "run_qwen_zh.py",
|
38 |
+
"Mini": "run_mini.py",
|
39 |
+
"DeepSeek (中文)": "run_deepseek_zh.py",
|
40 |
+
"Default": "run.py",
|
41 |
+
"GAIA Roleplaying": "run_gaia_roleplaying.py",
|
42 |
+
}
|
43 |
+
|
44 |
+
# 脚本描述
|
45 |
+
SCRIPT_DESCRIPTIONS = {
|
46 |
+
"Qwen Mini (中文)": "使用阿里云Qwen模型的中文版本,适合中文问答和任务",
|
47 |
+
"Qwen (中文)": "使用阿里云Qwen模型,支持多种工具和功能",
|
48 |
+
"Mini": "轻量级版本,使用OpenAI GPT-4o模型",
|
49 |
+
"DeepSeek (中文)": "使用DeepSeek模型,适合非多模态任务",
|
50 |
+
"Default": "默认OWL实现,使用OpenAI GPT-4o模型和全套工具",
|
51 |
+
"GAIA Roleplaying": "GAIA基准测试实现,用于评估模型能力",
|
52 |
+
}
|
53 |
+
|
54 |
+
# 环境变量分组
|
55 |
+
ENV_GROUPS = {
|
56 |
+
"模型API": [
|
57 |
+
{
|
58 |
+
"name": "OPENAI_API_KEY",
|
59 |
+
"label": "OpenAI API密钥",
|
60 |
+
"type": "password",
|
61 |
+
"required": False,
|
62 |
+
"help": "OpenAI API密钥,用于访问GPT模型。获取方式:https://platform.openai.com/api-keys",
|
63 |
+
},
|
64 |
+
{
|
65 |
+
"name": "OPENAI_API_BASE_URL",
|
66 |
+
"label": "OpenAI API基础URL",
|
67 |
+
"type": "text",
|
68 |
+
"required": False,
|
69 |
+
"help": "OpenAI API的基础URL,可选。如果使用代理或自定义端点,请设置此项。",
|
70 |
+
},
|
71 |
+
{
|
72 |
+
"name": "QWEN_API_KEY",
|
73 |
+
"label": "阿里云Qwen API密钥",
|
74 |
+
"type": "password",
|
75 |
+
"required": False,
|
76 |
+
"help": "阿里云Qwen API密钥,用于访问Qwen模型。获取方式:https://help.aliyun.com/zh/model-studio/developer-reference/get-api-key",
|
77 |
+
},
|
78 |
+
{
|
79 |
+
"name": "DEEPSEEK_API_KEY",
|
80 |
+
"label": "DeepSeek API密钥",
|
81 |
+
"type": "password",
|
82 |
+
"required": False,
|
83 |
+
"help": "DeepSeek API密钥,用于访问DeepSeek模型。获取方式:https://platform.deepseek.com/api_keys",
|
84 |
+
},
|
85 |
+
],
|
86 |
+
"搜索工具": [
|
87 |
+
{
|
88 |
+
"name": "GOOGLE_API_KEY",
|
89 |
+
"label": "Google API密钥",
|
90 |
+
"type": "password",
|
91 |
+
"required": False,
|
92 |
+
"help": "Google搜索API密钥,用于网络搜索功能。获取方式:https://developers.google.com/custom-search/v1/overview",
|
93 |
+
},
|
94 |
+
{
|
95 |
+
"name": "SEARCH_ENGINE_ID",
|
96 |
+
"label": "搜索引擎ID",
|
97 |
+
"type": "text",
|
98 |
+
"required": False,
|
99 |
+
"help": "Google自定义搜索引擎ID,与Google API密钥配合使用。获取方式:https://developers.google.com/custom-search/v1/overview",
|
100 |
+
},
|
101 |
+
],
|
102 |
+
"其他工具": [
|
103 |
+
{
|
104 |
+
"name": "HF_TOKEN",
|
105 |
+
"label": "Hugging Face令牌",
|
106 |
+
"type": "password",
|
107 |
+
"required": False,
|
108 |
+
"help": "Hugging Face API令牌,用于访问Hugging Face模型和数据集。获取方式:https://huggingface.co/join",
|
109 |
+
},
|
110 |
+
{
|
111 |
+
"name": "CHUNKR_API_KEY",
|
112 |
+
"label": "Chunkr API密钥",
|
113 |
+
"type": "password",
|
114 |
+
"required": False,
|
115 |
+
"help": "Chunkr API密钥,用于文档处理功能。获取方式:https://chunkr.ai/",
|
116 |
+
},
|
117 |
+
{
|
118 |
+
"name": "FIRECRAWL_API_KEY",
|
119 |
+
"label": "Firecrawl API密钥",
|
120 |
+
"type": "password",
|
121 |
+
"required": False,
|
122 |
+
"help": "Firecrawl API密钥,用于网页爬取功能。获取方式:https://www.firecrawl.dev/",
|
123 |
+
},
|
124 |
+
],
|
125 |
+
"自定义环境变量": [], # 用户自定义的环境变量将存储在这里
|
126 |
+
}
|
127 |
+
|
128 |
+
|
129 |
+
def get_script_info(script_name):
|
130 |
+
"""获取脚本的详细信息"""
|
131 |
+
return SCRIPT_DESCRIPTIONS.get(script_name, "无描述信息")
|
132 |
+
|
133 |
+
|
134 |
+
def load_env_vars():
|
135 |
+
"""加载环境变量"""
|
136 |
+
env_vars = {}
|
137 |
+
# 尝试从.env文件加载
|
138 |
+
dotenv.load_dotenv()
|
139 |
+
|
140 |
+
# ���取所有环境变量
|
141 |
+
for group in ENV_GROUPS.values():
|
142 |
+
for var in group:
|
143 |
+
env_vars[var["name"]] = os.environ.get(var["name"], "")
|
144 |
+
|
145 |
+
# 加载.env文件中可能存在的其他环境变量
|
146 |
+
if Path(".env").exists():
|
147 |
+
with open(".env", "r", encoding="utf-8") as f:
|
148 |
+
for line in f:
|
149 |
+
line = line.strip()
|
150 |
+
if line and not line.startswith("#") and "=" in line:
|
151 |
+
key, value = line.split("=", 1)
|
152 |
+
key = key.strip()
|
153 |
+
value = value.strip().strip("\"'")
|
154 |
+
|
155 |
+
# 检查是否是已知的环境变量
|
156 |
+
known_var = False
|
157 |
+
for group in ENV_GROUPS.values():
|
158 |
+
if any(var["name"] == key for var in group):
|
159 |
+
known_var = True
|
160 |
+
break
|
161 |
+
|
162 |
+
# 如果不是已知的环境变量,添加到自定义环境变量组
|
163 |
+
if not known_var and key not in env_vars:
|
164 |
+
ENV_GROUPS["自定义环境变量"].append(
|
165 |
+
{
|
166 |
+
"name": key,
|
167 |
+
"label": key,
|
168 |
+
"type": "text",
|
169 |
+
"required": False,
|
170 |
+
"help": "用户自定义环境变量",
|
171 |
+
}
|
172 |
+
)
|
173 |
+
env_vars[key] = value
|
174 |
+
|
175 |
+
return env_vars
|
176 |
+
|
177 |
+
|
178 |
+
def save_env_vars(env_vars):
|
179 |
+
"""保存环境变量到.env文件"""
|
180 |
+
# 读取现有的.env文件内容
|
181 |
+
env_path = Path(".env")
|
182 |
+
existing_content = {}
|
183 |
+
|
184 |
+
if env_path.exists():
|
185 |
+
with open(env_path, "r", encoding="utf-8") as f:
|
186 |
+
for line in f:
|
187 |
+
line = line.strip()
|
188 |
+
if line and not line.startswith("#") and "=" in line:
|
189 |
+
key, value = line.split("=", 1)
|
190 |
+
existing_content[key.strip()] = value.strip()
|
191 |
+
|
192 |
+
# 更新环境变量
|
193 |
+
for key, value in env_vars.items():
|
194 |
+
if value: # 只保存非空值
|
195 |
+
# 确保值是字符串形式,并用引号包裹
|
196 |
+
value = str(value) # 确保值是字符串
|
197 |
+
if not (value.startswith('"') and value.endswith('"')) and not (
|
198 |
+
value.startswith("'") and value.endswith("'")
|
199 |
+
):
|
200 |
+
value = f'"{value}"'
|
201 |
+
existing_content[key] = value
|
202 |
+
# 同时更新当前进程的环境变量
|
203 |
+
os.environ[key] = value.strip("\"'")
|
204 |
+
|
205 |
+
# 写入.env文件
|
206 |
+
with open(env_path, "w", encoding="utf-8") as f:
|
207 |
+
for key, value in existing_content.items():
|
208 |
+
f.write(f"{key}={value}\n")
|
209 |
+
|
210 |
+
return "✅ 环境变量已保存"
|
211 |
+
|
212 |
+
|
213 |
+
def add_custom_env_var(name, value, var_type):
|
214 |
+
"""添加自定义环境变量"""
|
215 |
+
if not name:
|
216 |
+
return "❌ 环境变量名不能为空", None
|
217 |
+
|
218 |
+
# 检查是否已存在同名环境变量
|
219 |
+
for group in ENV_GROUPS.values():
|
220 |
+
if any(var["name"] == name for var in group):
|
221 |
+
return f"❌ 环境变量 {name} 已存在", None
|
222 |
+
|
223 |
+
# 添加到自定义环境变量组
|
224 |
+
ENV_GROUPS["自定义环境变量"].append(
|
225 |
+
{
|
226 |
+
"name": name,
|
227 |
+
"label": name,
|
228 |
+
"type": var_type,
|
229 |
+
"required": False,
|
230 |
+
"help": "用户自定义环境变量",
|
231 |
+
}
|
232 |
+
)
|
233 |
+
|
234 |
+
# 保存环境变量
|
235 |
+
env_vars = {name: value}
|
236 |
+
save_env_vars(env_vars)
|
237 |
+
|
238 |
+
# 返回成功消息和更新后的环境变量组
|
239 |
+
return f"✅ 已添加环境变量 {name}", ENV_GROUPS["自定义环境变量"]
|
240 |
+
|
241 |
+
|
242 |
+
def terminate_process():
|
243 |
+
"""终止当前运行的进程"""
|
244 |
+
global current_process
|
245 |
+
|
246 |
+
with process_lock:
|
247 |
+
if current_process is not None and current_process.poll() is None:
|
248 |
+
# 在Windows上使用CTRL_BREAK_EVENT,在Unix上使用SIGTERM
|
249 |
+
if os.name == "nt":
|
250 |
+
current_process.send_signal(signal.CTRL_BREAK_EVENT)
|
251 |
+
else:
|
252 |
+
current_process.terminate()
|
253 |
+
|
254 |
+
# 等待进程终止
|
255 |
+
try:
|
256 |
+
current_process.wait(timeout=5)
|
257 |
+
except subprocess.TimeoutExpired:
|
258 |
+
# 如果进程没有在5秒内终止,强制终止
|
259 |
+
current_process.kill()
|
260 |
+
|
261 |
+
log_queue.put("进程已终止\n")
|
262 |
+
return "✅ 进程已终止"
|
263 |
+
else:
|
264 |
+
return "❌ 没有正在运行的进程"
|
265 |
+
|
266 |
+
|
267 |
+
def run_script(script_dropdown, question, progress=gr.Progress()):
|
268 |
+
"""运行选定的脚本并返回输出"""
|
269 |
+
global current_process
|
270 |
+
|
271 |
+
script_name = SCRIPTS.get(script_dropdown)
|
272 |
+
if not script_name:
|
273 |
+
return "❌ 无效的脚本选择", "", "", "", None
|
274 |
+
|
275 |
+
if not question.strip():
|
276 |
+
return "请输入问题!", "", "", "", None
|
277 |
+
|
278 |
+
# 清空日志队列
|
279 |
+
while not log_queue.empty():
|
280 |
+
log_queue.get()
|
281 |
+
|
282 |
+
# 创建日志目录
|
283 |
+
log_dir = Path("logs")
|
284 |
+
log_dir.mkdir(exist_ok=True)
|
285 |
+
|
286 |
+
# 创建带时间戳的日志文件
|
287 |
+
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
288 |
+
log_file = log_dir / f"{script_name.replace('.py', '')}_{timestamp}.log"
|
289 |
+
|
290 |
+
# 构建命令
|
291 |
+
cmd = [
|
292 |
+
sys.executable,
|
293 |
+
os.path.join("owl", "script_adapter.py"),
|
294 |
+
os.path.join("owl", script_name),
|
295 |
+
]
|
296 |
+
|
297 |
+
# 创建环境变量副本并添加问题
|
298 |
+
env = os.environ.copy()
|
299 |
+
env["OWL_QUESTION"] = question
|
300 |
+
|
301 |
+
# 启动进程
|
302 |
+
with process_lock:
|
303 |
+
current_process = subprocess.Popen(
|
304 |
+
cmd,
|
305 |
+
stdout=subprocess.PIPE,
|
306 |
+
stderr=subprocess.STDOUT,
|
307 |
+
text=True,
|
308 |
+
bufsize=1,
|
309 |
+
env=env,
|
310 |
+
)
|
311 |
+
|
312 |
+
# 创建线程来读取输出
|
313 |
+
def read_output():
|
314 |
+
try:
|
315 |
+
with open(log_file, "w", encoding="utf-8") as f:
|
316 |
+
for line in iter(current_process.stdout.readline, ""):
|
317 |
+
if line:
|
318 |
+
# 写入日志文件
|
319 |
+
f.write(line)
|
320 |
+
f.flush()
|
321 |
+
# 添加到队列
|
322 |
+
log_queue.put(line)
|
323 |
+
except Exception as e:
|
324 |
+
log_queue.put(f"读取输出时出错: {str(e)}\n")
|
325 |
+
|
326 |
+
# 启动读取线程
|
327 |
+
threading.Thread(target=read_output, daemon=True).start()
|
328 |
+
|
329 |
+
# 收集日志
|
330 |
+
logs = []
|
331 |
+
progress(0, desc="正在运行...")
|
332 |
+
|
333 |
+
# 等待进程完成或超时
|
334 |
+
start_time = time.time()
|
335 |
+
timeout = 1800 # 30分钟超时
|
336 |
+
|
337 |
+
while current_process.poll() is None:
|
338 |
+
# 检查是否超时
|
339 |
+
if time.time() - start_time > timeout:
|
340 |
+
with process_lock:
|
341 |
+
if current_process.poll() is None:
|
342 |
+
if os.name == "nt":
|
343 |
+
current_process.send_signal(signal.CTRL_BREAK_EVENT)
|
344 |
+
else:
|
345 |
+
current_process.terminate()
|
346 |
+
log_queue.put("执行超时,已终止进程\n")
|
347 |
+
break
|
348 |
+
|
349 |
+
# 从队列获取日志
|
350 |
+
while not log_queue.empty():
|
351 |
+
log = log_queue.get()
|
352 |
+
logs.append(log)
|
353 |
+
|
354 |
+
# 更新进度
|
355 |
+
elapsed = time.time() - start_time
|
356 |
+
progress(min(elapsed / 300, 0.99), desc="正在运行...")
|
357 |
+
|
358 |
+
# 短暂休眠以减少CPU使用
|
359 |
+
time.sleep(0.1)
|
360 |
+
|
361 |
+
# 每秒更新一次日志显示
|
362 |
+
yield (
|
363 |
+
status_message(current_process),
|
364 |
+
extract_answer(logs),
|
365 |
+
"".join(logs),
|
366 |
+
str(log_file),
|
367 |
+
None,
|
368 |
+
)
|
369 |
+
|
370 |
+
# 获取剩余日志
|
371 |
+
while not log_queue.empty():
|
372 |
+
logs.append(log_queue.get())
|
373 |
+
|
374 |
+
# 提取聊天历史(如果有)
|
375 |
+
chat_history = extract_chat_history(logs)
|
376 |
+
|
377 |
+
# 返回最终状态和日志
|
378 |
+
return (
|
379 |
+
status_message(current_process),
|
380 |
+
extract_answer(logs),
|
381 |
+
"".join(logs),
|
382 |
+
str(log_file),
|
383 |
+
chat_history,
|
384 |
+
)
|
385 |
+
|
386 |
+
|
387 |
+
def status_message(process):
|
388 |
+
"""根据进程状态返回状态消息"""
|
389 |
+
if process.poll() is None:
|
390 |
+
return "⏳ 正在运行..."
|
391 |
+
elif process.returncode == 0:
|
392 |
+
return "✅ 执行成功"
|
393 |
+
else:
|
394 |
+
return f"❌ 执行失败 (返回码: {process.returncode})"
|
395 |
+
|
396 |
+
|
397 |
+
def extract_answer(logs):
|
398 |
+
"""从日志中提取答案"""
|
399 |
+
answer = ""
|
400 |
+
for log in logs:
|
401 |
+
if "Answer:" in log:
|
402 |
+
answer = log.split("Answer:", 1)[1].strip()
|
403 |
+
break
|
404 |
+
return answer
|
405 |
+
|
406 |
+
|
407 |
+
def extract_chat_history(logs):
|
408 |
+
"""尝试从日志中提取聊天历史"""
|
409 |
+
try:
|
410 |
+
chat_json_str = ""
|
411 |
+
capture_json = False
|
412 |
+
|
413 |
+
for log in logs:
|
414 |
+
if "chat_history" in log:
|
415 |
+
# 开始捕获JSON
|
416 |
+
start_idx = log.find("[")
|
417 |
+
if start_idx != -1:
|
418 |
+
capture_json = True
|
419 |
+
chat_json_str = log[start_idx:]
|
420 |
+
elif capture_json:
|
421 |
+
# 继续捕获JSON直到找到匹配的结束括号
|
422 |
+
chat_json_str += log
|
423 |
+
if "]" in log:
|
424 |
+
# 找到结束括号,尝试解析JSON
|
425 |
+
end_idx = chat_json_str.rfind("]") + 1
|
426 |
+
if end_idx > 0:
|
427 |
+
try:
|
428 |
+
# 清理可能的额外文本
|
429 |
+
json_str = chat_json_str[:end_idx].strip()
|
430 |
+
chat_data = json.loads(json_str)
|
431 |
+
|
432 |
+
# 格式化为Gradio聊天组件可用的格式
|
433 |
+
formatted_chat = []
|
434 |
+
for msg in chat_data:
|
435 |
+
if "role" in msg and "content" in msg:
|
436 |
+
role = "用户" if msg["role"] == "user" else "助手"
|
437 |
+
formatted_chat.append([role, msg["content"]])
|
438 |
+
return formatted_chat
|
439 |
+
except json.JSONDecodeError:
|
440 |
+
# 如果解析失败,继续捕获
|
441 |
+
pass
|
442 |
+
except Exception:
|
443 |
+
# 其他错误,停止捕获
|
444 |
+
capture_json = False
|
445 |
+
except Exception:
|
446 |
+
pass
|
447 |
+
return None
|
448 |
+
|
449 |
+
|
450 |
+
def create_ui():
|
451 |
+
"""创建Gradio界面"""
|
452 |
+
# 加载环境变量
|
453 |
+
env_vars = load_env_vars()
|
454 |
+
|
455 |
+
with gr.Blocks(theme=gr.themes.Soft(primary_hue="blue")) as app:
|
456 |
+
gr.Markdown(
|
457 |
+
"""
|
458 |
+
# 🦉 OWL 智能助手运行平台
|
459 |
+
|
460 |
+
选择一个模型并输入您的问题,系统将运行相应的脚本并显示结果。
|
461 |
+
"""
|
462 |
+
)
|
463 |
+
|
464 |
+
with gr.Tabs() as tabs:
|
465 |
+
with gr.TabItem("运行模式"):
|
466 |
+
with gr.Row():
|
467 |
+
with gr.Column(scale=1):
|
468 |
+
# 确保默认值是SCRIPTS中存在的键
|
469 |
+
default_script = list(SCRIPTS.keys())[0] if SCRIPTS else None
|
470 |
+
script_dropdown = gr.Dropdown(
|
471 |
+
choices=list(SCRIPTS.keys()),
|
472 |
+
value=default_script,
|
473 |
+
label="选择模式",
|
474 |
+
)
|
475 |
+
|
476 |
+
script_info = gr.Textbox(
|
477 |
+
value=get_script_info(default_script)
|
478 |
+
if default_script
|
479 |
+
else "",
|
480 |
+
label="模型描述",
|
481 |
+
interactive=False,
|
482 |
+
)
|
483 |
+
|
484 |
+
script_dropdown.change(
|
485 |
+
fn=lambda x: get_script_info(x),
|
486 |
+
inputs=script_dropdown,
|
487 |
+
outputs=script_info,
|
488 |
+
)
|
489 |
+
|
490 |
+
question_input = gr.Textbox(
|
491 |
+
lines=5, placeholder="请输入您的问题...", label="问题"
|
492 |
+
)
|
493 |
+
|
494 |
+
gr.Markdown(
|
495 |
+
"""
|
496 |
+
> **注意**: 您输入的问题将替换脚本中的默认问题。系统会自动处理问题的替换,确保您的问题被正确使用。
|
497 |
+
"""
|
498 |
+
)
|
499 |
+
|
500 |
+
with gr.Row():
|
501 |
+
run_button = gr.Button("运行", variant="primary")
|
502 |
+
stop_button = gr.Button("终止", variant="stop")
|
503 |
+
|
504 |
+
with gr.Column(scale=2):
|
505 |
+
with gr.Tabs():
|
506 |
+
with gr.TabItem("结果"):
|
507 |
+
status_output = gr.Textbox(label="状态")
|
508 |
+
answer_output = gr.Textbox(label="回答", lines=10)
|
509 |
+
log_file_output = gr.Textbox(label="日志文件路径")
|
510 |
+
|
511 |
+
with gr.TabItem("运行日志"):
|
512 |
+
log_output = gr.Textbox(label="完整日志", lines=25)
|
513 |
+
|
514 |
+
with gr.TabItem("聊天历史"):
|
515 |
+
chat_output = gr.Chatbot(label="对话历史")
|
516 |
+
|
517 |
+
# 示例问题
|
518 |
+
examples = [
|
519 |
+
[
|
520 |
+
"Qwen Mini (中文)",
|
521 |
+
"浏览亚马逊并找出一款对程序员有吸引力的产品。请提供产品名称和价格",
|
522 |
+
],
|
523 |
+
[
|
524 |
+
"DeepSeek (中文)",
|
525 |
+
"请分析GitHub上CAMEL-AI项目的最新统计数据。找出该项目的星标数量、贡献者数量和最近的活跃度。然后,创建一个简单的Excel表格来展示这些数据,并生成一个柱状图来可视化这些指标。最后,总结CAMEL项目的受欢迎程度和发展趋势。",
|
526 |
+
],
|
527 |
+
[
|
528 |
+
"Default",
|
529 |
+
"Navigate to Amazon.com and identify one product that is attractive to coders. Please provide me with the product name and price. No need to verify your answer.",
|
530 |
+
],
|
531 |
+
]
|
532 |
+
|
533 |
+
gr.Examples(examples=examples, inputs=[script_dropdown, question_input])
|
534 |
+
|
535 |
+
with gr.TabItem("环境变量配置"):
|
536 |
+
env_inputs = {}
|
537 |
+
save_status = gr.Textbox(label="保存状态", interactive=False)
|
538 |
+
|
539 |
+
# 添加自定义环境变量部分
|
540 |
+
with gr.Accordion("添加自定义环境变量", open=True):
|
541 |
+
with gr.Row():
|
542 |
+
new_var_name = gr.Textbox(
|
543 |
+
label="环境变量名", placeholder="例如:MY_CUSTOM_API_KEY"
|
544 |
+
)
|
545 |
+
new_var_value = gr.Textbox(
|
546 |
+
label="环境变量值", placeholder="输入值"
|
547 |
+
)
|
548 |
+
new_var_type = gr.Dropdown(
|
549 |
+
choices=["text", "password"], value="text", label="类型"
|
550 |
+
)
|
551 |
+
|
552 |
+
add_var_button = gr.Button("添加环境变量", variant="primary")
|
553 |
+
add_var_status = gr.Textbox(label="添加状态", interactive=False)
|
554 |
+
|
555 |
+
# 自定义环境变量列表
|
556 |
+
custom_vars_list = gr.JSON(
|
557 |
+
value=ENV_GROUPS["自定义环境变量"],
|
558 |
+
label="已添加的自定义环境变量",
|
559 |
+
visible=len(ENV_GROUPS["自定义环境变量"]) > 0,
|
560 |
+
)
|
561 |
+
|
562 |
+
# 添加环境变量按钮点击事件
|
563 |
+
add_var_button.click(
|
564 |
+
fn=add_custom_env_var,
|
565 |
+
inputs=[new_var_name, new_var_value, new_var_type],
|
566 |
+
outputs=[add_var_status, custom_vars_list],
|
567 |
+
)
|
568 |
+
|
569 |
+
# 现有环境变量配置
|
570 |
+
for group_name, vars in ENV_GROUPS.items():
|
571 |
+
if (
|
572 |
+
group_name != "自定义环境变量" or len(vars) > 0
|
573 |
+
): # 只显示非空的自定义环境变量组
|
574 |
+
with gr.Accordion(
|
575 |
+
group_name, open=(group_name != "自定义环境变量")
|
576 |
+
):
|
577 |
+
for var in vars:
|
578 |
+
# 添加帮助信息
|
579 |
+
gr.Markdown(f"**{var['help']}**")
|
580 |
+
|
581 |
+
if var["type"] == "password":
|
582 |
+
env_inputs[var["name"]] = gr.Textbox(
|
583 |
+
value=env_vars.get(var["name"], ""),
|
584 |
+
label=var["label"],
|
585 |
+
placeholder=f"请输入{var['label']}",
|
586 |
+
type="password",
|
587 |
+
)
|
588 |
+
else:
|
589 |
+
env_inputs[var["name"]] = gr.Textbox(
|
590 |
+
value=env_vars.get(var["name"], ""),
|
591 |
+
label=var["label"],
|
592 |
+
placeholder=f"请输入{var['label']}",
|
593 |
+
)
|
594 |
+
|
595 |
+
save_button = gr.Button("保存环境变量", variant="primary")
|
596 |
+
|
597 |
+
# 保存环境变量
|
598 |
+
save_inputs = [
|
599 |
+
env_inputs[var_name]
|
600 |
+
for group in ENV_GROUPS.values()
|
601 |
+
for var in group
|
602 |
+
for var_name in [var["name"]]
|
603 |
+
if var_name in env_inputs
|
604 |
+
]
|
605 |
+
save_button.click(
|
606 |
+
fn=lambda *values: save_env_vars(
|
607 |
+
dict(
|
608 |
+
zip(
|
609 |
+
[
|
610 |
+
var["name"]
|
611 |
+
for group in ENV_GROUPS.values()
|
612 |
+
for var in group
|
613 |
+
if var["name"] in env_inputs
|
614 |
+
],
|
615 |
+
values,
|
616 |
+
)
|
617 |
+
)
|
618 |
+
),
|
619 |
+
inputs=save_inputs,
|
620 |
+
outputs=save_status,
|
621 |
+
)
|
622 |
+
|
623 |
+
# 运行脚本
|
624 |
+
run_button.click(
|
625 |
+
fn=run_script,
|
626 |
+
inputs=[script_dropdown, question_input],
|
627 |
+
outputs=[
|
628 |
+
status_output,
|
629 |
+
answer_output,
|
630 |
+
log_output,
|
631 |
+
log_file_output,
|
632 |
+
chat_output,
|
633 |
+
],
|
634 |
+
show_progress=True,
|
635 |
+
)
|
636 |
+
|
637 |
+
# 终止运行
|
638 |
+
stop_button.click(fn=terminate_process, inputs=[], outputs=[status_output])
|
639 |
+
|
640 |
+
# 添加页脚
|
641 |
+
gr.Markdown(
|
642 |
+
"""
|
643 |
+
### 📝 使用说明
|
644 |
+
|
645 |
+
- 选择一个模型并输入您的问题
|
646 |
+
- 点击"运行"按钮开始执行
|
647 |
+
- 如需终止运行,点击"终止"按钮
|
648 |
+
- 在"结果"标签页查看执行状态和回答
|
649 |
+
- 在"运行日志"标签页查看完整日志
|
650 |
+
- 在"聊天历史"标签页查看对话历史(如果有)
|
651 |
+
- 在"环境变量配置"标签页配置API密钥和其他环境变量
|
652 |
+
- 您可以添加自定义环境变量,满足特殊需求
|
653 |
+
|
654 |
+
### ⚠️ 注意事项
|
655 |
+
|
656 |
+
- 运行某些模型可能需要API密钥,请确保在"环境变量配置"标签页中设置了相应的环境变量
|
657 |
+
- 某些脚本可能需要较长时间运行,请耐心等待
|
658 |
+
- 如果运行超过30分钟,进程将自动终止
|
659 |
+
- 您输入的问题将替换脚本中的默认问题,确保问题与所选模型兼容
|
660 |
+
"""
|
661 |
+
)
|
662 |
+
|
663 |
+
return app
|
664 |
+
|
665 |
+
|
666 |
+
if __name__ == "__main__":
|
667 |
+
# 创建并启动应用
|
668 |
+
app = create_ui()
|
669 |
+
app.queue().launch(share=True)
|
owl/run.py
ADDED
@@ -0,0 +1,138 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
from dotenv import load_dotenv
|
15 |
+
from camel.models import ModelFactory
|
16 |
+
from camel.toolkits import (
|
17 |
+
AudioAnalysisToolkit,
|
18 |
+
CodeExecutionToolkit,
|
19 |
+
ExcelToolkit,
|
20 |
+
ImageAnalysisToolkit,
|
21 |
+
SearchToolkit,
|
22 |
+
VideoAnalysisToolkit,
|
23 |
+
WebToolkit,
|
24 |
+
)
|
25 |
+
from camel.types import ModelPlatformType, ModelType
|
26 |
+
from camel.logger import set_log_level
|
27 |
+
|
28 |
+
from utils import OwlRolePlaying, run_society, DocumentProcessingToolkit
|
29 |
+
|
30 |
+
load_dotenv()
|
31 |
+
|
32 |
+
set_log_level(level="DEBUG")
|
33 |
+
|
34 |
+
|
35 |
+
def construct_society(question: str) -> OwlRolePlaying:
|
36 |
+
r"""Construct a society of agents based on the given question.
|
37 |
+
|
38 |
+
Args:
|
39 |
+
question (str): The task or question to be addressed by the society.
|
40 |
+
|
41 |
+
Returns:
|
42 |
+
OwlRolePlaying: A configured society of agents ready to address the question.
|
43 |
+
"""
|
44 |
+
|
45 |
+
# Create models for different components
|
46 |
+
models = {
|
47 |
+
"user": ModelFactory.create(
|
48 |
+
model_platform=ModelPlatformType.OPENAI,
|
49 |
+
model_type=ModelType.GPT_4O,
|
50 |
+
model_config_dict={"temperature": 0},
|
51 |
+
),
|
52 |
+
"assistant": ModelFactory.create(
|
53 |
+
model_platform=ModelPlatformType.OPENAI,
|
54 |
+
model_type=ModelType.GPT_4O,
|
55 |
+
model_config_dict={"temperature": 0},
|
56 |
+
),
|
57 |
+
"web": ModelFactory.create(
|
58 |
+
model_platform=ModelPlatformType.OPENAI,
|
59 |
+
model_type=ModelType.GPT_4O,
|
60 |
+
model_config_dict={"temperature": 0},
|
61 |
+
),
|
62 |
+
"planning": ModelFactory.create(
|
63 |
+
model_platform=ModelPlatformType.OPENAI,
|
64 |
+
model_type=ModelType.GPT_4O,
|
65 |
+
model_config_dict={"temperature": 0},
|
66 |
+
),
|
67 |
+
"video": ModelFactory.create(
|
68 |
+
model_platform=ModelPlatformType.OPENAI,
|
69 |
+
model_type=ModelType.GPT_4O,
|
70 |
+
model_config_dict={"temperature": 0},
|
71 |
+
),
|
72 |
+
"image": ModelFactory.create(
|
73 |
+
model_platform=ModelPlatformType.OPENAI,
|
74 |
+
model_type=ModelType.GPT_4O,
|
75 |
+
model_config_dict={"temperature": 0},
|
76 |
+
),
|
77 |
+
"document": ModelFactory.create(
|
78 |
+
model_platform=ModelPlatformType.OPENAI,
|
79 |
+
model_type=ModelType.GPT_4O,
|
80 |
+
model_config_dict={"temperature": 0},
|
81 |
+
),
|
82 |
+
}
|
83 |
+
|
84 |
+
# Configure toolkits
|
85 |
+
tools = [
|
86 |
+
*WebToolkit(
|
87 |
+
headless=False, # Set to True for headless mode (e.g., on remote servers)
|
88 |
+
web_agent_model=models["web"],
|
89 |
+
planning_agent_model=models["planning"],
|
90 |
+
).get_tools(),
|
91 |
+
*VideoAnalysisToolkit(model=models["video"]).get_tools(),
|
92 |
+
*AudioAnalysisToolkit().get_tools(), # This requires OpenAI Key
|
93 |
+
*CodeExecutionToolkit(sandbox="subprocess", verbose=True).get_tools(),
|
94 |
+
*ImageAnalysisToolkit(model=models["image"]).get_tools(),
|
95 |
+
SearchToolkit().search_duckduckgo,
|
96 |
+
SearchToolkit().search_google, # Comment this out if you don't have google search
|
97 |
+
SearchToolkit().search_wiki,
|
98 |
+
*ExcelToolkit().get_tools(),
|
99 |
+
*DocumentProcessingToolkit(model=models["document"]).get_tools(),
|
100 |
+
]
|
101 |
+
|
102 |
+
# Configure agent roles and parameters
|
103 |
+
user_agent_kwargs = {"model": models["user"]}
|
104 |
+
assistant_agent_kwargs = {"model": models["assistant"], "tools": tools}
|
105 |
+
|
106 |
+
# Configure task parameters
|
107 |
+
task_kwargs = {
|
108 |
+
"task_prompt": question,
|
109 |
+
"with_task_specify": False,
|
110 |
+
}
|
111 |
+
|
112 |
+
# Create and return the society
|
113 |
+
society = OwlRolePlaying(
|
114 |
+
**task_kwargs,
|
115 |
+
user_role_name="user",
|
116 |
+
user_agent_kwargs=user_agent_kwargs,
|
117 |
+
assistant_role_name="assistant",
|
118 |
+
assistant_agent_kwargs=assistant_agent_kwargs,
|
119 |
+
)
|
120 |
+
|
121 |
+
return society
|
122 |
+
|
123 |
+
|
124 |
+
def main():
|
125 |
+
r"""Main function to run the OWL system with an example question."""
|
126 |
+
# Example research question
|
127 |
+
question = "Navigate to Amazon.com and identify one product that is attractive to coders. Please provide me with the product name and price. No need to verify your answer."
|
128 |
+
|
129 |
+
# Construct and run the society
|
130 |
+
society = construct_society(question)
|
131 |
+
answer, chat_history, token_count = run_society(society)
|
132 |
+
|
133 |
+
# Output the result
|
134 |
+
print(f"\033[94mAnswer: {answer}\033[0m")
|
135 |
+
|
136 |
+
|
137 |
+
if __name__ == "__main__":
|
138 |
+
main()
|
owl/run_deepseek_zh.py
ADDED
@@ -0,0 +1,141 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
|
15 |
+
|
16 |
+
# To run this file, you need to configure the DeepSeek API key
|
17 |
+
# You can obtain your API key from DeepSeek platform: https://platform.deepseek.com/api_keys
|
18 |
+
# Set it as DEEPSEEK_API_KEY="your-api-key" in your .env file or add it to your environment variables
|
19 |
+
|
20 |
+
|
21 |
+
from dotenv import load_dotenv
|
22 |
+
|
23 |
+
|
24 |
+
from camel.models import ModelFactory
|
25 |
+
from camel.toolkits import (
|
26 |
+
CodeExecutionToolkit,
|
27 |
+
ExcelToolkit,
|
28 |
+
SearchToolkit,
|
29 |
+
)
|
30 |
+
from camel.types import ModelPlatformType, ModelType
|
31 |
+
|
32 |
+
|
33 |
+
from utils import OwlRolePlaying, run_society, DocumentProcessingToolkit
|
34 |
+
|
35 |
+
from camel.logger import set_log_level
|
36 |
+
|
37 |
+
set_log_level(level="DEBUG")
|
38 |
+
|
39 |
+
load_dotenv()
|
40 |
+
|
41 |
+
|
42 |
+
def construct_society(question: str) -> OwlRolePlaying:
|
43 |
+
r"""Construct a society of agents based on the given question.
|
44 |
+
|
45 |
+
Args:
|
46 |
+
question (str): The task or question to be addressed by the society.
|
47 |
+
|
48 |
+
Returns:
|
49 |
+
OwlRolePlaying: A configured society of agents ready to address the question.
|
50 |
+
"""
|
51 |
+
|
52 |
+
# Create models for different components
|
53 |
+
models = {
|
54 |
+
"user": ModelFactory.create(
|
55 |
+
model_platform=ModelPlatformType.DEEPSEEK,
|
56 |
+
model_type=ModelType.DEEPSEEK_CHAT,
|
57 |
+
model_config_dict={"temperature": 0},
|
58 |
+
),
|
59 |
+
"assistant": ModelFactory.create(
|
60 |
+
model_platform=ModelPlatformType.DEEPSEEK,
|
61 |
+
model_type=ModelType.DEEPSEEK_CHAT,
|
62 |
+
model_config_dict={"temperature": 0},
|
63 |
+
),
|
64 |
+
"web": ModelFactory.create(
|
65 |
+
model_platform=ModelPlatformType.DEEPSEEK,
|
66 |
+
model_type=ModelType.DEEPSEEK_CHAT,
|
67 |
+
model_config_dict={"temperature": 0},
|
68 |
+
),
|
69 |
+
"planning": ModelFactory.create(
|
70 |
+
model_platform=ModelPlatformType.DEEPSEEK,
|
71 |
+
model_type=ModelType.DEEPSEEK_CHAT,
|
72 |
+
model_config_dict={"temperature": 0},
|
73 |
+
),
|
74 |
+
"video": ModelFactory.create(
|
75 |
+
model_platform=ModelPlatformType.DEEPSEEK,
|
76 |
+
model_type=ModelType.DEEPSEEK_CHAT,
|
77 |
+
model_config_dict={"temperature": 0},
|
78 |
+
),
|
79 |
+
"image": ModelFactory.create(
|
80 |
+
model_platform=ModelPlatformType.DEEPSEEK,
|
81 |
+
model_type=ModelType.DEEPSEEK_CHAT,
|
82 |
+
model_config_dict={"temperature": 0},
|
83 |
+
),
|
84 |
+
"document": ModelFactory.create(
|
85 |
+
model_platform=ModelPlatformType.DEEPSEEK,
|
86 |
+
model_type=ModelType.DEEPSEEK_CHAT,
|
87 |
+
model_config_dict={"temperature": 0},
|
88 |
+
),
|
89 |
+
}
|
90 |
+
|
91 |
+
# Configure toolkits
|
92 |
+
tools = [
|
93 |
+
*CodeExecutionToolkit(sandbox="subprocess", verbose=True).get_tools(),
|
94 |
+
SearchToolkit().search_duckduckgo,
|
95 |
+
SearchToolkit().search_wiki,
|
96 |
+
*ExcelToolkit().get_tools(),
|
97 |
+
*DocumentProcessingToolkit(model=models["document"]).get_tools(),
|
98 |
+
]
|
99 |
+
|
100 |
+
# Configure agent roles and parameters
|
101 |
+
user_agent_kwargs = {"model": models["user"]}
|
102 |
+
assistant_agent_kwargs = {"model": models["assistant"], "tools": tools}
|
103 |
+
|
104 |
+
# Configure task parameters
|
105 |
+
task_kwargs = {
|
106 |
+
"task_prompt": question,
|
107 |
+
"with_task_specify": False,
|
108 |
+
}
|
109 |
+
|
110 |
+
# Create and return the society
|
111 |
+
society = OwlRolePlaying(
|
112 |
+
**task_kwargs,
|
113 |
+
user_role_name="user",
|
114 |
+
user_agent_kwargs=user_agent_kwargs,
|
115 |
+
assistant_role_name="assistant",
|
116 |
+
assistant_agent_kwargs=assistant_agent_kwargs,
|
117 |
+
output_language="Chinese",
|
118 |
+
)
|
119 |
+
|
120 |
+
return society
|
121 |
+
|
122 |
+
|
123 |
+
def main():
|
124 |
+
r"""Main function to run the OWL system with an example question."""
|
125 |
+
# Example research question
|
126 |
+
question = (
|
127 |
+
"请分析GitHub上CAMEL-AI项目的最新统计数据。找出该项目的星标数量、"
|
128 |
+
"贡献者数量和最近的活跃度。然后,创建一个简单的Excel表格来展示这些数据,"
|
129 |
+
"并生成一个柱状图来可视化这些指标。最后,总结CAMEL项目的受欢迎程度和发展趋势。"
|
130 |
+
)
|
131 |
+
|
132 |
+
# Construct and run the society
|
133 |
+
society = construct_society(question)
|
134 |
+
answer, chat_history, token_count = run_society(society)
|
135 |
+
|
136 |
+
# Output the result
|
137 |
+
print(f"\033[94mAnswer: {answer}\033[0m")
|
138 |
+
|
139 |
+
|
140 |
+
if __name__ == "__main__":
|
141 |
+
main()
|
owl/run_gaia_roleplaying.py
ADDED
@@ -0,0 +1,135 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
|
15 |
+
from dotenv import load_dotenv
|
16 |
+
|
17 |
+
|
18 |
+
import os
|
19 |
+
|
20 |
+
from camel.models import ModelFactory
|
21 |
+
from camel.logger import get_logger
|
22 |
+
from camel.toolkits import (
|
23 |
+
AudioAnalysisToolkit,
|
24 |
+
CodeExecutionToolkit,
|
25 |
+
ExcelToolkit,
|
26 |
+
ImageAnalysisToolkit,
|
27 |
+
SearchToolkit,
|
28 |
+
VideoAnalysisToolkit,
|
29 |
+
WebToolkit,
|
30 |
+
)
|
31 |
+
from camel.types import ModelPlatformType, ModelType
|
32 |
+
from camel.configs import ChatGPTConfig
|
33 |
+
|
34 |
+
from utils import GAIABenchmark
|
35 |
+
from camel.logger import set_log_level
|
36 |
+
|
37 |
+
set_log_level(level="DEBUG")
|
38 |
+
|
39 |
+
load_dotenv()
|
40 |
+
|
41 |
+
logger = get_logger(__name__)
|
42 |
+
|
43 |
+
# Configuration
|
44 |
+
LEVEL = 1
|
45 |
+
SAVE_RESULT = True
|
46 |
+
test_idx = [0]
|
47 |
+
|
48 |
+
|
49 |
+
def main():
|
50 |
+
"""Main function to run the GAIA benchmark."""
|
51 |
+
# Create cache directory
|
52 |
+
cache_dir = "tmp/"
|
53 |
+
os.makedirs(cache_dir, exist_ok=True)
|
54 |
+
|
55 |
+
# Create models for different components
|
56 |
+
models = {
|
57 |
+
"user": ModelFactory.create(
|
58 |
+
model_platform=ModelPlatformType.OPENAI,
|
59 |
+
model_type=ModelType.GPT_4O,
|
60 |
+
model_config_dict=ChatGPTConfig(temperature=0, top_p=1).as_dict(),
|
61 |
+
),
|
62 |
+
"assistant": ModelFactory.create(
|
63 |
+
model_platform=ModelPlatformType.OPENAI,
|
64 |
+
model_type=ModelType.GPT_4O,
|
65 |
+
model_config_dict=ChatGPTConfig(temperature=0, top_p=1).as_dict(),
|
66 |
+
),
|
67 |
+
"web": ModelFactory.create(
|
68 |
+
model_platform=ModelPlatformType.OPENAI,
|
69 |
+
model_type=ModelType.GPT_4O,
|
70 |
+
model_config_dict=ChatGPTConfig(temperature=0, top_p=1).as_dict(),
|
71 |
+
),
|
72 |
+
"planning": ModelFactory.create(
|
73 |
+
model_platform=ModelPlatformType.OPENAI,
|
74 |
+
model_type=ModelType.GPT_4O,
|
75 |
+
model_config_dict=ChatGPTConfig(temperature=0, top_p=1).as_dict(),
|
76 |
+
),
|
77 |
+
"video": ModelFactory.create(
|
78 |
+
model_platform=ModelPlatformType.OPENAI,
|
79 |
+
model_type=ModelType.GPT_4O,
|
80 |
+
model_config_dict=ChatGPTConfig(temperature=0, top_p=1).as_dict(),
|
81 |
+
),
|
82 |
+
"image": ModelFactory.create(
|
83 |
+
model_platform=ModelPlatformType.OPENAI,
|
84 |
+
model_type=ModelType.GPT_4O,
|
85 |
+
model_config_dict=ChatGPTConfig(temperature=0, top_p=1).as_dict(),
|
86 |
+
),
|
87 |
+
}
|
88 |
+
|
89 |
+
# Configure toolkits
|
90 |
+
tools = [
|
91 |
+
*WebToolkit(
|
92 |
+
headless=False, # Set to True for headless mode (e.g., on remote servers)
|
93 |
+
web_agent_model=models["web"],
|
94 |
+
planning_agent_model=models["planning"],
|
95 |
+
).get_tools(),
|
96 |
+
*VideoAnalysisToolkit(
|
97 |
+
model=models["video"]
|
98 |
+
).get_tools(), # This requires OpenAI Key
|
99 |
+
*AudioAnalysisToolkit().get_tools(), # This requires OpenAI Key
|
100 |
+
*CodeExecutionToolkit(sandbox="subprocess", verbose=True).get_tools(),
|
101 |
+
*ImageAnalysisToolkit(model=models["image"]).get_tools(),
|
102 |
+
*SearchToolkit().get_tools(),
|
103 |
+
*ExcelToolkit().get_tools(),
|
104 |
+
]
|
105 |
+
|
106 |
+
# Configure agent roles and parameters
|
107 |
+
user_agent_kwargs = {"model": models["user"]}
|
108 |
+
assistant_agent_kwargs = {"model": models["assistant"], "tools": tools}
|
109 |
+
|
110 |
+
# Initialize benchmark
|
111 |
+
benchmark = GAIABenchmark(data_dir="data/gaia", save_to="results/result.json")
|
112 |
+
|
113 |
+
# Print benchmark information
|
114 |
+
print(f"Number of validation examples: {len(benchmark.valid)}")
|
115 |
+
print(f"Number of test examples: {len(benchmark.test)}")
|
116 |
+
|
117 |
+
# Run benchmark
|
118 |
+
result = benchmark.run(
|
119 |
+
on="valid",
|
120 |
+
level=LEVEL,
|
121 |
+
idx=test_idx,
|
122 |
+
save_result=SAVE_RESULT,
|
123 |
+
user_role_name="user",
|
124 |
+
user_agent_kwargs=user_agent_kwargs,
|
125 |
+
assistant_role_name="assistant",
|
126 |
+
assistant_agent_kwargs=assistant_agent_kwargs,
|
127 |
+
)
|
128 |
+
|
129 |
+
# Output results
|
130 |
+
logger.success(f"Correct: {result['correct']}, Total: {result['total']}")
|
131 |
+
logger.success(f"Accuracy: {result['accuracy']}")
|
132 |
+
|
133 |
+
|
134 |
+
if __name__ == "__main__":
|
135 |
+
main()
|
owl/run_mini.py
ADDED
@@ -0,0 +1,112 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
from dotenv import load_dotenv
|
15 |
+
|
16 |
+
from camel.models import ModelFactory
|
17 |
+
from camel.toolkits import (
|
18 |
+
SearchToolkit,
|
19 |
+
WebToolkit,
|
20 |
+
)
|
21 |
+
from camel.types import ModelPlatformType, ModelType
|
22 |
+
from camel.logger import set_log_level
|
23 |
+
|
24 |
+
from utils import OwlRolePlaying, run_society
|
25 |
+
|
26 |
+
load_dotenv()
|
27 |
+
set_log_level(level="DEBUG")
|
28 |
+
|
29 |
+
|
30 |
+
def construct_society(question: str) -> OwlRolePlaying:
|
31 |
+
r"""Construct a society of agents based on the given question.
|
32 |
+
|
33 |
+
Args:
|
34 |
+
question (str): The task or question to be addressed by the society.
|
35 |
+
|
36 |
+
Returns:
|
37 |
+
OwlRolePlaying: A configured society of agents ready to address the
|
38 |
+
question.
|
39 |
+
"""
|
40 |
+
|
41 |
+
# Create models for different components
|
42 |
+
models = {
|
43 |
+
"user": ModelFactory.create(
|
44 |
+
model_platform=ModelPlatformType.OPENAI,
|
45 |
+
model_type=ModelType.GPT_4O,
|
46 |
+
model_config_dict={"temperature": 0},
|
47 |
+
),
|
48 |
+
"assistant": ModelFactory.create(
|
49 |
+
model_platform=ModelPlatformType.OPENAI,
|
50 |
+
model_type=ModelType.GPT_4O,
|
51 |
+
model_config_dict={"temperature": 0},
|
52 |
+
),
|
53 |
+
"web": ModelFactory.create(
|
54 |
+
model_platform=ModelPlatformType.OPENAI,
|
55 |
+
model_type=ModelType.GPT_4O,
|
56 |
+
model_config_dict={"temperature": 0},
|
57 |
+
),
|
58 |
+
"planning": ModelFactory.create(
|
59 |
+
model_platform=ModelPlatformType.OPENAI,
|
60 |
+
model_type=ModelType.GPT_4O,
|
61 |
+
model_config_dict={"temperature": 0},
|
62 |
+
),
|
63 |
+
}
|
64 |
+
|
65 |
+
# Configure toolkits
|
66 |
+
tools = [
|
67 |
+
*WebToolkit(
|
68 |
+
headless=False, # Set to True for headless mode (e.g., on remote servers)
|
69 |
+
web_agent_model=models["web"],
|
70 |
+
planning_agent_model=models["planning"],
|
71 |
+
).get_tools(),
|
72 |
+
SearchToolkit().search_duckduckgo,
|
73 |
+
SearchToolkit().search_wiki,
|
74 |
+
]
|
75 |
+
|
76 |
+
# Configure agent roles and parameters
|
77 |
+
user_agent_kwargs = {"model": models["user"]}
|
78 |
+
assistant_agent_kwargs = {"model": models["assistant"], "tools": tools}
|
79 |
+
|
80 |
+
# Configure task parameters
|
81 |
+
task_kwargs = {
|
82 |
+
"task_prompt": question,
|
83 |
+
"with_task_specify": False,
|
84 |
+
}
|
85 |
+
|
86 |
+
# Create and return the society
|
87 |
+
society = OwlRolePlaying(
|
88 |
+
**task_kwargs,
|
89 |
+
user_role_name="user",
|
90 |
+
user_agent_kwargs=user_agent_kwargs,
|
91 |
+
assistant_role_name="assistant",
|
92 |
+
assistant_agent_kwargs=assistant_agent_kwargs,
|
93 |
+
)
|
94 |
+
|
95 |
+
return society
|
96 |
+
|
97 |
+
|
98 |
+
def main():
|
99 |
+
r"""Main function to run the OWL system with an example question."""
|
100 |
+
# Example research question
|
101 |
+
question = "Navigate to Amazon.com and identify one product that is attractive to coders. Please provide me with the product name and price. No need to verify your answer."
|
102 |
+
|
103 |
+
# Construct and run the society
|
104 |
+
society = construct_society(question)
|
105 |
+
answer, chat_history, token_count = run_society(society)
|
106 |
+
|
107 |
+
# Output the result
|
108 |
+
print(f"\033[94mAnswer: {answer}\033[0m")
|
109 |
+
|
110 |
+
|
111 |
+
if __name__ == "__main__":
|
112 |
+
main()
|
owl/run_openai_compatiable_model.py
ADDED
@@ -0,0 +1,136 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
import os
|
15 |
+
|
16 |
+
from dotenv import load_dotenv
|
17 |
+
from camel.models import ModelFactory
|
18 |
+
from camel.toolkits import (
|
19 |
+
CodeExecutionToolkit,
|
20 |
+
ExcelToolkit,
|
21 |
+
ImageAnalysisToolkit,
|
22 |
+
SearchToolkit,
|
23 |
+
WebToolkit,
|
24 |
+
)
|
25 |
+
from camel.types import ModelPlatformType
|
26 |
+
|
27 |
+
from utils import OwlRolePlaying, run_society
|
28 |
+
|
29 |
+
from camel.logger import set_log_level
|
30 |
+
|
31 |
+
set_log_level(level="DEBUG")
|
32 |
+
|
33 |
+
load_dotenv()
|
34 |
+
|
35 |
+
|
36 |
+
def construct_society(question: str) -> OwlRolePlaying:
|
37 |
+
r"""Construct a society of agents based on the given question.
|
38 |
+
|
39 |
+
Args:
|
40 |
+
question (str): The task or question to be addressed by the society.
|
41 |
+
|
42 |
+
Returns:
|
43 |
+
OwlRolePlaying: A configured society of agents ready to address the question.
|
44 |
+
"""
|
45 |
+
|
46 |
+
# Create models for different components
|
47 |
+
models = {
|
48 |
+
"user": ModelFactory.create(
|
49 |
+
model_platform=ModelPlatformType.OPENAI_COMPATIBLE_MODEL,
|
50 |
+
model_type="qwen-max",
|
51 |
+
api_key=os.getenv("QWEN_API_KEY"),
|
52 |
+
url="https://dashscope.aliyuncs.com/compatible-mode/v1",
|
53 |
+
model_config_dict={"temperature": 0.4, "max_tokens": 4096},
|
54 |
+
),
|
55 |
+
"assistant": ModelFactory.create(
|
56 |
+
model_platform=ModelPlatformType.OPENAI_COMPATIBLE_MODEL,
|
57 |
+
model_type="qwen-max",
|
58 |
+
api_key=os.getenv("QWEN_API_KEY"),
|
59 |
+
url="https://dashscope.aliyuncs.com/compatible-mode/v1",
|
60 |
+
model_config_dict={"temperature": 0.4, "max_tokens": 4096},
|
61 |
+
),
|
62 |
+
"web": ModelFactory.create(
|
63 |
+
model_platform=ModelPlatformType.OPENAI_COMPATIBLE_MODEL,
|
64 |
+
model_type="qwen-vl-max",
|
65 |
+
api_key=os.getenv("QWEN_API_KEY"),
|
66 |
+
url="https://dashscope.aliyuncs.com/compatible-mode/v1",
|
67 |
+
model_config_dict={"temperature": 0.4, "max_tokens": 4096},
|
68 |
+
),
|
69 |
+
"planning": ModelFactory.create(
|
70 |
+
model_platform=ModelPlatformType.OPENAI_COMPATIBLE_MODEL,
|
71 |
+
model_type="qwen-max",
|
72 |
+
api_key=os.getenv("QWEN_API_KEY"),
|
73 |
+
url="https://dashscope.aliyuncs.com/compatible-mode/v1",
|
74 |
+
model_config_dict={"temperature": 0.4, "max_tokens": 4096},
|
75 |
+
),
|
76 |
+
"image": ModelFactory.create(
|
77 |
+
model_platform=ModelPlatformType.OPENAI_COMPATIBLE_MODEL,
|
78 |
+
model_type="qwen-vl-max",
|
79 |
+
api_key=os.getenv("QWEN_API_KEY"),
|
80 |
+
url="https://dashscope.aliyuncs.com/compatible-mode/v1",
|
81 |
+
model_config_dict={"temperature": 0.4, "max_tokens": 4096},
|
82 |
+
),
|
83 |
+
}
|
84 |
+
|
85 |
+
# Configure toolkits
|
86 |
+
tools = [
|
87 |
+
*WebToolkit(
|
88 |
+
headless=False, # Set to True for headless mode (e.g., on remote servers)
|
89 |
+
web_agent_model=models["web"],
|
90 |
+
planning_agent_model=models["planning"],
|
91 |
+
).get_tools(),
|
92 |
+
*CodeExecutionToolkit(sandbox="subprocess", verbose=True).get_tools(),
|
93 |
+
*ImageAnalysisToolkit(model=models["image"]).get_tools(),
|
94 |
+
SearchToolkit().search_duckduckgo,
|
95 |
+
SearchToolkit().search_google, # Comment this out if you don't have google search
|
96 |
+
SearchToolkit().search_wiki,
|
97 |
+
*ExcelToolkit().get_tools(),
|
98 |
+
]
|
99 |
+
|
100 |
+
# Configure agent roles and parameters
|
101 |
+
user_agent_kwargs = {"model": models["user"]}
|
102 |
+
assistant_agent_kwargs = {"model": models["assistant"], "tools": tools}
|
103 |
+
|
104 |
+
# Configure task parameters
|
105 |
+
task_kwargs = {
|
106 |
+
"task_prompt": question,
|
107 |
+
"with_task_specify": False,
|
108 |
+
}
|
109 |
+
|
110 |
+
# Create and return the society
|
111 |
+
society = OwlRolePlaying(
|
112 |
+
**task_kwargs,
|
113 |
+
user_role_name="user",
|
114 |
+
user_agent_kwargs=user_agent_kwargs,
|
115 |
+
assistant_role_name="assistant",
|
116 |
+
assistant_agent_kwargs=assistant_agent_kwargs,
|
117 |
+
)
|
118 |
+
|
119 |
+
return society
|
120 |
+
|
121 |
+
|
122 |
+
def main():
|
123 |
+
r"""Main function to run the OWL system with an example question."""
|
124 |
+
# Example research question
|
125 |
+
question = "Navigate to Amazon.com and identify one product that is attractive to coders. Please provide me with the product name and price. No need to verify your answer."
|
126 |
+
|
127 |
+
# Construct and run the society
|
128 |
+
society = construct_society(question)
|
129 |
+
answer, chat_history, token_count = run_society(society)
|
130 |
+
|
131 |
+
# Output the result
|
132 |
+
print(f"\033[94mAnswer: {answer}\033[0m")
|
133 |
+
|
134 |
+
|
135 |
+
if __name__ == "__main__":
|
136 |
+
main()
|
owl/run_qwen_mini_zh.py
ADDED
@@ -0,0 +1,102 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
|
15 |
+
# To run this file, you need to configure the Qwen API key
|
16 |
+
# You can obtain your API key from Bailian platform: bailian.console.aliyun.com
|
17 |
+
# Set it as QWEN_API_KEY="your-api-key" in your .env file or add it to your environment variables
|
18 |
+
|
19 |
+
from dotenv import load_dotenv
|
20 |
+
|
21 |
+
from camel.models import ModelFactory
|
22 |
+
from camel.toolkits import WebToolkit, SearchToolkit
|
23 |
+
from camel.types import ModelPlatformType, ModelType
|
24 |
+
|
25 |
+
from utils import OwlRolePlaying, run_society
|
26 |
+
|
27 |
+
from camel.logger import set_log_level
|
28 |
+
|
29 |
+
set_log_level(level="DEBUG")
|
30 |
+
|
31 |
+
load_dotenv()
|
32 |
+
|
33 |
+
|
34 |
+
def construct_society(question: str) -> OwlRolePlaying:
|
35 |
+
r"""Construct the society based on the question."""
|
36 |
+
|
37 |
+
user_role_name = "user"
|
38 |
+
assistant_role_name = "assistant"
|
39 |
+
|
40 |
+
user_model = ModelFactory.create(
|
41 |
+
model_platform=ModelPlatformType.QWEN,
|
42 |
+
model_type=ModelType.QWEN_VL_MAX,
|
43 |
+
model_config_dict={"temperature": 0},
|
44 |
+
)
|
45 |
+
|
46 |
+
assistant_model = ModelFactory.create(
|
47 |
+
model_platform=ModelPlatformType.QWEN,
|
48 |
+
model_type=ModelType.QWEN_VL_MAX,
|
49 |
+
model_config_dict={"temperature": 0},
|
50 |
+
)
|
51 |
+
|
52 |
+
planning_model = ModelFactory.create(
|
53 |
+
model_platform=ModelPlatformType.QWEN,
|
54 |
+
model_type=ModelType.QWEN_VL_MAX,
|
55 |
+
model_config_dict={"temperature": 0},
|
56 |
+
)
|
57 |
+
|
58 |
+
web_model = ModelFactory.create(
|
59 |
+
model_platform=ModelPlatformType.QWEN,
|
60 |
+
model_type=ModelType.QWEN_VL_MAX,
|
61 |
+
model_config_dict={"temperature": 0},
|
62 |
+
)
|
63 |
+
|
64 |
+
tools_list = [
|
65 |
+
*WebToolkit(
|
66 |
+
headless=False,
|
67 |
+
web_agent_model=web_model,
|
68 |
+
planning_agent_model=planning_model,
|
69 |
+
output_language="Chinese",
|
70 |
+
).get_tools(),
|
71 |
+
SearchToolkit().search_duckduckgo,
|
72 |
+
]
|
73 |
+
|
74 |
+
user_role_name = "user"
|
75 |
+
user_agent_kwargs = dict(model=user_model)
|
76 |
+
assistant_role_name = "assistant"
|
77 |
+
assistant_agent_kwargs = dict(model=assistant_model, tools=tools_list)
|
78 |
+
|
79 |
+
task_kwargs = {
|
80 |
+
"task_prompt": question,
|
81 |
+
"with_task_specify": False,
|
82 |
+
}
|
83 |
+
|
84 |
+
society = OwlRolePlaying(
|
85 |
+
**task_kwargs,
|
86 |
+
user_role_name=user_role_name,
|
87 |
+
user_agent_kwargs=user_agent_kwargs,
|
88 |
+
assistant_role_name=assistant_role_name,
|
89 |
+
assistant_agent_kwargs=assistant_agent_kwargs,
|
90 |
+
output_language="Chinese",
|
91 |
+
)
|
92 |
+
|
93 |
+
return society
|
94 |
+
|
95 |
+
|
96 |
+
# Example case
|
97 |
+
question = "浏览亚马逊并找出一款对程序员有吸引力的产品。请提供产品名称和价格"
|
98 |
+
|
99 |
+
society = construct_society(question)
|
100 |
+
answer, chat_history, token_count = run_society(society)
|
101 |
+
|
102 |
+
print(f"\033[94mAnswer: {answer}\033[0m")
|
owl/run_qwen_zh.py
ADDED
@@ -0,0 +1,145 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
|
15 |
+
# To run this file, you need to configure the Qwen API key
|
16 |
+
# You can obtain your API key from Bailian platform: bailian.console.aliyun.com
|
17 |
+
# Set it as QWEN_API_KEY="your-api-key" in your .env file or add it to your environment variables
|
18 |
+
|
19 |
+
from dotenv import load_dotenv
|
20 |
+
from camel.models import ModelFactory
|
21 |
+
from camel.toolkits import (
|
22 |
+
CodeExecutionToolkit,
|
23 |
+
ExcelToolkit,
|
24 |
+
ImageAnalysisToolkit,
|
25 |
+
SearchToolkit,
|
26 |
+
VideoAnalysisToolkit,
|
27 |
+
WebToolkit,
|
28 |
+
)
|
29 |
+
from camel.types import ModelPlatformType, ModelType
|
30 |
+
|
31 |
+
from utils import OwlRolePlaying, run_society, DocumentProcessingToolkit
|
32 |
+
|
33 |
+
from camel.logger import set_log_level
|
34 |
+
|
35 |
+
set_log_level(level="DEBUG")
|
36 |
+
|
37 |
+
load_dotenv()
|
38 |
+
|
39 |
+
|
40 |
+
def construct_society(question: str) -> OwlRolePlaying:
|
41 |
+
"""
|
42 |
+
Construct a society of agents based on the given question.
|
43 |
+
|
44 |
+
Args:
|
45 |
+
question (str): The task or question to be addressed by the society.
|
46 |
+
|
47 |
+
Returns:
|
48 |
+
OwlRolePlaying: A configured society of agents ready to address the question.
|
49 |
+
"""
|
50 |
+
|
51 |
+
# Create models for different components
|
52 |
+
models = {
|
53 |
+
"user": ModelFactory.create(
|
54 |
+
model_platform=ModelPlatformType.QWEN,
|
55 |
+
model_type=ModelType.QWEN_VL_MAX,
|
56 |
+
model_config_dict={"temperature": 0},
|
57 |
+
),
|
58 |
+
"assistant": ModelFactory.create(
|
59 |
+
model_platform=ModelPlatformType.QWEN,
|
60 |
+
model_type=ModelType.QWEN_VL_MAX,
|
61 |
+
model_config_dict={"temperature": 0},
|
62 |
+
),
|
63 |
+
"web": ModelFactory.create(
|
64 |
+
model_platform=ModelPlatformType.QWEN,
|
65 |
+
model_type=ModelType.QWEN_VL_MAX,
|
66 |
+
model_config_dict={"temperature": 0},
|
67 |
+
),
|
68 |
+
"planning": ModelFactory.create(
|
69 |
+
model_platform=ModelPlatformType.QWEN,
|
70 |
+
model_type=ModelType.QWEN_VL_MAX,
|
71 |
+
model_config_dict={"temperature": 0},
|
72 |
+
),
|
73 |
+
"video": ModelFactory.create(
|
74 |
+
model_platform=ModelPlatformType.QWEN,
|
75 |
+
model_type=ModelType.QWEN_VL_MAX,
|
76 |
+
model_config_dict={"temperature": 0},
|
77 |
+
),
|
78 |
+
"image": ModelFactory.create(
|
79 |
+
model_platform=ModelPlatformType.QWEN,
|
80 |
+
model_type=ModelType.QWEN_VL_MAX,
|
81 |
+
model_config_dict={"temperature": 0},
|
82 |
+
),
|
83 |
+
"document": ModelFactory.create(
|
84 |
+
model_platform=ModelPlatformType.QWEN,
|
85 |
+
model_type=ModelType.QWEN_VL_MAX,
|
86 |
+
model_config_dict={"temperature": 0},
|
87 |
+
),
|
88 |
+
}
|
89 |
+
|
90 |
+
# Configure toolkits
|
91 |
+
tools = [
|
92 |
+
*WebToolkit(
|
93 |
+
headless=False, # Set to True for headless mode (e.g., on remote servers)
|
94 |
+
web_agent_model=models["web"],
|
95 |
+
planning_agent_model=models["planning"],
|
96 |
+
output_language="Chinese",
|
97 |
+
).get_tools(),
|
98 |
+
*VideoAnalysisToolkit(model=models["video"]).get_tools(),
|
99 |
+
*CodeExecutionToolkit(sandbox="subprocess", verbose=True).get_tools(),
|
100 |
+
*ImageAnalysisToolkit(model=models["image"]).get_tools(),
|
101 |
+
SearchToolkit().search_duckduckgo,
|
102 |
+
SearchToolkit().search_google, # Comment this out if you don't have google search
|
103 |
+
SearchToolkit().search_wiki,
|
104 |
+
*ExcelToolkit().get_tools(),
|
105 |
+
*DocumentProcessingToolkit(model=models["document"]).get_tools(),
|
106 |
+
]
|
107 |
+
|
108 |
+
# Configure agent roles and parameters
|
109 |
+
user_agent_kwargs = {"model": models["user"]}
|
110 |
+
assistant_agent_kwargs = {"model": models["assistant"], "tools": tools}
|
111 |
+
|
112 |
+
# Configure task parameters
|
113 |
+
task_kwargs = {
|
114 |
+
"task_prompt": question,
|
115 |
+
"with_task_specify": False,
|
116 |
+
}
|
117 |
+
|
118 |
+
# Create and return the society
|
119 |
+
society = OwlRolePlaying(
|
120 |
+
**task_kwargs,
|
121 |
+
user_role_name="user",
|
122 |
+
user_agent_kwargs=user_agent_kwargs,
|
123 |
+
assistant_role_name="assistant",
|
124 |
+
assistant_agent_kwargs=assistant_agent_kwargs,
|
125 |
+
output_language="Chinese",
|
126 |
+
)
|
127 |
+
|
128 |
+
return society
|
129 |
+
|
130 |
+
|
131 |
+
def main():
|
132 |
+
r"""Main function to run the OWL system with an example question."""
|
133 |
+
# Example research question
|
134 |
+
question = "浏览亚马逊并找出一款对程序员有吸引力的产品。请提供产品名称和价格"
|
135 |
+
|
136 |
+
# Construct and run the society
|
137 |
+
society = construct_society(question)
|
138 |
+
answer, chat_history, token_count = run_society(society)
|
139 |
+
|
140 |
+
# Output the result
|
141 |
+
print(f"\033[94mAnswer: {answer}\033[0m")
|
142 |
+
|
143 |
+
|
144 |
+
if __name__ == "__main__":
|
145 |
+
main()
|
owl/script_adapter.py
ADDED
@@ -0,0 +1,263 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
import os
|
15 |
+
import sys
|
16 |
+
import importlib.util
|
17 |
+
import re
|
18 |
+
from pathlib import Path
|
19 |
+
import traceback
|
20 |
+
|
21 |
+
|
22 |
+
def load_module_from_path(module_name, file_path):
|
23 |
+
"""从文件路径加载Python模块"""
|
24 |
+
try:
|
25 |
+
spec = importlib.util.spec_from_file_location(module_name, file_path)
|
26 |
+
if spec is None:
|
27 |
+
print(f"错误: 无法从 {file_path} 创建模块规范")
|
28 |
+
return None
|
29 |
+
|
30 |
+
module = importlib.util.module_from_spec(spec)
|
31 |
+
sys.modules[module_name] = module
|
32 |
+
spec.loader.exec_module(module)
|
33 |
+
return module
|
34 |
+
except Exception as e:
|
35 |
+
print(f"加载模块时出错: {e}")
|
36 |
+
traceback.print_exc()
|
37 |
+
return None
|
38 |
+
|
39 |
+
|
40 |
+
def run_script_with_env_question(script_name):
|
41 |
+
"""使用环境变量中的问题运行脚本"""
|
42 |
+
# 获取环境变量中的问题
|
43 |
+
question = os.environ.get("OWL_QUESTION")
|
44 |
+
if not question:
|
45 |
+
print("错误: 未设置OWL_QUESTION环境变量")
|
46 |
+
sys.exit(1)
|
47 |
+
|
48 |
+
# 脚本路径
|
49 |
+
script_path = Path(script_name).resolve()
|
50 |
+
if not script_path.exists():
|
51 |
+
print(f"错误: 脚本 {script_path} 不存在")
|
52 |
+
sys.exit(1)
|
53 |
+
|
54 |
+
# 创建临时文件路径
|
55 |
+
temp_script_path = script_path.with_name(f"temp_{script_path.name}")
|
56 |
+
|
57 |
+
try:
|
58 |
+
# 读取脚本内容
|
59 |
+
try:
|
60 |
+
with open(script_path, "r", encoding="utf-8") as f:
|
61 |
+
content = f.read()
|
62 |
+
except Exception as e:
|
63 |
+
print(f"读取脚本文件时出错: {e}")
|
64 |
+
sys.exit(1)
|
65 |
+
|
66 |
+
# 检查脚本是否有main函数
|
67 |
+
has_main = re.search(r"def\s+main\s*\(\s*\)\s*:", content) is not None
|
68 |
+
|
69 |
+
# 转义问题中的特殊字符
|
70 |
+
escaped_question = (
|
71 |
+
question.replace("\\", "\\\\").replace('"', '\\"').replace("'", "\\'")
|
72 |
+
)
|
73 |
+
|
74 |
+
# 查找脚本中所有的question赋值 - 改进的正则表达式
|
75 |
+
# 匹配单行和多行字符串赋值
|
76 |
+
question_assignments = re.findall(
|
77 |
+
r'question\s*=\s*(?:["\'].*?["\']|""".*?"""|\'\'\'.*?\'\'\'|\(.*?\))',
|
78 |
+
content,
|
79 |
+
re.DOTALL,
|
80 |
+
)
|
81 |
+
print(f"在脚本中找到 {len(question_assignments)} 个question赋值")
|
82 |
+
|
83 |
+
# 修改脚本内容,替换所有的question赋值
|
84 |
+
modified_content = content
|
85 |
+
|
86 |
+
# 如果脚本中有question赋值,替换所有的赋值
|
87 |
+
if question_assignments:
|
88 |
+
for assignment in question_assignments:
|
89 |
+
modified_content = modified_content.replace(
|
90 |
+
assignment, f'question = "{escaped_question}"'
|
91 |
+
)
|
92 |
+
print(f"已替换脚本中的所有question赋值为: {question}")
|
93 |
+
else:
|
94 |
+
# 如果没有找到question赋值,尝试在main函数前插入
|
95 |
+
if has_main:
|
96 |
+
main_match = re.search(r"def\s+main\s*\(\s*\)\s*:", content)
|
97 |
+
if main_match:
|
98 |
+
insert_pos = main_match.start()
|
99 |
+
modified_content = (
|
100 |
+
content[:insert_pos]
|
101 |
+
+ f'\n# 用户输入的问题\nquestion = "{escaped_question}"\n\n'
|
102 |
+
+ content[insert_pos:]
|
103 |
+
)
|
104 |
+
print(f"已在main函数前插入问题: {question}")
|
105 |
+
else:
|
106 |
+
# 如果没有main函数,在文件开头插入
|
107 |
+
modified_content = (
|
108 |
+
f'# 用户输入的问题\nquestion = "{escaped_question}"\n\n' + content
|
109 |
+
)
|
110 |
+
print(f"已在文件开头插入问题: {question}")
|
111 |
+
|
112 |
+
# 添加monkey patch代码,确保construct_society函数使用用户的问题
|
113 |
+
monkey_patch_code = f"""
|
114 |
+
# 确保construct_society函数使用用户的问题
|
115 |
+
original_construct_society = globals().get('construct_society')
|
116 |
+
if original_construct_society:
|
117 |
+
def patched_construct_society(*args, **kwargs):
|
118 |
+
# 忽略传入的参数,始终使用用户的问题
|
119 |
+
return original_construct_society("{escaped_question}")
|
120 |
+
|
121 |
+
# 替换原始函数
|
122 |
+
globals()['construct_society'] = patched_construct_society
|
123 |
+
print("已修补construct_society函数,确保使用用户问题")
|
124 |
+
"""
|
125 |
+
|
126 |
+
# 在文件末尾添加monkey patch代码
|
127 |
+
modified_content += monkey_patch_code
|
128 |
+
|
129 |
+
# 如果脚本没有调用main函数,添加调用代码
|
130 |
+
if has_main and "__main__" not in content:
|
131 |
+
modified_content += """
|
132 |
+
|
133 |
+
# 确保调用main函数
|
134 |
+
if __name__ == "__main__":
|
135 |
+
main()
|
136 |
+
"""
|
137 |
+
print("已添加main函数调用代码")
|
138 |
+
|
139 |
+
# 如果脚本没有construct_society调用,添加调用代码
|
140 |
+
if (
|
141 |
+
"construct_society" in content
|
142 |
+
and "run_society" in content
|
143 |
+
and "Answer:" not in content
|
144 |
+
):
|
145 |
+
modified_content += f"""
|
146 |
+
|
147 |
+
# 确保执行construct_society和run_society
|
148 |
+
if "construct_society" in globals() and "run_society" in globals():
|
149 |
+
try:
|
150 |
+
society = construct_society("{escaped_question}")
|
151 |
+
from utils import run_society
|
152 |
+
answer, chat_history, token_count = run_society(society)
|
153 |
+
print(f"Answer: {{answer}}")
|
154 |
+
except Exception as e:
|
155 |
+
print(f"运行时出错: {{e}}")
|
156 |
+
import traceback
|
157 |
+
traceback.print_exc()
|
158 |
+
"""
|
159 |
+
print("已添加construct_society和run_society调用代码")
|
160 |
+
|
161 |
+
# 执行修改后的脚本
|
162 |
+
try:
|
163 |
+
# 将脚本目录添加到sys.path
|
164 |
+
script_dir = script_path.parent
|
165 |
+
if str(script_dir) not in sys.path:
|
166 |
+
sys.path.insert(0, str(script_dir))
|
167 |
+
|
168 |
+
# 创建临时文件
|
169 |
+
try:
|
170 |
+
with open(temp_script_path, "w", encoding="utf-8") as f:
|
171 |
+
f.write(modified_content)
|
172 |
+
print(f"已创建临时脚本文件: {temp_script_path}")
|
173 |
+
except Exception as e:
|
174 |
+
print(f"创建临时脚本文件时出错: {e}")
|
175 |
+
sys.exit(1)
|
176 |
+
|
177 |
+
try:
|
178 |
+
# 直接执行临时脚本
|
179 |
+
print("开始执行脚本...")
|
180 |
+
|
181 |
+
# 如果有main函数,加载模块并调用main
|
182 |
+
if has_main:
|
183 |
+
# 加载临时模块
|
184 |
+
module_name = f"temp_{script_path.stem}"
|
185 |
+
module = load_module_from_path(module_name, temp_script_path)
|
186 |
+
|
187 |
+
if module is None:
|
188 |
+
print(f"错误: 无法加载模块 {module_name}")
|
189 |
+
sys.exit(1)
|
190 |
+
|
191 |
+
# 确保模块中有question变量,并且值是用户输入的问题
|
192 |
+
setattr(module, "question", question)
|
193 |
+
|
194 |
+
# 如果模块中有construct_society函数,修补它
|
195 |
+
if hasattr(module, "construct_society"):
|
196 |
+
original_func = module.construct_society
|
197 |
+
|
198 |
+
def patched_func(*args, **kwargs):
|
199 |
+
return original_func(question)
|
200 |
+
|
201 |
+
module.construct_society = patched_func
|
202 |
+
print("已在模块级别修补construct_society函数")
|
203 |
+
|
204 |
+
# 调用main函数
|
205 |
+
if hasattr(module, "main"):
|
206 |
+
print("调用main函数...")
|
207 |
+
module.main()
|
208 |
+
else:
|
209 |
+
print(f"错误: 脚本 {script_path} 中没有main函数")
|
210 |
+
sys.exit(1)
|
211 |
+
else:
|
212 |
+
# 如果没有main函数,直接执行修改后的脚本
|
213 |
+
print("直接执行脚本内容...")
|
214 |
+
# 使用更安全的方式执行脚本
|
215 |
+
with open(temp_script_path, "r", encoding="utf-8") as f:
|
216 |
+
script_code = f.read()
|
217 |
+
|
218 |
+
# 创建一个安全的全局命名空间
|
219 |
+
safe_globals = {
|
220 |
+
"__file__": str(temp_script_path),
|
221 |
+
"__name__": "__main__",
|
222 |
+
}
|
223 |
+
# 添加内置函数
|
224 |
+
safe_globals.update(
|
225 |
+
{k: v for k, v in globals().items() if k in ["__builtins__"]}
|
226 |
+
)
|
227 |
+
|
228 |
+
# 执行脚本
|
229 |
+
exec(script_code, safe_globals)
|
230 |
+
|
231 |
+
except Exception as e:
|
232 |
+
print(f"执行脚本时出错: {e}")
|
233 |
+
traceback.print_exc()
|
234 |
+
sys.exit(1)
|
235 |
+
|
236 |
+
except Exception as e:
|
237 |
+
print(f"处理脚本时出错: {e}")
|
238 |
+
traceback.print_exc()
|
239 |
+
sys.exit(1)
|
240 |
+
|
241 |
+
except Exception as e:
|
242 |
+
print(f"处理脚本时出错: {e}")
|
243 |
+
traceback.print_exc()
|
244 |
+
sys.exit(1)
|
245 |
+
|
246 |
+
finally:
|
247 |
+
# 删除临时文件
|
248 |
+
if temp_script_path.exists():
|
249 |
+
try:
|
250 |
+
temp_script_path.unlink()
|
251 |
+
print(f"已删除临时脚本文件: {temp_script_path}")
|
252 |
+
except Exception as e:
|
253 |
+
print(f"删除临时脚本文件时出错: {e}")
|
254 |
+
|
255 |
+
|
256 |
+
if __name__ == "__main__":
|
257 |
+
# 检查命令行参数
|
258 |
+
if len(sys.argv) < 2:
|
259 |
+
print("用法: python script_adapter.py <script_path>")
|
260 |
+
sys.exit(1)
|
261 |
+
|
262 |
+
# 运行指定的脚本
|
263 |
+
run_script_with_env_question(sys.argv[1])
|
owl/utils/__init__.py
ADDED
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
|
15 |
+
from .common import extract_pattern
|
16 |
+
from .enhanced_role_playing import OwlRolePlaying, OwlGAIARolePlaying, run_society
|
17 |
+
from .gaia import GAIABenchmark
|
18 |
+
from .document_toolkit import DocumentProcessingToolkit
|
19 |
+
|
20 |
+
__all__ = [
|
21 |
+
"extract_pattern",
|
22 |
+
"OwlRolePlaying",
|
23 |
+
"OwlGAIARolePlaying",
|
24 |
+
"run_society",
|
25 |
+
"GAIABenchmark",
|
26 |
+
"DocumentProcessingToolkit",
|
27 |
+
]
|
owl/utils/common.py
ADDED
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
import sys
|
15 |
+
|
16 |
+
sys.path.append("../")
|
17 |
+
|
18 |
+
import re
|
19 |
+
from typing import Optional
|
20 |
+
from camel.logger import get_logger
|
21 |
+
|
22 |
+
logger = get_logger(__name__)
|
23 |
+
|
24 |
+
|
25 |
+
def extract_pattern(content: str, pattern: str) -> Optional[str]:
|
26 |
+
try:
|
27 |
+
_pattern = rf"<{pattern}>(.*?)</{pattern}>"
|
28 |
+
match = re.search(_pattern, content, re.DOTALL)
|
29 |
+
if match:
|
30 |
+
text = match.group(1)
|
31 |
+
return text.strip()
|
32 |
+
else:
|
33 |
+
return None
|
34 |
+
except Exception as e:
|
35 |
+
logger.warning(f"Error extracting answer: {e}, current content: {content}")
|
36 |
+
return None
|
owl/utils/document_toolkit.py
ADDED
@@ -0,0 +1,324 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
|
15 |
+
from camel.toolkits.base import BaseToolkit
|
16 |
+
from camel.toolkits.function_tool import FunctionTool
|
17 |
+
from camel.toolkits import ImageAnalysisToolkit, ExcelToolkit
|
18 |
+
from camel.utils import retry_on_error
|
19 |
+
from camel.logger import get_logger
|
20 |
+
from camel.models import BaseModelBackend
|
21 |
+
from docx2markdown._docx_to_markdown import docx_to_markdown
|
22 |
+
from chunkr_ai import Chunkr
|
23 |
+
import requests
|
24 |
+
import mimetypes
|
25 |
+
import json
|
26 |
+
from typing import List, Optional, Tuple, Literal
|
27 |
+
from urllib.parse import urlparse
|
28 |
+
import os
|
29 |
+
import subprocess
|
30 |
+
import xmltodict
|
31 |
+
import nest_asyncio
|
32 |
+
|
33 |
+
nest_asyncio.apply()
|
34 |
+
|
35 |
+
logger = get_logger(__name__)
|
36 |
+
|
37 |
+
|
38 |
+
class DocumentProcessingToolkit(BaseToolkit):
|
39 |
+
r"""A class representing a toolkit for processing document and return the content of the document.
|
40 |
+
|
41 |
+
This class provides method for processing docx, pdf, pptx, etc. It cannot process excel files.
|
42 |
+
"""
|
43 |
+
|
44 |
+
def __init__(
|
45 |
+
self, cache_dir: Optional[str] = None, model: Optional[BaseModelBackend] = None
|
46 |
+
):
|
47 |
+
self.image_tool = ImageAnalysisToolkit(model=model)
|
48 |
+
# self.audio_tool = AudioAnalysisToolkit()
|
49 |
+
self.excel_tool = ExcelToolkit()
|
50 |
+
|
51 |
+
self.cache_dir = "tmp/"
|
52 |
+
if cache_dir:
|
53 |
+
self.cache_dir = cache_dir
|
54 |
+
|
55 |
+
@retry_on_error()
|
56 |
+
def extract_document_content(self, document_path: str) -> Tuple[bool, str]:
|
57 |
+
r"""Extract the content of a given document (or url) and return the processed text.
|
58 |
+
It may filter out some information, resulting in inaccurate content.
|
59 |
+
|
60 |
+
Args:
|
61 |
+
document_path (str): The path of the document to be processed, either a local path or a URL. It can process image, audio files, zip files and webpages, etc.
|
62 |
+
|
63 |
+
Returns:
|
64 |
+
Tuple[bool, str]: A tuple containing a boolean indicating whether the document was processed successfully, and the content of the document (if success).
|
65 |
+
"""
|
66 |
+
import asyncio
|
67 |
+
|
68 |
+
logger.debug(
|
69 |
+
f"Calling extract_document_content function with document_path=`{document_path}`"
|
70 |
+
)
|
71 |
+
|
72 |
+
if any(document_path.endswith(ext) for ext in [".jpg", ".jpeg", ".png"]):
|
73 |
+
res = self.image_tool.ask_question_about_image(
|
74 |
+
document_path, "Please make a detailed caption about the image."
|
75 |
+
)
|
76 |
+
return True, res
|
77 |
+
|
78 |
+
# if any(document_path.endswith(ext) for ext in ['.mp3', '.wav']):
|
79 |
+
# res = self.audio_tool.ask_question_about_audio(document_path, "Please transcribe the audio content to text.")
|
80 |
+
# return True, res
|
81 |
+
|
82 |
+
if any(document_path.endswith(ext) for ext in ["xls", "xlsx"]):
|
83 |
+
res = self.excel_tool.extract_excel_content(document_path)
|
84 |
+
return True, res
|
85 |
+
|
86 |
+
if any(document_path.endswith(ext) for ext in ["zip"]):
|
87 |
+
extracted_files = self._unzip_file(document_path)
|
88 |
+
return True, f"The extracted files are: {extracted_files}"
|
89 |
+
|
90 |
+
if any(document_path.endswith(ext) for ext in ["json", "jsonl", "jsonld"]):
|
91 |
+
with open(document_path, "r", encoding="utf-8") as f:
|
92 |
+
content = json.load(f)
|
93 |
+
f.close()
|
94 |
+
return True, content
|
95 |
+
|
96 |
+
if any(document_path.endswith(ext) for ext in ["py"]):
|
97 |
+
with open(document_path, "r", encoding="utf-8") as f:
|
98 |
+
content = f.read()
|
99 |
+
f.close()
|
100 |
+
return True, content
|
101 |
+
|
102 |
+
if any(document_path.endswith(ext) for ext in ["xml"]):
|
103 |
+
data = None
|
104 |
+
with open(document_path, "r", encoding="utf-8") as f:
|
105 |
+
content = f.read()
|
106 |
+
f.close()
|
107 |
+
|
108 |
+
try:
|
109 |
+
data = xmltodict.parse(content)
|
110 |
+
logger.debug(f"The extracted xml data is: {data}")
|
111 |
+
return True, data
|
112 |
+
|
113 |
+
except Exception:
|
114 |
+
logger.debug(f"The raw xml data is: {content}")
|
115 |
+
return True, content
|
116 |
+
|
117 |
+
if self._is_webpage(document_path):
|
118 |
+
extracted_text = self._extract_webpage_content(document_path)
|
119 |
+
return True, extracted_text
|
120 |
+
|
121 |
+
else:
|
122 |
+
# judge if url
|
123 |
+
parsed_url = urlparse(document_path)
|
124 |
+
is_url = all([parsed_url.scheme, parsed_url.netloc])
|
125 |
+
if not is_url:
|
126 |
+
if not os.path.exists(document_path):
|
127 |
+
return False, f"Document not found at path: {document_path}."
|
128 |
+
|
129 |
+
# if is docx file, use docx2markdown to convert it
|
130 |
+
if document_path.endswith(".docx"):
|
131 |
+
if is_url:
|
132 |
+
tmp_path = self._download_file(document_path)
|
133 |
+
else:
|
134 |
+
tmp_path = document_path
|
135 |
+
|
136 |
+
file_name = os.path.basename(tmp_path)
|
137 |
+
md_file_path = f"{file_name}.md"
|
138 |
+
docx_to_markdown(tmp_path, md_file_path)
|
139 |
+
|
140 |
+
# load content of md file
|
141 |
+
with open(md_file_path, "r") as f:
|
142 |
+
extracted_text = f.read()
|
143 |
+
f.close()
|
144 |
+
return True, extracted_text
|
145 |
+
try:
|
146 |
+
result = asyncio.run(self._extract_content_with_chunkr(document_path))
|
147 |
+
raise ValueError("Chunkr is not available.")
|
148 |
+
return True, result
|
149 |
+
|
150 |
+
except Exception as e:
|
151 |
+
logger.warning(
|
152 |
+
f"Error occurred while using chunkr to process document: {e}"
|
153 |
+
)
|
154 |
+
if document_path.endswith(".pdf"):
|
155 |
+
# try using pypdf to extract text from pdf
|
156 |
+
try:
|
157 |
+
from PyPDF2 import PdfReader
|
158 |
+
|
159 |
+
if is_url:
|
160 |
+
tmp_path = self._download_file(document_path)
|
161 |
+
document_path = tmp_path
|
162 |
+
|
163 |
+
# Open file in binary mode for PdfReader
|
164 |
+
f = open(document_path, "rb")
|
165 |
+
reader = PdfReader(f)
|
166 |
+
extracted_text = ""
|
167 |
+
for page in reader.pages:
|
168 |
+
extracted_text += page.extract_text()
|
169 |
+
f.close()
|
170 |
+
|
171 |
+
return True, extracted_text
|
172 |
+
|
173 |
+
except Exception as pdf_error:
|
174 |
+
logger.error(
|
175 |
+
f"Error occurred while processing pdf: {pdf_error}"
|
176 |
+
)
|
177 |
+
return (
|
178 |
+
False,
|
179 |
+
f"Error occurred while processing pdf: {pdf_error}",
|
180 |
+
)
|
181 |
+
|
182 |
+
# If we get here, either it's not a PDF or PDF processing failed
|
183 |
+
logger.error(f"Error occurred while processing document: {e}")
|
184 |
+
return False, f"Error occurred while processing document: {e}"
|
185 |
+
|
186 |
+
def _is_webpage(self, url: str) -> bool:
|
187 |
+
r"""Judge whether the given URL is a webpage."""
|
188 |
+
try:
|
189 |
+
parsed_url = urlparse(url)
|
190 |
+
is_url = all([parsed_url.scheme, parsed_url.netloc])
|
191 |
+
if not is_url:
|
192 |
+
return False
|
193 |
+
|
194 |
+
path = parsed_url.path
|
195 |
+
file_type, _ = mimetypes.guess_type(path)
|
196 |
+
if file_type is not None and "text/html" in file_type:
|
197 |
+
return True
|
198 |
+
|
199 |
+
response = requests.head(url, allow_redirects=True, timeout=10)
|
200 |
+
content_type = response.headers.get("Content-Type", "").lower()
|
201 |
+
|
202 |
+
if "text/html" in content_type:
|
203 |
+
return True
|
204 |
+
else:
|
205 |
+
return False
|
206 |
+
|
207 |
+
except requests.exceptions.RequestException as e:
|
208 |
+
# raise RuntimeError(f"Error while checking the URL: {e}")
|
209 |
+
logger.warning(f"Error while checking the URL: {e}")
|
210 |
+
return False
|
211 |
+
|
212 |
+
except TypeError:
|
213 |
+
return True
|
214 |
+
|
215 |
+
@retry_on_error()
|
216 |
+
async def _extract_content_with_chunkr(
|
217 |
+
self,
|
218 |
+
document_path: str,
|
219 |
+
output_format: Literal["json", "markdown"] = "markdown",
|
220 |
+
) -> str:
|
221 |
+
chunkr = Chunkr(api_key=os.getenv("CHUNKR_API_KEY"))
|
222 |
+
|
223 |
+
result = await chunkr.upload(document_path)
|
224 |
+
|
225 |
+
# result = chunkr.upload(document_path)
|
226 |
+
|
227 |
+
if result.status == "Failed":
|
228 |
+
logger.error(
|
229 |
+
f"Error while processing document {document_path}: {result.message}"
|
230 |
+
)
|
231 |
+
return f"Error while processing document: {result.message}"
|
232 |
+
|
233 |
+
# extract document name
|
234 |
+
document_name = os.path.basename(document_path)
|
235 |
+
output_file_path: str
|
236 |
+
|
237 |
+
if output_format == "json":
|
238 |
+
output_file_path = f"{document_name}.json"
|
239 |
+
result.json(output_file_path)
|
240 |
+
|
241 |
+
elif output_format == "markdown":
|
242 |
+
output_file_path = f"{document_name}.md"
|
243 |
+
result.markdown(output_file_path)
|
244 |
+
|
245 |
+
else:
|
246 |
+
return "Invalid output format."
|
247 |
+
|
248 |
+
with open(output_file_path, "r") as f:
|
249 |
+
extracted_text = f.read()
|
250 |
+
f.close()
|
251 |
+
return extracted_text
|
252 |
+
|
253 |
+
@retry_on_error()
|
254 |
+
def _extract_webpage_content(self, url: str) -> str:
|
255 |
+
api_key = os.getenv("FIRECRAWL_API_KEY")
|
256 |
+
from firecrawl import FirecrawlApp
|
257 |
+
|
258 |
+
# Initialize the FirecrawlApp with your API key
|
259 |
+
app = FirecrawlApp(api_key=api_key)
|
260 |
+
|
261 |
+
data = app.crawl_url(
|
262 |
+
url, params={"limit": 1, "scrapeOptions": {"formats": ["markdown"]}}
|
263 |
+
)
|
264 |
+
logger.debug(f"Extractred data from {url}: {data}")
|
265 |
+
if len(data["data"]) == 0:
|
266 |
+
if data["success"]:
|
267 |
+
return "No content found on the webpage."
|
268 |
+
else:
|
269 |
+
return "Error while crawling the webpage."
|
270 |
+
|
271 |
+
return str(data["data"][0]["markdown"])
|
272 |
+
|
273 |
+
def _download_file(self, url: str):
|
274 |
+
r"""Download a file from a URL and save it to the cache directory."""
|
275 |
+
try:
|
276 |
+
response = requests.get(url, stream=True)
|
277 |
+
response.raise_for_status()
|
278 |
+
file_name = url.split("/")[-1]
|
279 |
+
|
280 |
+
file_path = os.path.join(self.cache_dir, file_name)
|
281 |
+
|
282 |
+
with open(file_path, "wb") as file:
|
283 |
+
for chunk in response.iter_content(chunk_size=8192):
|
284 |
+
file.write(chunk)
|
285 |
+
|
286 |
+
return file_path
|
287 |
+
|
288 |
+
except requests.exceptions.RequestException as e:
|
289 |
+
print(f"Error downloading the file: {e}")
|
290 |
+
|
291 |
+
def _get_formatted_time(self) -> str:
|
292 |
+
import time
|
293 |
+
|
294 |
+
return time.strftime("%m%d%H%M")
|
295 |
+
|
296 |
+
def _unzip_file(self, zip_path: str) -> List[str]:
|
297 |
+
if not zip_path.endswith(".zip"):
|
298 |
+
raise ValueError("Only .zip files are supported")
|
299 |
+
|
300 |
+
zip_name = os.path.splitext(os.path.basename(zip_path))[0]
|
301 |
+
extract_path = os.path.join(self.cache_dir, zip_name)
|
302 |
+
os.makedirs(extract_path, exist_ok=True)
|
303 |
+
|
304 |
+
try:
|
305 |
+
subprocess.run(["unzip", "-o", zip_path, "-d", extract_path], check=True)
|
306 |
+
except subprocess.CalledProcessError as e:
|
307 |
+
raise RuntimeError(f"Failed to unzip file: {e}")
|
308 |
+
|
309 |
+
extracted_files = []
|
310 |
+
for root, _, files in os.walk(extract_path):
|
311 |
+
for file in files:
|
312 |
+
extracted_files.append(os.path.join(root, file))
|
313 |
+
|
314 |
+
return extracted_files
|
315 |
+
|
316 |
+
def get_tools(self) -> List[FunctionTool]:
|
317 |
+
r"""Returns a list of FunctionTool objects representing the functions in the toolkit.
|
318 |
+
|
319 |
+
Returns:
|
320 |
+
List[FunctionTool]: A list of FunctionTool objects representing the functions in the toolkit.
|
321 |
+
"""
|
322 |
+
return [
|
323 |
+
FunctionTool(self.extract_document_content),
|
324 |
+
] # Added closing triple quotes here
|
owl/utils/enhanced_role_playing.py
ADDED
@@ -0,0 +1,431 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
|
15 |
+
from typing import Dict, List, Optional, Tuple
|
16 |
+
|
17 |
+
|
18 |
+
from camel.agents import ChatAgent
|
19 |
+
from camel.responses import ChatAgentResponse
|
20 |
+
from camel.messages.base import BaseMessage
|
21 |
+
from camel.societies import RolePlaying
|
22 |
+
from camel.logger import get_logger
|
23 |
+
|
24 |
+
|
25 |
+
from copy import deepcopy
|
26 |
+
|
27 |
+
logger = get_logger(__name__)
|
28 |
+
|
29 |
+
|
30 |
+
class OwlRolePlaying(RolePlaying):
|
31 |
+
def __init__(self, **kwargs):
|
32 |
+
self.user_role_name = kwargs.get("user_role_name", "user")
|
33 |
+
self.assistant_role_name = kwargs.get("assistant_role_name", "assistant")
|
34 |
+
|
35 |
+
self.output_language = kwargs.get("output_language", None)
|
36 |
+
|
37 |
+
self.user_agent_kwargs: dict = kwargs.get("user_agent_kwargs", {})
|
38 |
+
self.assistant_agent_kwargs: dict = kwargs.get("assistant_agent_kwargs", {})
|
39 |
+
|
40 |
+
self.output_language = kwargs.get("output_language", None)
|
41 |
+
|
42 |
+
super().__init__(**kwargs)
|
43 |
+
|
44 |
+
init_user_sys_msg, init_assistant_sys_msg = self._construct_gaia_sys_msgs()
|
45 |
+
|
46 |
+
self.assistant_agent: ChatAgent
|
47 |
+
self.user_agent: ChatAgent
|
48 |
+
self.assistant_sys_msg: Optional[BaseMessage]
|
49 |
+
self.user_sys_msg: Optional[BaseMessage]
|
50 |
+
|
51 |
+
# self.is_reasoning_task = self._judge_if_reasoning_task(self.task_prompt)
|
52 |
+
|
53 |
+
# if self.is_reasoning_task:
|
54 |
+
# logger.info("The task is judged as a reasoning or coding task. The assistant agent will use the reasoning model O3-MINI.")
|
55 |
+
# else:
|
56 |
+
# logger.info("The assistant agent will use the default model.")
|
57 |
+
|
58 |
+
self._init_agents(
|
59 |
+
init_assistant_sys_msg,
|
60 |
+
init_user_sys_msg,
|
61 |
+
assistant_agent_kwargs=self.assistant_agent_kwargs,
|
62 |
+
user_agent_kwargs=self.user_agent_kwargs,
|
63 |
+
output_language=self.output_language,
|
64 |
+
# is_reasoning_task=self.is_reasoning_task
|
65 |
+
)
|
66 |
+
|
67 |
+
def _init_agents(
|
68 |
+
self,
|
69 |
+
init_assistant_sys_msg: BaseMessage,
|
70 |
+
init_user_sys_msg: BaseMessage,
|
71 |
+
assistant_agent_kwargs: Optional[Dict] = None,
|
72 |
+
user_agent_kwargs: Optional[Dict] = None,
|
73 |
+
output_language: Optional[str] = None,
|
74 |
+
is_reasoning_task: bool = False,
|
75 |
+
) -> None:
|
76 |
+
r"""Initialize assistant and user agents with their system messages.
|
77 |
+
|
78 |
+
Args:
|
79 |
+
init_assistant_sys_msg (BaseMessage): Assistant agent's initial
|
80 |
+
system message.
|
81 |
+
init_user_sys_msg (BaseMessage): User agent's initial system
|
82 |
+
message.
|
83 |
+
assistant_agent_kwargs (Dict, optional): Additional arguments to
|
84 |
+
pass to the assistant agent. (default: :obj:`None`)
|
85 |
+
user_agent_kwargs (Dict, optional): Additional arguments to
|
86 |
+
pass to the user agent. (default: :obj:`None`)
|
87 |
+
output_language (str, optional): The language to be output by the
|
88 |
+
agents. (default: :obj:`None`)
|
89 |
+
"""
|
90 |
+
if self.model is not None:
|
91 |
+
if assistant_agent_kwargs is None:
|
92 |
+
assistant_agent_kwargs = {"model": self.model}
|
93 |
+
elif "model" not in assistant_agent_kwargs:
|
94 |
+
assistant_agent_kwargs.update(dict(model=self.model))
|
95 |
+
if user_agent_kwargs is None:
|
96 |
+
user_agent_kwargs = {"model": self.model}
|
97 |
+
elif "model" not in user_agent_kwargs:
|
98 |
+
user_agent_kwargs.update(dict(model=self.model))
|
99 |
+
|
100 |
+
# # If the task is a reasoning task, the assistant agent should use the reasoning model O3-MINI
|
101 |
+
# if is_reasoning_task:
|
102 |
+
# assistant_agent_kwargs['model'] = ModelFactory.create(
|
103 |
+
# model_platform=ModelPlatformType.OPENAI,
|
104 |
+
# model_type=ModelType.O3_MINI,
|
105 |
+
# )
|
106 |
+
|
107 |
+
self.assistant_agent = ChatAgent(
|
108 |
+
init_assistant_sys_msg,
|
109 |
+
output_language=output_language,
|
110 |
+
**(assistant_agent_kwargs or {}),
|
111 |
+
)
|
112 |
+
self.assistant_sys_msg = self.assistant_agent.system_message
|
113 |
+
|
114 |
+
self.user_agent = ChatAgent(
|
115 |
+
init_user_sys_msg,
|
116 |
+
output_language=output_language,
|
117 |
+
**(user_agent_kwargs or {}),
|
118 |
+
)
|
119 |
+
self.user_sys_msg = self.user_agent.system_message
|
120 |
+
|
121 |
+
# def _judge_if_reasoning_task(self, question: str) -> bool:
|
122 |
+
# r"""Judge if the question is a reasoning task."""
|
123 |
+
|
124 |
+
# LLM = OpenAIModel(model_type=ModelType.O3_MINI)
|
125 |
+
# prompt = f"""
|
126 |
+
# Please judge whether the following question is a reasoning or coding task, which can be solved by reasoning without leveraging external resources, or is suitable for writing code to solve the task.
|
127 |
+
# If it is a reasoning or coding task, please return only "yes".
|
128 |
+
# If it is not a reasoning or coding task, please return only "no".
|
129 |
+
# Note:
|
130 |
+
# - If the question required some world knowledge to answer the question, please carefully judge it, because the model's own knowledge is often unreliable.
|
131 |
+
# - If it is suitable for writing codes (e.g. process excel files, write simulation codes, etc.), in most cases, it can be considered as a coding task.
|
132 |
+
# Question: <question>{question}</question>
|
133 |
+
# """
|
134 |
+
# messages = [{"role": "user", "content": prompt}]
|
135 |
+
# resp = LLM.run(messages)
|
136 |
+
# if 'yes' in resp.choices[0].message.content.lower():
|
137 |
+
# return True
|
138 |
+
# else:
|
139 |
+
# return False
|
140 |
+
|
141 |
+
def _construct_gaia_sys_msgs(self):
|
142 |
+
user_system_prompt = f"""
|
143 |
+
===== RULES OF USER =====
|
144 |
+
Never forget you are a user and I am a assistant. Never flip roles! You will always instruct me. We share a common interest in collaborating to successfully complete a task.
|
145 |
+
I must help you to complete a difficult task.
|
146 |
+
You must instruct me based on my expertise and your needs to solve the task step by step. The format of your instruction is: `Instruction: [YOUR INSTRUCTION]`, where "Instruction" describes a sub-task or question.
|
147 |
+
You must give me one instruction at a time.
|
148 |
+
I must write a response that appropriately solves the requested instruction.
|
149 |
+
You should instruct me not ask me questions.
|
150 |
+
|
151 |
+
Please note that the task may be very complicated. Do not attempt to solve the task by single step. You must instruct me to find the answer step by step.
|
152 |
+
Here are some tips that will help you to give more valuable instructions about our task to me:
|
153 |
+
<tips>
|
154 |
+
- I have various tools to use, such as search toolkit, web browser simulation toolkit, document relevant toolkit, code execution toolkit, etc. Thus, You must think how human will solve the task step-by-step, and give me instructions just like that. For example, one may first use google search to get some initial information and the target url, then retrieve the content of the url, or do some web browser interaction to find the answer.
|
155 |
+
- Although the task is complex, the answer does exist. If you can’t find the answer using the current scheme, try to re-plan and use other ways to find the answer, e.g. using other tools or methods that can achieve similar results.
|
156 |
+
- Always remind me to verify my final answer about the overall task. This work can be done by using multiple tools(e.g., screenshots, webpage analysis, etc.), or something else.
|
157 |
+
- If I have written code, please remind me to run the code and get the result.
|
158 |
+
- Search results typically do not provide precise answers. It is not likely to find the answer directly using search toolkit only, the search query should be concise and focuses on finding sources rather than direct answers, as it always need to use other tools to further process the url, e.g. interact with the webpage, extract webpage content, etc.
|
159 |
+
- If the question mentions youtube video, in most cases you have to process the content of the mentioned video.
|
160 |
+
- For downloading files, you can either use the web browser simulation toolkit or write codes (for example, the github content can be downloaded via https://raw.githubusercontent.com/...).
|
161 |
+
- Flexibly write codes to solve some problems, such as excel relevant tasks.
|
162 |
+
</tips>
|
163 |
+
|
164 |
+
Now, here is the overall task: <task>{self.task_prompt}</task>. Never forget our task!
|
165 |
+
|
166 |
+
Now you must start to instruct me to solve the task step-by-step. Do not add anything else other than your instruction!
|
167 |
+
Keep giving me instructions until you think the task is completed.
|
168 |
+
When the task is completed, you must only reply with a single word <TASK_DONE>.
|
169 |
+
Never say <TASK_DONE> unless my responses have solved your task.
|
170 |
+
"""
|
171 |
+
|
172 |
+
assistant_system_prompt = f"""
|
173 |
+
===== RULES OF ASSISTANT =====
|
174 |
+
Never forget you are a assistant and I am a user. Never flip roles! Never instruct me! You have to utilize your available tools to solve the task I assigned.
|
175 |
+
We share a common interest in collaborating to successfully complete a complex task.
|
176 |
+
You must help me to complete the task.
|
177 |
+
|
178 |
+
Here is our overall task: {self.task_prompt}. Never forget our task!
|
179 |
+
|
180 |
+
I must instruct you based on your expertise and my needs to complete the task. An instruction is typically a sub-task or question.
|
181 |
+
|
182 |
+
You must leverage your available tools, try your best to solve the problem, and explain your solutions.
|
183 |
+
Unless I say the task is completed, you should always start with:
|
184 |
+
Solution: [YOUR_SOLUTION]
|
185 |
+
[YOUR_SOLUTION] should be specific, including detailed explanations and provide preferable detailed implementations and examples and lists for task-solving.
|
186 |
+
|
187 |
+
Please note that our overall task may be very complicated. Here are some tips that may help you solve the task:
|
188 |
+
<tips>
|
189 |
+
- If one way fails to provide an answer, try other ways or methods. The answer does exists.
|
190 |
+
- If the search snippet is unhelpful but the URL comes from an authoritative source, try visit the website for more details.
|
191 |
+
- When looking for specific numerical values (e.g., dollar amounts), prioritize reliable sources and avoid relying only on search snippets.
|
192 |
+
- When solving tasks that require web searches, check Wikipedia first before exploring other websites.
|
193 |
+
- When trying to solve math problems, you can try to write python code and use sympy library to solve the problem.
|
194 |
+
- Always verify the accuracy of your final answers! Try cross-checking the answers by other ways. (e.g., screenshots, webpage analysis, etc.).
|
195 |
+
- Do not be overly confident in your own knowledge. Searching can provide a broader perspective and help validate existing knowledge.
|
196 |
+
- After writing codes, do not forget to run the code and get the result. If it encounters an error, try to debug it.
|
197 |
+
- When a tool fails to run, or the code does not run correctly, never assume that it returns the correct result and continue to reason based on the assumption, because the assumed result cannot lead you to the correct answer. The right way is to think about the reason for the error and try again.
|
198 |
+
- Search results typically do not provide precise answers. It is not likely to find the answer directly using search toolkit only, the search query should be concise and focuses on finding sources rather than direct answers, as it always need to use other tools to further process the url, e.g. interact with the webpage, extract webpage content, etc.
|
199 |
+
- For downloading files, you can either use the web browser simulation toolkit or write codes.
|
200 |
+
</tips>
|
201 |
+
|
202 |
+
"""
|
203 |
+
|
204 |
+
user_sys_msg = BaseMessage.make_user_message(
|
205 |
+
role_name=self.user_role_name, content=user_system_prompt
|
206 |
+
)
|
207 |
+
|
208 |
+
assistant_sys_msg = BaseMessage.make_assistant_message(
|
209 |
+
role_name=self.assistant_role_name, content=assistant_system_prompt
|
210 |
+
)
|
211 |
+
|
212 |
+
return user_sys_msg, assistant_sys_msg
|
213 |
+
|
214 |
+
def step(
|
215 |
+
self, assistant_msg: BaseMessage
|
216 |
+
) -> Tuple[ChatAgentResponse, ChatAgentResponse]:
|
217 |
+
user_response = self.user_agent.step(assistant_msg)
|
218 |
+
if user_response.terminated or user_response.msgs is None:
|
219 |
+
return (
|
220 |
+
ChatAgentResponse(msgs=[], terminated=False, info={}),
|
221 |
+
ChatAgentResponse(
|
222 |
+
msgs=[],
|
223 |
+
terminated=user_response.terminated,
|
224 |
+
info=user_response.info,
|
225 |
+
),
|
226 |
+
)
|
227 |
+
user_msg = self._reduce_message_options(user_response.msgs)
|
228 |
+
|
229 |
+
modified_user_msg = deepcopy(user_msg)
|
230 |
+
|
231 |
+
if "TASK_DONE" not in user_msg.content:
|
232 |
+
modified_user_msg.content += f"""\n
|
233 |
+
Here are auxiliary information about the overall task, which may help you understand the intent of the current task:
|
234 |
+
<auxiliary_information>
|
235 |
+
{self.task_prompt}
|
236 |
+
</auxiliary_information>
|
237 |
+
If there are available tools and you want to call them, never say 'I will ...', but first call the tool and reply based on tool call's result, and tell me which tool you have called.
|
238 |
+
"""
|
239 |
+
|
240 |
+
else:
|
241 |
+
# The task is done, and the assistant agent need to give the final answer about the original task
|
242 |
+
modified_user_msg.content += f"""\n
|
243 |
+
Now please make a final answer of the original task based on our conversation : <task>{self.task_prompt}</task>
|
244 |
+
"""
|
245 |
+
|
246 |
+
# process assistant's response
|
247 |
+
assistant_response = self.assistant_agent.step(modified_user_msg)
|
248 |
+
if assistant_response.terminated or assistant_response.msgs is None:
|
249 |
+
return (
|
250 |
+
ChatAgentResponse(
|
251 |
+
msgs=[],
|
252 |
+
terminated=assistant_response.terminated,
|
253 |
+
info=assistant_response.info,
|
254 |
+
),
|
255 |
+
ChatAgentResponse(
|
256 |
+
msgs=[user_msg], terminated=False, info=user_response.info
|
257 |
+
),
|
258 |
+
)
|
259 |
+
assistant_msg = self._reduce_message_options(assistant_response.msgs)
|
260 |
+
|
261 |
+
modified_assistant_msg = deepcopy(assistant_msg)
|
262 |
+
if "TASK_DONE" not in user_msg.content:
|
263 |
+
modified_assistant_msg.content += f"""\n
|
264 |
+
Provide me with the next instruction and input (if needed) based on my response and our current task: <task>{self.task_prompt}</task>
|
265 |
+
Before producing the final answer, please check whether I have rechecked the final answer using different toolkit as much as possible. If not, please remind me to do that.
|
266 |
+
If I have written codes, remind me to run the codes.
|
267 |
+
If you think our task is done, reply with `TASK_DONE` to end our conversation.
|
268 |
+
"""
|
269 |
+
|
270 |
+
# return the modified messages
|
271 |
+
return (
|
272 |
+
ChatAgentResponse(
|
273 |
+
msgs=[modified_assistant_msg],
|
274 |
+
terminated=assistant_response.terminated,
|
275 |
+
info=assistant_response.info,
|
276 |
+
),
|
277 |
+
ChatAgentResponse(
|
278 |
+
msgs=[modified_user_msg],
|
279 |
+
terminated=user_response.terminated,
|
280 |
+
info=user_response.info,
|
281 |
+
),
|
282 |
+
)
|
283 |
+
|
284 |
+
|
285 |
+
class OwlGAIARolePlaying(OwlRolePlaying):
|
286 |
+
def __init__(self, **kwargs):
|
287 |
+
super().__init__(**kwargs)
|
288 |
+
|
289 |
+
def step(
|
290 |
+
self, assistant_msg: BaseMessage
|
291 |
+
) -> Tuple[ChatAgentResponse, ChatAgentResponse]:
|
292 |
+
user_response = self.user_agent.step(assistant_msg)
|
293 |
+
if user_response.terminated or user_response.msgs is None:
|
294 |
+
return (
|
295 |
+
ChatAgentResponse(msgs=[], terminated=False, info={}),
|
296 |
+
ChatAgentResponse(
|
297 |
+
msgs=[],
|
298 |
+
terminated=user_response.terminated,
|
299 |
+
info=user_response.info,
|
300 |
+
),
|
301 |
+
)
|
302 |
+
user_msg = self._reduce_message_options(user_response.msgs)
|
303 |
+
if (
|
304 |
+
"n" in self.user_agent.model_config_dict.keys()
|
305 |
+
and self.user_agent.model_config_dict["n"] > 1
|
306 |
+
):
|
307 |
+
self.user_agent.record_message(user_msg)
|
308 |
+
|
309 |
+
modified_user_msg = deepcopy(user_msg)
|
310 |
+
|
311 |
+
if "TASK_DONE" not in user_msg.content:
|
312 |
+
modified_user_msg.content += f"""\n
|
313 |
+
Here are auxiliary information about the overall task, which may help you understand the intent of the current task:
|
314 |
+
<auxiliary_information>
|
315 |
+
{self.task_prompt}
|
316 |
+
</auxiliary_information>
|
317 |
+
If there are available tools and you want to call them, never say 'I will ...', but first call the tool and reply based on tool call's result, and tell me which tool you have called.
|
318 |
+
"""
|
319 |
+
|
320 |
+
else:
|
321 |
+
# The task is done, and the assistant agent need to give the final answer about the original task
|
322 |
+
modified_user_msg.content += f"""\n
|
323 |
+
Now please make a final answer of the original task based on our conversation : <task>{self.task_prompt}</task>
|
324 |
+
Please pay special attention to the format in which the answer is presented.
|
325 |
+
You should first analyze the answer format required by the question and then output the final answer that meets the format requirements.
|
326 |
+
Your response should include the following content:
|
327 |
+
- `analysis`: enclosed by <analysis> </analysis>, a detailed analysis of the reasoning result.
|
328 |
+
- `final_answer`: enclosed by <final_answer> </final_answer>, the final answer to the question.
|
329 |
+
Here are some hint about the final answer:
|
330 |
+
<hint>
|
331 |
+
Your final answer must be output exactly in the format specified by the question. It should be a number OR as few words as possible OR a comma separated list of numbers and/or strings:
|
332 |
+
- If you are asked for a number, don't use comma to write your number neither use units such as $ or percent sign unless specified otherwise.
|
333 |
+
- If you are asked for a string, don't use articles, neither abbreviations (e.g. for cities), and write the digits in plain text unless specified otherwise.
|
334 |
+
- If you are asked for a comma separated list, apply the above rules depending of whether the element to be put in the list is a number or a string.
|
335 |
+
</hint>
|
336 |
+
"""
|
337 |
+
|
338 |
+
# process assistant's response
|
339 |
+
assistant_response = self.assistant_agent.step(modified_user_msg)
|
340 |
+
if assistant_response.terminated or assistant_response.msgs is None:
|
341 |
+
return (
|
342 |
+
ChatAgentResponse(
|
343 |
+
msgs=[],
|
344 |
+
terminated=assistant_response.terminated,
|
345 |
+
info=assistant_response.info,
|
346 |
+
),
|
347 |
+
ChatAgentResponse(
|
348 |
+
msgs=[user_msg], terminated=False, info=user_response.info
|
349 |
+
),
|
350 |
+
)
|
351 |
+
assistant_msg = self._reduce_message_options(assistant_response.msgs)
|
352 |
+
|
353 |
+
modified_assistant_msg = deepcopy(assistant_msg)
|
354 |
+
if "TASK_DONE" not in user_msg.content:
|
355 |
+
modified_assistant_msg.content += f"""\n
|
356 |
+
Provide me with the next instruction and input (if needed) based on my response and our current task: <task>{self.task_prompt}</task>
|
357 |
+
Before producing the final answer, please check whether I have rechecked the final answer using different toolkit as much as possible. If not, please remind me to do that.
|
358 |
+
If I have written codes, remind me to run the codes.
|
359 |
+
If you think our task is done, reply with `TASK_DONE` to end our conversation.
|
360 |
+
"""
|
361 |
+
|
362 |
+
# return the modified messages
|
363 |
+
return (
|
364 |
+
ChatAgentResponse(
|
365 |
+
msgs=[modified_assistant_msg],
|
366 |
+
terminated=assistant_response.terminated,
|
367 |
+
info=assistant_response.info,
|
368 |
+
),
|
369 |
+
ChatAgentResponse(
|
370 |
+
msgs=[modified_user_msg],
|
371 |
+
terminated=user_response.terminated,
|
372 |
+
info=user_response.info,
|
373 |
+
),
|
374 |
+
)
|
375 |
+
|
376 |
+
|
377 |
+
def run_society(
|
378 |
+
society: RolePlaying, round_limit: int = 15
|
379 |
+
) -> Tuple[str, List[dict], dict]:
|
380 |
+
overall_completion_token_count = 0
|
381 |
+
overall_prompt_token_count = 0
|
382 |
+
|
383 |
+
chat_history = []
|
384 |
+
init_prompt = """
|
385 |
+
Now please give me instructions to solve over overall task step by step. If the task requires some specific knowledge, please instruct me to use tools to complete the task.
|
386 |
+
"""
|
387 |
+
input_msg = society.init_chat(init_prompt)
|
388 |
+
for _round in range(round_limit):
|
389 |
+
assistant_response, user_response = society.step(input_msg)
|
390 |
+
overall_completion_token_count += (
|
391 |
+
assistant_response.info["usage"]["completion_tokens"]
|
392 |
+
+ user_response.info["usage"]["completion_tokens"]
|
393 |
+
)
|
394 |
+
overall_prompt_token_count += (
|
395 |
+
assistant_response.info["usage"]["prompt_tokens"]
|
396 |
+
+ user_response.info["usage"]["prompt_tokens"]
|
397 |
+
)
|
398 |
+
|
399 |
+
# convert tool call to dict
|
400 |
+
tool_call_records: List[dict] = []
|
401 |
+
for tool_call in assistant_response.info["tool_calls"]:
|
402 |
+
tool_call_records.append(tool_call.as_dict())
|
403 |
+
|
404 |
+
_data = {
|
405 |
+
"user": user_response.msg.content,
|
406 |
+
"assistant": assistant_response.msg.content,
|
407 |
+
"tool_calls": tool_call_records,
|
408 |
+
}
|
409 |
+
|
410 |
+
chat_history.append(_data)
|
411 |
+
logger.info(f"Round #{_round} user_response:\n {user_response.msgs[0].content}")
|
412 |
+
logger.info(
|
413 |
+
f"Round #{_round} assistant_response:\n {assistant_response.msgs[0].content}"
|
414 |
+
)
|
415 |
+
|
416 |
+
if (
|
417 |
+
assistant_response.terminated
|
418 |
+
or user_response.terminated
|
419 |
+
or "TASK_DONE" in user_response.msg.content
|
420 |
+
):
|
421 |
+
break
|
422 |
+
|
423 |
+
input_msg = assistant_response.msg
|
424 |
+
|
425 |
+
answer = chat_history[-1]["assistant"]
|
426 |
+
token_info = {
|
427 |
+
"completion_token_count": overall_completion_token_count,
|
428 |
+
"prompt_token_count": overall_prompt_token_count,
|
429 |
+
}
|
430 |
+
|
431 |
+
return answer, chat_history, token_info
|
owl/utils/gaia.py
ADDED
@@ -0,0 +1,412 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
import sys
|
15 |
+
|
16 |
+
sys.path.append("../")
|
17 |
+
|
18 |
+
import json
|
19 |
+
import random
|
20 |
+
import re
|
21 |
+
import string
|
22 |
+
from pathlib import Path
|
23 |
+
from typing import Any, Dict, List, Literal, Optional, Union, Tuple
|
24 |
+
|
25 |
+
from tqdm import tqdm
|
26 |
+
from camel.benchmarks import BaseBenchmark
|
27 |
+
from camel.tasks import Task
|
28 |
+
from camel.logger import get_logger
|
29 |
+
|
30 |
+
from .common import extract_pattern
|
31 |
+
from .enhanced_role_playing import run_society, OwlGAIARolePlaying
|
32 |
+
|
33 |
+
logger = get_logger(__name__)
|
34 |
+
|
35 |
+
|
36 |
+
class GAIABenchmark(BaseBenchmark):
|
37 |
+
r"""GAIA Benchmark adapted from `"GAIA: a benchmark for General AI
|
38 |
+
Assistants"
|
39 |
+
<https://huggingface.co/datasets/gaia-benchmark/GAIA>`_.
|
40 |
+
|
41 |
+
Args:
|
42 |
+
data_dir (str): The directory to save the data.
|
43 |
+
save_to (str): The file to save the results.
|
44 |
+
processes (int, optional): The number of processes to use.
|
45 |
+
(default: :obj:`1`)
|
46 |
+
"""
|
47 |
+
|
48 |
+
def __init__(
|
49 |
+
self,
|
50 |
+
data_dir: str,
|
51 |
+
save_to: str,
|
52 |
+
processes: int = 1,
|
53 |
+
):
|
54 |
+
r"""Initialize the GAIA benchmark.
|
55 |
+
|
56 |
+
Args:
|
57 |
+
data_dir (str): The directory to save the data.
|
58 |
+
save_to (str): The file to save the results.
|
59 |
+
processes (int, optional): The number of processes to use for
|
60 |
+
parallel processing. (default: :obj:`1`)
|
61 |
+
"""
|
62 |
+
super().__init__("gaia", data_dir, save_to, processes)
|
63 |
+
|
64 |
+
def download(self):
|
65 |
+
r"""Download the GAIA dataset."""
|
66 |
+
from huggingface_hub import snapshot_download
|
67 |
+
|
68 |
+
snapshot_download(
|
69 |
+
repo_id="gaia-benchmark/GAIA",
|
70 |
+
repo_type="dataset",
|
71 |
+
local_dir=self.data_dir,
|
72 |
+
local_dir_use_symlinks=True,
|
73 |
+
)
|
74 |
+
|
75 |
+
def _check_task_completed(self, task_id: str) -> bool:
|
76 |
+
for data in self._results:
|
77 |
+
if data["task_id"] == task_id:
|
78 |
+
return True
|
79 |
+
return False
|
80 |
+
|
81 |
+
def dump_tasks(self, save_path: str, datas):
|
82 |
+
constructed_data = []
|
83 |
+
for idx, data in enumerate(datas):
|
84 |
+
tmp_dict = {
|
85 |
+
"idx": idx,
|
86 |
+
"task_id": data["task_id"],
|
87 |
+
"Question": data["Question"],
|
88 |
+
"Level": data["Level"],
|
89 |
+
"Final answer": data["Final answer"],
|
90 |
+
"Annotation Metadata": data["Annotator Metadata"],
|
91 |
+
}
|
92 |
+
|
93 |
+
constructed_data.append(tmp_dict)
|
94 |
+
with open(save_path, "w", encoding="utf-8") as f:
|
95 |
+
json.dump(constructed_data, f, indent=4)
|
96 |
+
f.close()
|
97 |
+
|
98 |
+
print(f"Successfully dumped tasks to {save_path}")
|
99 |
+
|
100 |
+
def load(self, force_download=False):
|
101 |
+
r"""Load the GAIA dataset.
|
102 |
+
|
103 |
+
Args:
|
104 |
+
force_download (bool, optional): Whether to
|
105 |
+
force download the data.
|
106 |
+
"""
|
107 |
+
if force_download:
|
108 |
+
logger.info("Force downloading data.")
|
109 |
+
self.download()
|
110 |
+
|
111 |
+
# Define validation and test directories
|
112 |
+
valid_dir = self.data_dir / "2023/validation"
|
113 |
+
test_dir = self.data_dir / "2023/test"
|
114 |
+
|
115 |
+
# Check if directories exist; if not, download the data
|
116 |
+
if not valid_dir.is_dir() or not test_dir.is_dir():
|
117 |
+
logger.info("Data not found. Downloading data.")
|
118 |
+
self.download()
|
119 |
+
|
120 |
+
# Load metadata for both validation and test datasets
|
121 |
+
for path, label in zip([valid_dir, test_dir], ["valid", "test"]):
|
122 |
+
self._data[label] = []
|
123 |
+
with open(path / "metadata.jsonl", "r") as f:
|
124 |
+
lines = f.readlines()
|
125 |
+
for line in lines:
|
126 |
+
data = json.loads(line)
|
127 |
+
if data["task_id"] == "0-0-0-0-0":
|
128 |
+
continue
|
129 |
+
if data["file_name"]:
|
130 |
+
data["file_name"] = path / data["file_name"]
|
131 |
+
self._data[label].append(data)
|
132 |
+
return self
|
133 |
+
|
134 |
+
@property
|
135 |
+
def train(self):
|
136 |
+
r"""Get the training set."""
|
137 |
+
raise NotImplementedError("GAIA does not have a training set.")
|
138 |
+
|
139 |
+
def run(
|
140 |
+
self,
|
141 |
+
user_role_name: str,
|
142 |
+
assistant_role_name: str,
|
143 |
+
user_agent_kwargs: dict,
|
144 |
+
assistant_agent_kwargs: dict,
|
145 |
+
on: Literal["train", "valid", "test"],
|
146 |
+
level: Union[int, List[int], Literal["all"]],
|
147 |
+
randomize: bool = False,
|
148 |
+
subset: Optional[int] = None,
|
149 |
+
idx: Optional[List[int]] = None,
|
150 |
+
save_result: bool = False,
|
151 |
+
) -> Dict[str, Any]:
|
152 |
+
# Validate inputs
|
153 |
+
if on not in ["valid", "test"]:
|
154 |
+
raise ValueError(
|
155 |
+
f"Invalid value for `on`: {on}, expected 'valid' or 'test'."
|
156 |
+
)
|
157 |
+
|
158 |
+
levels = (
|
159 |
+
[1, 2, 3]
|
160 |
+
if level == "all"
|
161 |
+
else [level]
|
162 |
+
if isinstance(level, int)
|
163 |
+
else level
|
164 |
+
)
|
165 |
+
if not all(isinstance(level, int) and level in [1, 2, 3] for level in levels):
|
166 |
+
raise ValueError(
|
167 |
+
f"Invalid value for `level`: {level}, expected 1, 2, 3 " "or 'all'."
|
168 |
+
)
|
169 |
+
logger.info(f"Running benchmark on {on} set at levels {levels}.")
|
170 |
+
datas = [data for data in self._data[on] if data["Level"] in levels]
|
171 |
+
# Shuffle and subset data if necessary
|
172 |
+
if randomize:
|
173 |
+
random.shuffle(datas)
|
174 |
+
if subset:
|
175 |
+
datas = datas[:subset]
|
176 |
+
|
177 |
+
if idx is not None:
|
178 |
+
# pick only the tasks with the specified idx
|
179 |
+
if len(idx) != 0:
|
180 |
+
datas = [datas[i] for i in idx]
|
181 |
+
|
182 |
+
logger.info(f"Number of tasks: {len(datas)}")
|
183 |
+
|
184 |
+
self._results = []
|
185 |
+
|
186 |
+
if save_result:
|
187 |
+
try:
|
188 |
+
with open(self.save_to, "r", encoding="utf-8") as f:
|
189 |
+
self._results = json.load(f)
|
190 |
+
f.close()
|
191 |
+
except Exception as e:
|
192 |
+
logger.warning(e)
|
193 |
+
# raise FileNotFoundError(f"{self.save_to} does not exist.")
|
194 |
+
|
195 |
+
# Process tasks
|
196 |
+
for task in tqdm(datas, desc="Running"):
|
197 |
+
if self._check_task_completed(task["task_id"]):
|
198 |
+
logger.success(
|
199 |
+
f"The following task is already completed:\n task id: {task['task_id']}, question: {task['Question']}"
|
200 |
+
)
|
201 |
+
continue
|
202 |
+
|
203 |
+
if_prepared_task, info = self._prepare_task(task)
|
204 |
+
if not if_prepared_task:
|
205 |
+
_result_info = {
|
206 |
+
"task_id": task["task_id"],
|
207 |
+
"question": task["Question"],
|
208 |
+
"level": task["Level"],
|
209 |
+
"model_answer": None,
|
210 |
+
"ground_truth": None,
|
211 |
+
"score": 0,
|
212 |
+
"history": None,
|
213 |
+
}
|
214 |
+
self._results.append(_result_info)
|
215 |
+
continue
|
216 |
+
try:
|
217 |
+
logger.info(f"Task Question: {task['Question']}")
|
218 |
+
logger.info(f"Required tools: {task['Annotator Metadata']['Tools']}")
|
219 |
+
|
220 |
+
task_kwargs = {
|
221 |
+
"task_prompt": task["Question"],
|
222 |
+
"with_task_specify": False,
|
223 |
+
}
|
224 |
+
|
225 |
+
society = OwlGAIARolePlaying(
|
226 |
+
**task_kwargs,
|
227 |
+
user_role_name=user_role_name,
|
228 |
+
user_agent_kwargs=user_agent_kwargs,
|
229 |
+
assistant_role_name=assistant_role_name,
|
230 |
+
assistant_agent_kwargs=assistant_agent_kwargs,
|
231 |
+
)
|
232 |
+
|
233 |
+
raw_answer, chat_history, token_info = run_society(society)
|
234 |
+
try:
|
235 |
+
answer = extract_pattern(raw_answer, "final_answer")
|
236 |
+
except Exception as e:
|
237 |
+
logger.error(
|
238 |
+
f"Error in extracting final answer from text {raw_answer}: {e}"
|
239 |
+
)
|
240 |
+
answer = None
|
241 |
+
|
242 |
+
logger.info(
|
243 |
+
f"Model answer: {answer}, Ground truth: {task['Final answer']}"
|
244 |
+
)
|
245 |
+
|
246 |
+
_result_info = {
|
247 |
+
"task_id": task["task_id"],
|
248 |
+
"question": task["Question"]
|
249 |
+
+ "Please decompose the task into several sub-tasks and find the answer step-by-step.",
|
250 |
+
"level": task["Level"],
|
251 |
+
"model_answer": answer,
|
252 |
+
"ground_truth": task["Final answer"],
|
253 |
+
"score": self.question_scorer(answer, task["Final answer"]),
|
254 |
+
"token_info": token_info,
|
255 |
+
"history": chat_history,
|
256 |
+
}
|
257 |
+
self._results.append(_result_info)
|
258 |
+
|
259 |
+
except Exception as e:
|
260 |
+
logger.error(f"Error in processing task: {e}")
|
261 |
+
|
262 |
+
if save_result:
|
263 |
+
with open(self.save_to, "w") as f:
|
264 |
+
json.dump(self._results, f, indent=4, ensure_ascii=False)
|
265 |
+
f.close()
|
266 |
+
|
267 |
+
return self._generate_summary()
|
268 |
+
|
269 |
+
def _prepare_task(self, task: Dict[str, Any]) -> Tuple[bool, str]:
|
270 |
+
r"""Prepare the task by validating and enriching its data."""
|
271 |
+
if task["file_name"]:
|
272 |
+
if isinstance(task["file_name"], Path):
|
273 |
+
task["file_name"] = str(task["file_name"])
|
274 |
+
|
275 |
+
file_path = Path(task["file_name"])
|
276 |
+
if not file_path.exists():
|
277 |
+
logger.info(f"Skipping task because file not found: {file_path}")
|
278 |
+
return False, f"Skipping task because file not found: {file_path}"
|
279 |
+
if file_path.suffix in [".pdf", ".docx", ".doc", ".txt"]:
|
280 |
+
task["Question"] += (
|
281 |
+
f" Here are the necessary document files: {file_path}"
|
282 |
+
)
|
283 |
+
|
284 |
+
elif file_path.suffix in [".jpg", ".jpeg", ".png"]:
|
285 |
+
task["Question"] += f" Here are the necessary image files: {file_path}"
|
286 |
+
|
287 |
+
elif file_path.suffix in [".xlsx", "xls", ".csv"]:
|
288 |
+
task["Question"] += (
|
289 |
+
f" Here are the necessary table files: {file_path}, for processing excel file, you can write python code and leverage excel toolkit to process the file step-by-step and get the information."
|
290 |
+
)
|
291 |
+
|
292 |
+
elif file_path.suffix in [".py"]:
|
293 |
+
task["Question"] += f" Here are the necessary python files: {file_path}"
|
294 |
+
|
295 |
+
else:
|
296 |
+
task["Question"] += f" Here are the necessary files: {file_path}"
|
297 |
+
|
298 |
+
return True, None
|
299 |
+
|
300 |
+
def _create_task(self, task: Dict[str, Any]) -> Task:
|
301 |
+
r"""Create a user message from a task.
|
302 |
+
|
303 |
+
Args:
|
304 |
+
task (Dict[str, Any]): The task to create the message from.
|
305 |
+
|
306 |
+
Returns:
|
307 |
+
Task: The task created from the input.
|
308 |
+
"""
|
309 |
+
return Task(id=str(task["task_id"]), content=task["Question"])
|
310 |
+
|
311 |
+
def _generate_summary(self) -> Dict[str, Any]:
|
312 |
+
r"""Generate and return a summary of the benchmark results."""
|
313 |
+
correct = sum(result["score"] for result in self._results)
|
314 |
+
return {
|
315 |
+
"total": len(self._results),
|
316 |
+
"correct": correct,
|
317 |
+
"results": self._results,
|
318 |
+
"accuracy": correct / len(self._results) if len(self._results) > 0 else 0,
|
319 |
+
}
|
320 |
+
|
321 |
+
def question_scorer(self, model_answer: str, ground_truth: str) -> bool:
|
322 |
+
r"""Scorer for the GAIA benchmark.
|
323 |
+
https://huggingface.co/spaces/gaia-benchmark/leaderboard/blob/main/
|
324 |
+
scorer.py
|
325 |
+
|
326 |
+
Args:
|
327 |
+
model_answer (str): The model answer.
|
328 |
+
ground_truth (str): The ground truth answer.
|
329 |
+
|
330 |
+
Returns:
|
331 |
+
bool: The score of the model
|
332 |
+
"""
|
333 |
+
|
334 |
+
def is_float(element: Any) -> bool:
|
335 |
+
try:
|
336 |
+
float(element)
|
337 |
+
return True
|
338 |
+
except ValueError:
|
339 |
+
return False
|
340 |
+
|
341 |
+
if is_float(ground_truth):
|
342 |
+
logger.info(f"Evaluating {model_answer} as a number.")
|
343 |
+
normalized_answer = self.normalize_number_str(model_answer)
|
344 |
+
return normalized_answer == float(ground_truth)
|
345 |
+
|
346 |
+
elif any(char in ground_truth for char in [",", ";"]):
|
347 |
+
logger.info(f"Evaluating {model_answer} as a comma separated list.")
|
348 |
+
gt_elems = self.split_string(ground_truth)
|
349 |
+
ma_elems = self.split_string(model_answer)
|
350 |
+
|
351 |
+
if len(gt_elems) != len(ma_elems):
|
352 |
+
logger.warning(
|
353 |
+
"Answer lists have different lengths, returning False.",
|
354 |
+
UserWarning,
|
355 |
+
)
|
356 |
+
return False
|
357 |
+
|
358 |
+
comparisons = []
|
359 |
+
for ma_elem, gt_elem in zip(ma_elems, gt_elems):
|
360 |
+
if is_float(gt_elem):
|
361 |
+
normalized_ma_elem = self.normalize_number_str(ma_elem)
|
362 |
+
comparisons.append(normalized_ma_elem == float(gt_elem))
|
363 |
+
else:
|
364 |
+
ma_elem = self.normalize_str(ma_elem, remove_punct=False)
|
365 |
+
gt_elem = self.normalize_str(gt_elem, remove_punct=False)
|
366 |
+
comparisons.append(ma_elem == gt_elem)
|
367 |
+
return all(comparisons)
|
368 |
+
else:
|
369 |
+
logger.info(f"Evaluating {model_answer} as a string.")
|
370 |
+
ma_elem = self.normalize_str(model_answer)
|
371 |
+
gt_elem = self.normalize_str(ground_truth)
|
372 |
+
return ma_elem == gt_elem
|
373 |
+
|
374 |
+
def normalize_number_str(self, number_str: str) -> float:
|
375 |
+
for char in ["$", "%", ","]:
|
376 |
+
number_str = number_str.replace(char, "")
|
377 |
+
try:
|
378 |
+
return float(number_str)
|
379 |
+
except ValueError:
|
380 |
+
logger.error(f"String {number_str} cannot be normalized to number str.")
|
381 |
+
return float("inf")
|
382 |
+
|
383 |
+
def split_string(self, s: str, char_list: Optional[List[str]] = None) -> list[str]:
|
384 |
+
r"""Split a string based on a list of characters.
|
385 |
+
|
386 |
+
Args:
|
387 |
+
s (str): The string to split.
|
388 |
+
char_list (Optional[List[str]], optional): T
|
389 |
+
he list of characters to split on.
|
390 |
+
(default: :obj:`None`)
|
391 |
+
"""
|
392 |
+
if char_list is None:
|
393 |
+
char_list = [",", ";"]
|
394 |
+
pattern = f"[{''.join(char_list)}]"
|
395 |
+
return re.split(pattern, s)
|
396 |
+
|
397 |
+
def normalize_str(self, input_str, remove_punct=True) -> str:
|
398 |
+
r"""Normalize a string.
|
399 |
+
|
400 |
+
Args:
|
401 |
+
input_str: The input string to normalize.
|
402 |
+
remove_punct: Whether to remove punctuation.
|
403 |
+
|
404 |
+
Returns:
|
405 |
+
str: The normalized string.
|
406 |
+
"""
|
407 |
+
no_spaces = re.sub(r"\s", "", input_str)
|
408 |
+
if remove_punct:
|
409 |
+
translator = str.maketrans("", "", string.punctuation)
|
410 |
+
return no_spaces.lower().translate(translator)
|
411 |
+
else:
|
412 |
+
return no_spaces.lower()
|
pyproject.toml
ADDED
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
[build-system]
|
2 |
+
requires = ["hatchling"]
|
3 |
+
build-backend = "hatchling.build"
|
4 |
+
|
5 |
+
[project]
|
6 |
+
name = "owl"
|
7 |
+
version = "0.0.1"
|
8 |
+
description = "Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation"
|
9 |
+
authors = [{ name = "CAMEL-AI.org" }]
|
10 |
+
requires-python = ">=3.10,<3.13"
|
11 |
+
readme = "README.md"
|
12 |
+
license = "Apache-2.0"
|
13 |
+
keywords = [
|
14 |
+
"optimized-workforce-learning",
|
15 |
+
"multi-agent-assistance",
|
16 |
+
"task-automation",
|
17 |
+
"real-world-tasks",
|
18 |
+
"artificial-intelligence",
|
19 |
+
"agent-collaboration",
|
20 |
+
"workforce-optimization",
|
21 |
+
"learning-systems"
|
22 |
+
]
|
23 |
+
dependencies = [
|
24 |
+
"camel-ai[all]==0.2.23",
|
25 |
+
"chunkr-ai>=0.0.41",
|
26 |
+
"docx2markdown>=0.1.1",
|
27 |
+
"gradio>=3.50.2",
|
28 |
+
]
|
29 |
+
|
30 |
+
[project.urls]
|
31 |
+
Homepage = "https://www.camel-ai.org/"
|
32 |
+
Repository = "https://github.com/camel-ai/owl"
|
33 |
+
Documentation = "https://docs.camel-ai.org"
|
34 |
+
|
35 |
+
[tool.hatch.build.targets.wheel]
|
36 |
+
packages = ["owl"]
|
37 |
+
|
38 |
+
[tool.mypy]
|
39 |
+
python_version = "3.11"
|
40 |
+
warn_return_any = false
|
41 |
+
warn_unused_configs = true
|
42 |
+
disallow_untyped_defs = false
|
43 |
+
disallow_incomplete_defs = false
|
44 |
+
check_untyped_defs = false
|
45 |
+
disallow_untyped_decorators = false
|
46 |
+
no_implicit_optional = false
|
47 |
+
strict_optional = false
|
48 |
+
ignore_missing_imports = true
|
49 |
+
allow_redefinition = true
|
50 |
+
disable_error_code = ["assignment", "arg-type", "return-value"]
|
51 |
+
|
52 |
+
[[tool.mypy.overrides]]
|
53 |
+
module = "camel.*"
|
54 |
+
ignore_missing_imports = true
|
55 |
+
follow_imports = "skip"
|
56 |
+
|
57 |
+
[[tool.mypy.overrides]]
|
58 |
+
module = "utils"
|
59 |
+
ignore_missing_imports = true
|
requirements.txt
CHANGED
@@ -1 +1,4 @@
|
|
1 |
-
|
|
|
|
|
|
|
|
1 |
+
camel-ai[all]==0.2.23
|
2 |
+
chunkr-ai>=0.0.41
|
3 |
+
docx2markdown>=0.1.1
|
4 |
+
gradio>=3.50.2
|
run_app.py
ADDED
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
#!/usr/bin/env python
|
15 |
+
# -*- coding: utf-8 -*-
|
16 |
+
|
17 |
+
"""
|
18 |
+
OWL 智能助手运行平台启动脚本
|
19 |
+
"""
|
20 |
+
|
21 |
+
import os
|
22 |
+
import sys
|
23 |
+
from pathlib import Path
|
24 |
+
|
25 |
+
|
26 |
+
def main():
|
27 |
+
"""主函数,启动OWL智能助手运行平台"""
|
28 |
+
# 确保当前目录是项目根目录
|
29 |
+
project_root = Path(__file__).resolve().parent
|
30 |
+
os.chdir(project_root)
|
31 |
+
|
32 |
+
# 创建日志目录
|
33 |
+
log_dir = project_root / "logs"
|
34 |
+
log_dir.mkdir(exist_ok=True)
|
35 |
+
|
36 |
+
# 导入并运行应用
|
37 |
+
sys.path.insert(0, str(project_root))
|
38 |
+
|
39 |
+
try:
|
40 |
+
from owl.app import create_ui
|
41 |
+
|
42 |
+
# 创建并启动应用
|
43 |
+
app = create_ui()
|
44 |
+
app.queue().launch(share=True)
|
45 |
+
|
46 |
+
except ImportError as e:
|
47 |
+
print(f"错误: 无法导入必要的模块。请确保已安装所有依赖项: {e}")
|
48 |
+
print("提示: 运行 'pip install -r requirements.txt' 安装所有依赖项")
|
49 |
+
sys.exit(1)
|
50 |
+
except Exception as e:
|
51 |
+
print(f"启动应用程序时出错: {e}")
|
52 |
+
import traceback
|
53 |
+
|
54 |
+
traceback.print_exc()
|
55 |
+
sys.exit(1)
|
56 |
+
|
57 |
+
|
58 |
+
if __name__ == "__main__":
|
59 |
+
main()
|
60 |
+
|
run_app.py.bak
ADDED
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
2 |
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
3 |
+
# you may not use this file except in compliance with the License.
|
4 |
+
# You may obtain a copy of the License at
|
5 |
+
#
|
6 |
+
# http://www.apache.org/licenses/LICENSE-2.0
|
7 |
+
#
|
8 |
+
# Unless required by applicable law or agreed to in writing, software
|
9 |
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
10 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
11 |
+
# See the License for the specific language governing permissions and
|
12 |
+
# limitations under the License.
|
13 |
+
# ========= Copyright 2023-2024 @ CAMEL-AI.org. All Rights Reserved. =========
|
14 |
+
#!/usr/bin/env python
|
15 |
+
# -*- coding: utf-8 -*-
|
16 |
+
|
17 |
+
"""
|
18 |
+
OWL 智能助手运行平台启动脚本
|
19 |
+
"""
|
20 |
+
|
21 |
+
import os
|
22 |
+
import sys
|
23 |
+
from pathlib import Path
|
24 |
+
|
25 |
+
|
26 |
+
def main():
|
27 |
+
"""主函数,启动OWL智能助手运行平台"""
|
28 |
+
# 确保当前目录是项目根目录
|
29 |
+
project_root = Path(__file__).resolve().parent
|
30 |
+
os.chdir(project_root)
|
31 |
+
|
32 |
+
# 创建日志目录
|
33 |
+
log_dir = project_root / "logs"
|
34 |
+
log_dir.mkdir(exist_ok=True)
|
35 |
+
|
36 |
+
# 导入并运行应用
|
37 |
+
sys.path.insert(0, str(project_root))
|
38 |
+
|
39 |
+
try:
|
40 |
+
from owl.app import create_ui
|
41 |
+
|
42 |
+
# 创建并启动应用
|
43 |
+
app = create_ui()
|
44 |
+
app.queue().launch(share=True)
|
45 |
+
|
46 |
+
except ImportError as e:
|
47 |
+
print(f"错误: 无法导入必要的模块。请确保已安装所有依赖项: {e}")
|
48 |
+
print("提示: 运行 'pip install -r requirements.txt' 安装所有依赖项")
|
49 |
+
sys.exit(1)
|
50 |
+
except Exception as e:
|
51 |
+
print(f"启动应用程序时出错: {e}")
|
52 |
+
import traceback
|
53 |
+
|
54 |
+
traceback.print_exc()
|
55 |
+
sys.exit(1)
|
56 |
+
|
57 |
+
|
58 |
+
if __name__ == "__main__":
|
59 |
+
main()
|
uv.lock
ADDED
The diff for this file is too large to render.
See raw diff
|
|