gpt-omni commited on
Commit
d030d66
Β·
1 Parent(s): 468db83

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -8
README.md CHANGED
@@ -1,20 +1,21 @@
1
 
2
  # Mini-Omni
3
 
4
- <p align="center">
5
  Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming
6
- <p>
 
7
 
8
  <p align="center">
9
  πŸ€— <a href="">Hugging Face</a> | πŸ“– <a href="https://github.com/gpt-omni/mini-omni">Github</a>
10
- | πŸ“‘ <a href="">Technical report (coming soon)</a>
11
- <p>
12
 
13
  Mini-Omni is an open-source multimodel large language model that can **hear, talk while thinking**. Featuring real-time end-to-end speech input and **streaming audio output** conversational capabilities.
14
 
15
  <p align="center">
16
  <img src="data/figures/frameworkv3.jpg" width="100%"/>
17
- <p>
18
 
19
 
20
  ## Features
@@ -29,7 +30,10 @@ Mini-Omni is an open-source multimodel large language model that can **hear, tal
29
 
30
  ## Demo
31
 
32
- ![](./data/demo_streamlit.mov)
 
 
 
33
 
34
  ## Install
35
 
@@ -71,7 +75,10 @@ API_URL=http://0.0.0.0:60808/chat python3 webui/omni_gradio.py
71
 
72
  example:
73
 
74
- ![](./data/demo_gradio.mov)
 
 
 
75
 
76
  **Local test**
77
 
@@ -89,4 +96,4 @@ python inference.py
89
  - [whisper](https://github.com/openai/whisper/) for audio encoding.
90
  - [snac](https://github.com/hubertsiuzdak/snac/) for audio decoding.
91
  - [CosyVoice](https://github.com/FunAudioLLM/CosyVoice) for generating synthetic speech.
92
- - [OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca) and [MOSS](https://github.com/OpenMOSS/MOSS/tree/main) for alignment.
 
1
 
2
  # Mini-Omni
3
 
4
+ <p align="center"><strong style="font-size: 18px;">
5
  Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming
6
+ </strong>
7
+ </p>
8
 
9
  <p align="center">
10
  πŸ€— <a href="">Hugging Face</a> | πŸ“– <a href="https://github.com/gpt-omni/mini-omni">Github</a>
11
+ | πŸ“‘ <a href="https://arxiv.org/abs/2408.16725">Technical report</a>
12
+ </p>
13
 
14
  Mini-Omni is an open-source multimodel large language model that can **hear, talk while thinking**. Featuring real-time end-to-end speech input and **streaming audio output** conversational capabilities.
15
 
16
  <p align="center">
17
  <img src="data/figures/frameworkv3.jpg" width="100%"/>
18
+ </p>
19
 
20
 
21
  ## Features
 
30
 
31
  ## Demo
32
 
33
+ NOTE: need to unmute first.
34
+
35
+ https://github.com/user-attachments/assets/03bdde05-9514-4748-b527-003bea57f118
36
+
37
 
38
  ## Install
39
 
 
75
 
76
  example:
77
 
78
+ NOTE: need to unmute first. Gradio seems can not play audio stream instantly, so the latency feels a bit longer.
79
+
80
+ https://github.com/user-attachments/assets/29187680-4c42-47ff-b352-f0ea333496d9
81
+
82
 
83
  **Local test**
84
 
 
96
  - [whisper](https://github.com/openai/whisper/) for audio encoding.
97
  - [snac](https://github.com/hubertsiuzdak/snac/) for audio decoding.
98
  - [CosyVoice](https://github.com/FunAudioLLM/CosyVoice) for generating synthetic speech.
99
+ - [OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca) and [MOSS](https://github.com/OpenMOSS/MOSS/tree/main) for alignment.