blackteay/GPT-SoVITS
1
0
Fork 0
mirror of https://github.com/RVC-Boss/GPT-SoVITS.git synced 2026-07-14 12:13:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs: document FunASR backend options (#2803)

Browse Source 
Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
This commit is contained in:
zhifu gao 2026-07-13 10:41:22 +08:00 committed by GitHub
parent 551918539c
commit be6a4f1e9d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 16 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
README.md
View File

@ -35,7 +35,7 @@ A Powerful Few-shot Voice Conversion and Text-to-Speech WebUI.<br><br>
3. **Cross-lingual Support:** Inference in languages different from the training dataset, currently supporting English, Japanese, Korean, Cantonese and Chinese.
4. **WebUI Tools:** Integrated tools include voice accompaniment separation, automatic training set segmentation, Chinese ASR, and text labeling, assisting beginners in creating training datasets and GPT/SoVITS models.
4. **WebUI Tools:** Integrated tools include voice accompaniment separation, automatic training set segmentation, multilingual ASR with [Fun-ASR-Nano](https://github.com/FunAudioLLM/Fun-ASR), [SenseVoice](https://github.com/FunAudioLLM/SenseVoice), and classic [FunASR](https://github.com/modelscope/FunASR), plus text labeling, assisting beginners in creating training datasets and GPT/SoVITS models.
**Check out our [demo video](https://www.bilibili.com/video/BV12g4y1m7Uw) here!**
@ -208,7 +208,7 @@ docker exec -it <GPT-SoVITS-CU126-Lite|GPT-SoVITS-CU128-Lite|GPT-SoVITS-CU126|GP
- The suggestion is to **directly specify the model type** in the model name and configuration file name, such as `mel_mand_roformer`, `bs_roformer`. If not specified, the features will be compared from the configuration file to determine which type of model it is. For example, the model `bs_roformer_ep_368_sdr_12.9628.ckpt` and its corresponding configuration file `bs_roformer_ep_368_sdr_12.9628.yaml` are a pair, `kim_mel_band_roformer.ckpt` and `kim_mel_band_roformer.yaml` are also a pair.
4. For Chinese ASR (additionally), download models from [Damo ASR Model](https://modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/files), [Damo VAD Model](https://modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch/files), and [Damo Punc Model](https://modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vocab272727-pytorch/files) and place them in `tools/asr/models`.
4. FunASR models are downloaded automatically on first use. The WebUI offers [Fun-ASR-Nano](https://github.com/FunAudioLLM/Fun-ASR) for multilingual and dialect ASR, [SenseVoice](https://github.com/FunAudioLLM/SenseVoice) for fast transcription, and classic Paraformer/UniASR through [FunASR](https://github.com/modelscope/FunASR) for Chinese and Cantonese. To preinstall the classic Chinese models for offline use, download the [ASR model](https://modelscope.cn/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/files), [VAD model](https://modelscope.cn/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch/files), and [punctuation model](https://modelscope.cn/models/iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch/files) into `tools/asr/models`.
5. For English or Japanese ASR (additionally), download models from [Faster Whisper Large V3](https://huggingface.co/Systran/faster-whisper-large-v3) and place them in `tools/asr/models`. Also, [other models](https://huggingface.co/Systran) may have the similar effect with smaller disk footprint.
@ -413,13 +413,13 @@ python audio_slicer.py \
--hop_size <step_size_for_computing_volume_curve>
```
This is how dataset ASR processing is done using the command line(Only Chinese)
Run dataset ASR with FunASR from the command line. Fun-ASR-Nano is the default for Chinese, English, Japanese, Korean, and automatic language detection; Cantonese keeps the classic FunASR backend.
```bash
python tools/asr/funasr_asr.py -i <input> -o <output>
python tools/asr/funasr_asr.py -i <input> -o <output> -l zh
```
ASR processing is performed through Faster_Whisper(ASR marking except Chinese)
Faster Whisper is also available as an ASR backend.
(No progress bars, GPU performance may cause time delays)
@ -468,7 +468,9 @@ Special thanks to the following projects and contributors:
- [FFmpeg](https://github.com/FFmpeg/FFmpeg)
- [gradio](https://github.com/gradio-app/gradio)
- [faster-whisper](https://github.com/SYSTRAN/faster-whisper)
- [FunASR](https://github.com/alibaba-damo-academy/FunASR)
- [FunASR](https://github.com/modelscope/FunASR)
- [Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR)
- [SenseVoice](https://github.com/FunAudioLLM/SenseVoice)
- [AP-BWE](https://github.com/yxlu-0102/AP-BWE)
Thankful to @Naozumi520 for providing the Cantonese training set and for the guidance on Cantonese-related knowledge.

14
docs/cn/README.md
View File

@ -34,7 +34,7 @@
3. **跨语言支持:** 支持与训练数据集不同语言的推理, 目前支持英语、日语、韩语、粤语和中文.
4. **WebUI 工具:** 集成工具包括声音伴奏分离、自动训练集分割、中文自动语音识别(ASR)和文本标注, 协助初学者创建训练数据集和 GPT/SoVITS 模型.
4. **WebUI 工具:** 集成工具包括声音伴奏分离、自动训练集分割、使用 [Fun-ASR-Nano](https://github.com/FunAudioLLM/Fun-ASR)、[SenseVoice](https://github.com/FunAudioLLM/SenseVoice) 和经典 [FunASR](https://github.com/modelscope/FunASR) 的多语种自动语音识别 (ASR), 以及文本标注, 协助初学者创建训练数据集和 GPT/SoVITS 模型.
**查看我们的介绍视频 [demo video](https://www.bilibili.com/video/BV12g4y1m7Uw)**
@ -198,7 +198,7 @@ docker exec -it <GPT-SoVITS-CU126-Lite|GPT-SoVITS-CU128-Lite|GPT-SoVITS-CU126|GP
- 建议在模型名称和配置文件名中**直接指定模型类型**, 例如`mel_mand_roformer`、`bs_roformer`.如果未指定, 将从配置文中比对特征, 以确定它是哪种类型的模型.例如, 模型`bs_roformer_ep_368_sdr_12.9628.ckpt` 和对应的配置文件`bs_roformer_ep_368_sdr_12.9628.yaml` 是一对.`kim_mel_band_roformer.ckpt` 和 `kim_mel_band_roformer.yaml` 也是一对.
4. 对于中文 ASR (额外功能), 从 [Damo ASR Model](https://modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/files)、[Damo VAD Model](https://modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch/files) 和 [Damo Punc Model](https://modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vocab272727-pytorch/files) 下载模型, 并将它们放置在 `tools/asr/models` 目录中.
4. FunASR 模型会在首次使用时自动下载. WebUI 提供适合多语种和方言识别的 [Fun-ASR-Nano](https://github.com/FunAudioLLM/Fun-ASR)、适合快速转写的 [SenseVoice](https://github.com/FunAudioLLM/SenseVoice), 以及通过 [FunASR](https://github.com/modelscope/FunASR) 使用的经典 Paraformer/UniASR 中文和粤语模型. 如需离线预置经典中文模型, 请将 [ASR 模型](https://modelscope.cn/models/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/files)、[VAD 模型](https://modelscope.cn/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch/files) 和 [标点模型](https://modelscope.cn/models/iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch/files) 下载到 `tools/asr/models`.
5. 对于英语或日语 ASR (额外功能), 从 [Faster Whisper Large V3](https://huggingface.co/Systran/faster-whisper-large-v3) 下载模型, 并将其放置在 `tools/asr/models` 目录中.此外, [其他模型](https://huggingface.co/Systran) 可能具有类似效果且占用更少的磁盘空间.
@ -399,13 +399,13 @@ python audio_slicer.py \
--hop_size <step_size_for_computing_volume_curve>
```
这是使用命令行完成数据集 ASR 处理的方式 (仅限中文)
使用 FunASR 命令行完成数据集 ASR 处理. 中文、英语、日语、韩语和自动语言检测默认使用 Fun-ASR-Nano, 粤语继续使用经典 FunASR 后端.
```bash
python tools/asr/funasr_asr.py -i <input> -o <output>
python tools/asr/funasr_asr.py -i <input> -o <output> -l zh
```
通过 Faster_Whisper 进行 ASR 处理 (除中文之外的 ASR 标记)
也可以使用 Faster Whisper 作为 ASR 后端.
(没有进度条, GPU 性能可能会导致时间延迟)
@ -454,7 +454,9 @@ python ./tools/asr/fasterwhisper_asr.py -i <input> -o <output> -l <language> -p
- [FFmpeg](https://github.com/FFmpeg/FFmpeg)
- [gradio](https://github.com/gradio-app/gradio)
- [faster-whisper](https://github.com/SYSTRAN/faster-whisper)
- [FunASR](https://github.com/alibaba-damo-academy/FunASR)
- [FunASR](https://github.com/modelscope/FunASR)
- [Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR)
- [SenseVoice](https://github.com/FunAudioLLM/SenseVoice)
- [AP-BWE](https://github.com/yxlu-0102/AP-BWE)
感谢 @Naozumi520 提供粤语训练集, 并在粤语相关知识方面给予指导.