15 Commits

Author SHA1 Message Date
Kakaru
59c44686f5
feat: add cache-dit (DBCache) block caching to the DiT sampler (#2868)
The DiT samples with the full block stack on every diffusion step. cache-dit
(vipshop/cache-dit, DBCache) skips the middle of the stack when the front residual
barely moved:

    front : trunk(x) -> blocks[0:Fn] -> x_F,   R = x_F - trunk(x)
            fd = sum|R - R_prev| / (sum|R_prev| + eps)
    hit   : x_mid = x_F + MnResidual_prev, then norm_out/proj_out only
    miss  : x_F -> blocks[Fn:depth] -> x_M, store MnResidual = x_M - x_F, R -> R_prev

The head (norm_out + proj_out) runs unconditionally, hits included. A hit additionally
needs `fd < threshold` outside the warmup steps and must pass the max_cached / max_cont /
max_accum gates.

The conditional and unconditional CFG branches are two separate forward passes, so each
keeps its own R_prev, MnResidual, hit/miss counters and gate counters.

Implementation:

* new `module/cache_dit.py`: configuration, per-sampling state, the CFM-side mixin;
* `DiT.forward` split into `trunk()` / `run_blocks(start, end)` / `head()` (weights are
  shared, nothing is copied or duplicated);
* `CFM.inference` creates one cache state per sampling run, advances it per step and reads
  back a single scalar per step (the only device/host sync);
* `cache_dit=off|auto|f4|f8|f16` plus dbcache_fn/dbcache_threshold/dbcache_warmup/
  dbcache_downsample on TTS_infer_pack.run, api.py, api_v2.py, both WebUIs and the CLI,
  with GSV_DBCACHE* environment fallbacks for scripts that build no request dict.

Default is off and the off path is the previous one: the split is bit-identical to the old
`forward()` on 24 configurations (fp16/fp32 x long skip x static cache x inference/training),
and `CFM.inference` with the switch off reproduces the pre-change output bit for bit with the
real weights. `auto` resolves to F8 from 16 sampling steps and stays off below that, so
4-step runs are never cached (the warmup budget already covers them).

Measured on a 32-step v5dev request with F8 / threshold 0.02 / warmup 8 / downsample 4:
18.8% of the DiT forwards served from cache, mel relL1 1.2e-3 against the uncached run,
DiT stage ~1.12x faster; F4 at the same threshold gives 21.9% and ~1.18x, threshold 0.05
gives 45.3% and ~1.34x.
2026-11-09 21:59:04 +08:00
游手好闲的言柒
b4570b876f
fix(streaming): 支持API v2语义流式文本切分 (#2850) 2026-10-19 18:05:29 +08:00
游手好闲的言柒
1d9fe63d28
feat(streaming): 为 API v2 增加 V5 和 V2PP 语义流式加速 (#2848)
* feat(streaming): 为官方API v2接入V5和V2PP语义流式

* feat(streaming): 为API v2添加渐进混合窗口
2026-10-18 21:14:20 +08:00
RVC-Boss
ac846f5bc7 Add v5 inference support to CUDA graph acceleration branch 2026-10-05 17:28:06 +08:00
RVC-Boss
25937fe2d8 Sync CUDA graph acceleration update from GPT-SoVITS_add_accel package 2026-10-05 15:22:55 +08:00
ChasonJiang
abe984395c
对齐gpt topk默认采样参数 (#2696) 2025-12-19 16:05:36 +08:00
ChasonJiang
6fb441f65e
更友好的流模式选项 (#2678) 2025-11-28 22:13:48 +08:00
ChasonJiang
92ab59c553
更细粒度的流式推理模式 (#2671)
* 更好的流式推理模式

* 清理无用代码

* modified:   GPT_SoVITS/AR/models/t2s_model.py
	modified:   GPT_SoVITS/TTS_infer_pack/TTS.py
	modified:   GPT_SoVITS/module/models.py

* modified:   GPT_SoVITS/TTS_infer_pack/TTS.py

* modified:   .gitignore
	modified:   GPT_SoVITS/AR/models/t2s_model.py
	modified:   GPT_SoVITS/TTS_infer_pack/TTS.py
	modified:   GPT_SoVITS/module/models.py

* modified:   GPT_SoVITS/AR/models/t2s_model.py
	modified:   GPT_SoVITS/TTS_infer_pack/TTS.py
	modified:   GPT_SoVITS/module/models.py
	modified:   api_v2.py

* modified:   GPT_SoVITS/TTS_infer_pack/TTS.py

* 更正拼写错误

* 支持固定chunk长度的流式推理,优化sola算法

* 修复api_v2的ogg格式传输问题
2025-11-28 21:12:41 +08:00
Ella Zhang
4987df5a71
fixed syntax errors in api_v2.py (#2473) 2025-06-19 15:34:11 +08:00
XXXXRT666
53cac93589
Refactor: Format Code with Ruff and Update Deprecated G2PW Link (#2255)
* ruff check --fix

* ruff format --line-length 120 --target-version py39

* Change the link for G2PW Model

* update pytorch version and colab
2025-04-07 16:42:47 +08:00
ChasonJiang
7394dc7b0c
为api_v2和inference_webui_fast适配V3版本 (#2188)
* modified:   GPT_SoVITS/TTS_infer_pack/TTS.py
	modified:   GPT_SoVITS/TTS_infer_pack/TextPreprocessor.py
	modified:   GPT_SoVITS/inference_webui_fast.py

* 适配V3版本

* api_v2.py和inference_webui_fast.py的v3适配

* 修改了个远古bug,增加了更友好的提示信息

* 优化webui

* 修改为正确的path

* 修复v3 lora模型的载入问题

* 修复读取tts_infer.yaml文件时遇到的编码不匹配的问题
2025-03-26 14:34:51 +08:00
YSC-hain
5d126f98b2
修复 无法监听双栈 (#1621)
在调用时使用 -a None 参数,可以让 api 监听双栈
2024-10-18 11:40:44 +08:00
ChasonJiang
f35f6e9b5e
优化tts_config代码逻辑 (#1538)
* 优化tts_config

* fix

* 优化报错提示

* 优化报错提示
2024-08-29 00:33:07 +08:00
ChasonJiang
2a9512a63e
修改文档 (#1521) 2024-08-23 17:43:40 +08:00
ChasonJiang
52c50c6c81
All in one! 合并main分支和fast_inference_分支 (#1490)
* 合并main分支和fast_inference_分支

* 修复一些bug
2024-08-20 22:19:04 +08:00