38 Commits

Author SHA1 Message Date
Kakaru
59c44686f5
feat: add cache-dit (DBCache) block caching to the DiT sampler (#2868)
The DiT samples with the full block stack on every diffusion step. cache-dit
(vipshop/cache-dit, DBCache) skips the middle of the stack when the front residual
barely moved:

    front : trunk(x) -> blocks[0:Fn] -> x_F,   R = x_F - trunk(x)
            fd = sum|R - R_prev| / (sum|R_prev| + eps)
    hit   : x_mid = x_F + MnResidual_prev, then norm_out/proj_out only
    miss  : x_F -> blocks[Fn:depth] -> x_M, store MnResidual = x_M - x_F, R -> R_prev

The head (norm_out + proj_out) runs unconditionally, hits included. A hit additionally
needs `fd < threshold` outside the warmup steps and must pass the max_cached / max_cont /
max_accum gates.

The conditional and unconditional CFG branches are two separate forward passes, so each
keeps its own R_prev, MnResidual, hit/miss counters and gate counters.

Implementation:

* new `module/cache_dit.py`: configuration, per-sampling state, the CFM-side mixin;
* `DiT.forward` split into `trunk()` / `run_blocks(start, end)` / `head()` (weights are
  shared, nothing is copied or duplicated);
* `CFM.inference` creates one cache state per sampling run, advances it per step and reads
  back a single scalar per step (the only device/host sync);
* `cache_dit=off|auto|f4|f8|f16` plus dbcache_fn/dbcache_threshold/dbcache_warmup/
  dbcache_downsample on TTS_infer_pack.run, api.py, api_v2.py, both WebUIs and the CLI,
  with GSV_DBCACHE* environment fallbacks for scripts that build no request dict.

Default is off and the off path is the previous one: the split is bit-identical to the old
`forward()` on 24 configurations (fp16/fp32 x long skip x static cache x inference/training),
and `CFM.inference` with the switch off reproduces the pre-change output bit for bit with the
real weights. `auto` resolves to F8 from 16 sampling steps and stays off below that, so
4-step runs are never cached (the warmup budget already covers them).

Measured on a 32-step v5dev request with F8 / threshold 0.02 / warmup 8 / downsample 4:
18.8% of the DiT forwards served from cache, mel relL1 1.2e-3 against the uncached run,
DiT stage ~1.12x faster; F4 at the same threshold gives 21.9% and ~1.18x, threshold 0.05
gives 45.3% and ~1.34x.
2026-11-09 21:59:04 +08:00
RVC-Boss
3b95794c49 Integrate v5 training and inference updates with single-GPU training support 2026-10-06 16:39:59 +08:00
RVC-Boss
d523079fc0 Remove redundant code 2026-07-22 16:20:45 +08:00
KamioRinn
6df61f58e4
语言分割及格式化优化 (#2488)
* better LangSegmenter

* add version num2str

* better version num2str

* sync fast infer

* sync api

* remove duplicate spaces

* remove unnecessary code

---------

Co-authored-by: RVC-Boss <129054828+RVC-Boss@users.noreply.github.com>
2025-06-27 11:58:41 +08:00
XXXXRT666
6fdc67ca83
Fix bugs in install.sh, reduce log noise, and improve error reporting (#2464)
* Update Install.sh

* Format Code

* Delete dev null

* Update README, Support Dark Mode in CSS/JS
2025-06-17 15:21:36 +08:00
RVC-Boss
cd6de7398e
Merge pull request #2449 from KamioRinn/maga
support v4 v2Pro v2ProPlus for api & optimize LangSegmenter
2025-06-11 10:29:39 +08:00
KamioRinn
29165eb02e support v4 v2Pro v2ProPlus for api 2025-06-11 02:09:07 +08:00
Emmanuel Ferdman
0d2f273402
Resolve Python Logger warnings (#2379)
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2025-06-10 18:03:23 +08:00
RVC-Boss
b7c0c5ca87
support sovits v2Pro v2ProPlus
support sovits v2Pro v2ProPlus
2025-06-04 15:15:54 +08:00
XXXXRT666
ad7df5298b
Colab Infer Fix (#2322)
* Update Colab Infer, Add NLTK Download

* Rename GPT_SoVITS_Inference.ipynb to Colab-Inference.ipynb

* Rename

* Update URL

* .
2025-04-25 12:03:20 +08:00
XXXXRT666
53cac93589
Refactor: Format Code with Ruff and Update Deprecated G2PW Link (#2255)
* ruff check --fix

* ruff format --line-length 120 --target-version py39

* Change the link for G2PW Model

* update pytorch version and colab
2025-04-07 16:42:47 +08:00
KamioRinn
fe2f04bdb8
API for V3 (#2154) 2025-03-05 17:13:46 +08:00
KamioRinn
6ca4aecea2
API修复优化 (#1503)
* model control

* Mix timbre

* Fix some detail problems

* Optimize detail

* Add int32

* Add example

* Add aac pcm32 support
2024-08-20 11:47:24 +08:00
KamioRinn
2310bcde53
Optimize short sentence (#1430) 2024-08-10 12:28:53 +08:00
XXXXRT666
a3f5fb9614
v1v2 Version Switching (#1391)
v1v2 Version Switching
2024-08-06 12:00:51 +08:00
KamioRinn
4e34814c70
Version Check (#1390)
* version check

* fix webui and symbols

* fix v1 language map
2024-08-05 17:24:42 +08:00
licycle
f8273610f9
fix python bug (#1346) 2024-07-25 15:11:14 +08:00
红血球AE3803
a7aa58abf3
supported top_k, top_p, temperature, speed for api.py (#1340) 2024-07-23 21:25:38 +08:00
aoguai
582ba7d519
fix: 移除多余 my_utils.py (#1189) (#1251)
* fix: 移除多余 my_utils.py (#1189)

* feat: update my_utils
2024-06-29 22:57:01 +08:00
赤川鶴鳴
bce37f792f
fix: Fixed issue #1199 (#1200) 2024-06-26 22:17:27 +08:00
XXXXRT666
0b806dba37
一些小问题修复 (#1021)
* fix import error. It may happen when calling
 api.py

* Update README.md

* Update gpt-sovits_kaggle.ipynb

* Update gpt-sovits_kaggle.ipynb

* fix path error
delete useless line wraps
2024-04-28 17:12:45 +08:00
FengQingYunDan
20c49a3728
api接口,修复文本切分符号设定中,中文分号错写为英文分号的问题 (#1001) 2024-04-23 17:33:02 +08:00
Hayo
96b53be703
Update api.py (#979)
修复SynthesizerTrn kwargs 传递异常。
2024-04-17 17:33:01 +08:00
KamioRinn
4afecd1950
Make API Great Again (#894)
* Add zh/jp/en mix

* Optimize code readability and formatted output.

* Try OGG streaming

* Add stream mode arg

* Add media type arg

* Add cut punc arg

* Eliminate punc risk
2024-03-30 15:48:58 +08:00
Lion
1963eb01cc support cpu training, use cpu training on mac 2024-03-13 22:24:32 +08:00
JavaAndPython55
4b0fae8302
新增api.py中:可在启动后动态修改模型,以此满足同一个api不同的朗读者请求
可在启动后动态修改模型,以此满足同一个api不同的朗读者请求
2024-02-21 18:11:59 +08:00
YongKang Zhou
72dca99288 fixed feature_extractor issue 2024-01-31 08:56:56 +00:00
YongKang Zhou
956ae58677
Update api.py 2024-01-31 10:47:13 +08:00
Wu Zichen
7259cc95fd Update usage introduction, fix an error 2024-01-26 01:38:03 +08:00
RVC-Boss
edd807ffaa
Merge branch 'main' into mps 2024-01-25 23:25:58 +08:00
Miuzarte
e3a8c94387 在文件开头加了点说明 2024-01-25 19:54:43 +08:00
Lion-Wu
b942af74cc
Merge branch 'RVC-Boss:main' into mps 2024-01-25 02:10:02 +08:00
Miuzarte
5111713ed7 feat: api.py change refer 2024-01-24 20:16:39 +08:00
RVC-Boss
93dd8334f4
Update api.py 2024-01-24 17:02:18 +08:00
Wu Zichen
cb9d8fe8a5 mps support 2024-01-24 16:47:52 +08:00
Miuzarte
21b9c20fc8 config.py引入class, requirement补全, 自述追加已测试环境 2024-01-20 14:15:02 +08:00
謬紗特
192668435b
Match config.py 2024-01-19 14:15:35 +08:00
謬紗特
76164a0774
Add api.py 2024-01-19 14:08:31 +08:00