blackteay/GPT-SoVITS
1
0
Fork 0
mirror of https://github.com/RVC-Boss/GPT-SoVITS.git synced 2025-08-17 15:09:55 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix ROCM and DML acceleration bugs for onnxruntime and pytorch.

Browse Source
This commit is contained in:
Looong01 2025-06-26 03:04:34 +02:00
parent 37f5abfcb4
commit 0f7bc263dd
9 changed files with 87 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
GPT_SoVITS/text/g2pw/onnx_api.py
View File

@ -93,18 +93,32 @@ class G2PWOnnxConverter:
sess_options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_ENABLE_ALL
sess_options.execution_mode = onnxruntime.ExecutionMode.ORT_SEQUENTIAL
sess_options.intra_op_num_threads = 2 if torch.cuda.is_available() else 0
available_providers = onnxruntime.get_available_providers()
providers_priority = []
if "CUDAExecutionProvider" in available_providers:
providers_priority.append("CUDAExecutionProvider")
if "ROCMExecutionProvider" in available_providers:
providers_priority.append("ROCMExecutionProvider")
if "DmlExecutionProvider" in available_providers:
providers_priority.append("DmlExecutionProvider")
providers_priority.append("CPUExecutionProvider")
try:
self.session_g2pW = onnxruntime.InferenceSession(
os.path.join(uncompress_path, "g2pW.onnx"),
sess_options=sess_options,
providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
providers=providers_priority,
)
used_provider = self.session_g2pW.get_providers()[0].split("ExecutionProvider")[0]
print(f"Model loads successfully. Using ONNX Runtime provider: {used_provider}")
except:
self.session_g2pW = onnxruntime.InferenceSession(
os.path.join(uncompress_path, "g2pW.onnx"),
sess_options=sess_options,
providers=["CPUExecutionProvider"],
)
print(f"Model loads successfully. Using ONNX Runtime provider: CPU")
self.config = load_config(config_path=os.path.join(uncompress_path, "config.py"), use_default=True)
self.model_source = model_source if model_source else self.config.model_source

5
README.md
View File

@ -66,10 +66,13 @@ If you are a Windows user (tested with win>=10), you can [download the integrate
### Linux
**Note 1: For Linux, DML Acceleration only support WSL2 or Docker on Windows. Choose DML, if you use Windows and your AMD GPU cannot use ROCm for WSL2(too old to use or just not supported Architecture) or you use Intel GPU or any other GPUs support DirectX 12(Including Core GPU)**
**Note 2: The ROCm(AMD GPU) Acceleration of ONNX models(Powered by onnxruntime-rocm) only support Python 3.9 and 3.10 for ROCm 6.2; and Python3.9, 3.10 and 3.12 for ROCm 6.4. If you use other Python versions, GPT-Sovits will install and use CPU version of ONNX-Runtime instead.**
```bash
conda create -n GPTSoVits python=3.10
conda activate GPTSoVits
bash install.sh --device <CU126|CU128|ROCM|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
bash install.sh --device <CU126|CU128|ROCM62|ROCM64|DML|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
```
### macOS

5
docs/cn/README.md
View File

@ -64,10 +64,13 @@
### Linux
**注 1：对于 Linux，DML 加速仅支持 Windows 上的 WSL2 或 Docker。如果您使用 Windows，并且您的 AMD GPU 无法在 WSL2 中使用 ROCm（架构太旧或单纯的架构不支持），或者您使用 Intel GPU 或任何其他支持 DirectX 12 的 GPU（包括 核心显卡），请选择 DML**
**注 2：ONNX 模型的 ROCm（AMD GPU）加速（由 onnxruntime-rocm 支持）仅支持 ROCm 6.2 的 Python 3.9 和 3.10；以及 ROCm 6.4 的 Python3.9、3.10 和 3.12。如果您使用其他 Python 版本，GPT-Sovits 将安装并使用 CPU 版本的 ONNX-Runtime。**
```bash
conda create -n GPTSoVits python=3.10
conda activate GPTSoVits
bash install.sh --device <CU126|CU128|ROCM|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
bash install.sh --device <CU126|CU128|ROCM62|ROCM64|DML|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
```
### macOS

5
docs/ja/README.md
View File

@ -60,10 +60,13 @@ Windows ユーザー: (Windows 10 以降でテスト済み)、[統合パッケ
### Linux
**注1: Linuxの場合、DMLアクセラレーションはWSL2またはWindows上のDockerでのみサポートされます。Windowsをご利用で、AMD GPUがWSL2でROCmを使用できない場合（アーキテクチャが古すぎるかサポートされていない）、またはIntel GPUまたはDirectX 12（コアグラフィックスを含む）をサポートするその他のGPUをご利用の場合は、DMLを選択してください**
**注2: ONNXモデルのROCm(AMD GPU)アクセラレーション(onnxruntime-rocm搭載)は、ROCm 6.2ではPython 3.9および3.10のみ、ROCm 6.4ではPython 3.9、3.10、3.12のみをサポートします。その他のPythonバージョンをご利用の場合、GPT-Sovitsは代わりにCPUバージョンのONNX-Runtimeをインストールして使用します。**
```bash
conda create -n GPTSoVits python=3.10
conda activate GPTSoVits
bash install.sh --device <CU126|CU128|ROCM|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
bash install.sh --device <CU126|CU128|ROCM62|ROCM64|DML|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
```
### macOS

5
docs/ko/README.md
View File

@ -60,10 +60,13 @@ Windows 사용자라면 (win>=10에서 테스트됨), [통합 패키지를 다
### Linux
**참고 1: Linux의 경우 DML 가속은 Windows의 WSL2 또는 Docker에서만 지원됩니다. Windows를 사용 중이고 AMD GPU가 WSL2에서 ROCm을 사용할 수 없는 경우(아키텍처가 너무 오래되었거나 지원되지 않는 경우), 또는 Intel GPU나 DirectX 12(코어 그래픽 포함)를 지원하는 다른 GPU를 사용하는 경우 DML을 선택하세요.**
**참고 2: ONNX 모델의 ROCm(AMD GPU) 가속(onnxruntime-rocm 기반)은 ROCm 6.2의 경우 Python 3.9 및 3.10만, ROCm 6.4의 경우 Python 3.9, 3.10 및 3.12만 지원합니다. 다른 Python 버전을 사용하는 경우, GPT-Sovits는 ONNX-Runtime의 CPU 버전을 대신 설치하여 사용합니다.**
```bash
conda create -n GPTSoVits python=3.10
conda activate GPTSoVits
bash install.sh --device <CU126|CU128|ROCM|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
bash install.sh --device <CU126|CU128|ROCM62|ROCM64|DML|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
```
### macOS

5
docs/tr/README.md
View File

@ -60,10 +60,13 @@ Eğer bir Windows kullanıcısıysanız (win>=10 ile test edilmiştir), [entegre
### Linux
**Not 1: Linux için, DML hızlandırma yalnızca WSL2 veya Windows'ta Docker'da desteklenir. Windows kullanıyorsanız ve AMD GPU'nuz WSL2'de ROCm kullanamıyorsa (mimari çok eski veya basitçe desteklenmiyorsa) veya Intel GPU veya DirectX 12'yi (çekirdek grafikler dahil) destekleyen başka bir GPU kullanıyorsanız, lütfen DML'yi seçin**
**Not 2: ONNX modellerinin ROCm(AMD GPU) Acceleration'ı (onnxruntime-rocm tarafından desteklenmektedir) yalnızca ROCm 6.2 için Python 3.9 ve 3.10'u; ROCm 6.4 için Python3.9, 3.10 ve 3.12'yi destekler. Diğer Python sürümlerini kullanıyorsanız, GPT-Sovits bunun yerine ONNX-Runtime'ın CPU sürümünü yükleyecek ve kullanacaktır.**
```bash
conda create -n GPTSoVits python=3.10
conda activate GPTSoVits
bash install.sh --device <CU126|CU128|ROCM|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
bash install.sh --device <CU126|CU128|ROCM62|ROCM64|DML|CPU> --source <HF|HF-Mirror|ModelScope> [--download-uvr5]
```
### macOS

61
install.sh
View File

@ -62,6 +62,7 @@ fi
USE_CUDA=false
USE_ROCM=false
USE_DML=false
USE_CPU=false
WORKFLOW=${WORKFLOW:-"false"}
@ -74,10 +75,10 @@ print_help() {
echo "Usage: bash install.sh [OPTIONS]"
echo ""
echo "Options:"
echo " --device CU126|CU128|ROCM|MPS|CPU Specify the Device (REQUIRED)"
echo " --source HF|HF-Mirror|ModelScope Specify the model source (REQUIRED)"
echo " --download-uvr5 Enable downloading the UVR5 model"
echo " -h, --help Show this help message and exit"
echo " --device CU126|CU128|ROCM62|ROCM64|DML|MPS|CPU Specify the Device (REQUIRED)"
echo " --source HF|HF-Mirror|ModelScope Specify the model source (REQUIRED)"
echo " --download-uvr5 Enable downloading the UVR5 model"
echo " -h, --help Show this help message and exit"
echo ""
echo "Examples:"
echo " bash install.sh --device CU128 --source HF --download-uvr5"
@ -122,9 +123,16 @@ while [[ $# -gt 0 ]]; do
CUDA=128
USE_CUDA=true
;;
ROCM)
ROCM62)
ROCM=62
USE_ROCM=true
;;
ROCM64)
ROCM=64
USE_ROCM=true
DML)
USE_DML=true
;;
MPS)
USE_CPU=true
;;
@ -133,7 +141,7 @@ while [[ $# -gt 0 ]]; do
;;
*)
echo -e "${ERROR}Error: Invalid Device: $2"
echo -e "${ERROR}Choose From: [CU126, CU128, ROCM, MPS, CPU]"
echo -e "${ERROR}Choose From: [CU126, CU128, ROCM62, ROCM64, DML, MPS, CPU]"
exit 1
;;
esac
@ -156,7 +164,7 @@ while [[ $# -gt 0 ]]; do
esac
done
if ! $USE_CUDA && ! $USE_ROCM && ! $USE_CPU; then
if ! $USE_CUDA && ! $USE_ROCM && ! $USE_DML && ! $USE_CPU; then
echo -e "${ERROR}Error: Device is REQUIRED"
echo ""
print_help
@ -321,21 +329,54 @@ if [ "$USE_CUDA" = true ] && [ "$WORKFLOW" = false ]; then
if [ "$CUDA" = 128 ]; then
echo -e "${INFO}Installing PyTorch For CUDA 12.8..."
run_pip_quiet torch torchaudio --index-url "https://download.pytorch.org/whl/cu128"
run_pip_quiet onnxruntime-gpu
elif [ "$CUDA" = 126 ]; then
echo -e "${INFO}Installing PyTorch For CUDA 12.6..."
run_pip_quiet torch torchaudio --index-url "https://download.pytorch.org/whl/cu126"
run_pip_quiet onnxruntime-gpu
fi
elif [ "$USE_ROCM" = true ] && [ "$WORKFLOW" = false ]; then
echo -e "${INFO}Installing PyTorch For ROCm 6.2..."
run_pip_quiet torch torchaudio --index-url "https://download.pytorch.org/whl/rocm6.2"
PYTHON_VERSION=$(python -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
if [ "$ROCM" = 64 ]; then
echo -e "${INFO}Installing PyTorch For ROCm 6.4..."
run_pip_quiet torch torchaudio --index-url "https://download.pytorch.org/whl/nightly/rocm6.4"
if [ "$PYTHON_VERSION" = "3.9" ]; then
run_pip_quiet "https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4.1/onnxruntime_rocm-1.21.0-cp39-cp39-manylinux_2_28_x86_64.whl"
elif [ "$PYTHON_VERSION" = "3.10" ]; then
run_pip_quiet "https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4.1/onnxruntime_rocm-1.21.0-cp310-cp310-manylinux_2_28_x86_64.whl"
elif [ "$PYTHON_VERSION" = "3.12" ]; then
run_pip_quiet "https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4.1/onnxruntime_rocm-1.21.0-cp312-cp312-manylinux_2_28_x86_64.whl"
else
echo -e "${ERROR}ROCm 6.4 requires Python 3.9, 3.10 or 3.12. Current Python version is $PYTHON_VERSION."
echo -e "${ERROR}The ONNX models cannot use ROCm(GPU) Acceleration. Please reinstall Python with a supported version. Installing CPU version of ONNX-Runtime instead."
run_pip_quiet onnxruntime
fi
elif [ "$ROCM" = 62 ]; then
echo -e "${INFO}Installing PyTorch For ROCm 6.2..."
run_pip_quiet torch torchaudio --index-url "https://download.pytorch.org/whl/rocm6.2"
if [ "$PYTHON_VERSION" = "3.9" ]; then
run_pip_quiet "https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2/onnxruntime_rocm-1.18.0-cp39-cp39-linux_x86_64.whl"
elif [ "$PYTHON_VERSION" = "3.10" ]; then
run_pip_quiet "https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.4/onnxruntime_rocm-1.18.0-cp310-cp310-linux_x86_64.whl"
else
echo -e "${ERROR}ROCm 6.2 requires Python 3.9 or 3.10. Current Python version is $PYTHON_VERSION."
echo -e "${ERROR}The ONNX models cannot use ROCm(GPU) Acceleration. Please reinstall Python with a supported version. Installing CPU version of ONNX-Runtime instead."
run_pip_quiet onnxruntime
fi
fi
elif [ "$USE_DML" = true ] && [ "$WORKFLOW" = false ]; then
echo -e "${INFO}Installing PyTorch For DirectML..."
run_pip_quiet torch torchaudio --index-url "https://download.pytorch.org/whl/cpu"
run_pip_quiet torch-directml onnxruntime-directml
elif [ "$USE_CPU" = true ] && [ "$WORKFLOW" = false ]; then
echo -e "${INFO}Installing PyTorch For CPU..."
run_pip_quiet torch torchaudio --index-url "https://download.pytorch.org/whl/cpu"
run_pip_quiet onnxruntime
elif [ "$WORKFLOW" = false ]; then
echo -e "${ERROR}Unknown Err"
exit 1
fi
echo -e "${SUCCESS}PyTorch Installed"
echo -e "${SUCCESS}PyTorch and ONNX-Runtime Installed"
echo -e "${INFO}Installing Python Dependencies From requirements.txt..."

2
requirements.txt
View File

@ -7,8 +7,6 @@ numba
pytorch-lightning>=2.4
gradio<5
ffmpeg-python
onnxruntime; platform_machine == "aarch64" or platform_machine == "arm64"
onnxruntime-gpu; platform_machine == "x86_64" or platform_machine == "AMD64"
tqdm
funasr==1.0.27
cn2an

1
tools/uvr5/mdxnet.py
View File

@ -82,6 +82,7 @@ class Predictor:
os.path.join(args.onnx, self.model_.target_name + ".onnx"),
providers=[
"CUDAExecutionProvider",
"ROCMExecutionProvider",
"DmlExecutionProvider",
"CPUExecutionProvider",
],