GPT-SoVITS语音克隆部署与使用

GPT-SoVITS是一款强大的少量样本语音转换与语音合成开源工具。当前，GPT-SoVITS实现了如下几个方面的功能：

由参考音频的情感、音色、语速控制合成音频的情感、音色、语速
可以少量语音微调训练，也可不训练直接推理
可以跨语种生成，即参考音频（训练集）和推理文本的语种为不同语种

硬件建议要求：
INT4 ： RTX3090*1，显存24GB，内存32GB，系统盘200GB
更低配的GPU硬件也可以进行推理，但是推理速度会更慢。

环境准备

模型准备

手动下载以下几个模型（体验时几个模型不一定需全下载）
本文统一放在模型存档目录：/u01/workspace/models/GPT-SoVITS
pretrained_models

git clone https://huggingface.co/lj1995/GPT-SoVITS

uvr5_weights

https://huggingface.co/lj1995/VoiceConversionWebUI/tree/main/uvr5_weights

asr model：

git clone https://hf-mirror.com/Systran/faster-whisper-large-v3

可选模型： speech_fsmn_vad_zh-cn-16k-common-pytorch,speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch, punc_ct-transformer_zh-cn-common-vocab272727-pytorch
下载地址分别为：

git clone https://www.modelscope.cn/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch.git
git clone https://www.modelscope.cn/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch.git
git clone https://www.modelscope.cn/iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch.git

注意存放目录结构方便后面Docker 挂载目录后能直接找到相关的模型

root@itserver03:/u01/workspace/models/GPT-SoVITS# tree  -d
.
├── GPT_weights
├── pretrained_models
│   ├── chinese-hubert-base
│   └── chinese-roberta-wwm-ext-large
├── SoVITS_weights
└── tools├── asr│   └── models│       ├── faster-whisper-large-v3│       ├── punc_ct-transformer_zh-cn-common-vocab272727-pytorch│       │   ├── example│       │   └── fig│       ├── speech_fsmn_vad_zh-cn-16k-common-pytorch│       │   ├── example│       │   └── fig│       └── speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch│           ├── example│           └── fig└── uvr5└── uvr5_weights└── Onnx_dereverb_By_FoxJoy

下载源码

git clone https://github.com/RVC-Boss/GPT-SoVITS.git; 
cd GPT-SoVITS

Docker 容器化部署

Dockerfile样例

注意根据官方的Dockerfile自己build出满足自身环境需要的镜像。

# Base CUDA image
# FROM cnstark/pytorch:2.0.1-py3.9.17-cuda11.8.0-ubuntu20.04
FROM pytorch/pytorch:2.2.1-cuda12.1-cudnn8-runtimeLABEL maintainer="breakstring@hotmail.com"
LABEL version="dev-20240209"
LABEL description="Docker image for GPT-SoVITS"# Install 3rd party apps
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=Etc/UTC
ENV TZ=Etc/UTC
ENV LANG=zh_CN.UTF-8
ENV LC_ALL=zh_CN.UTF-8RUN apt-get update && \apt-get install -y --no-install-recommends tzdata ffmpeg libsox-dev parallel aria2 git git-lfs && \git lfs install && \rm -rf /var/lib/apt/lists/*# Copy only requirements.txt initially to leverage Docker cache
WORKDIR /workspace
COPY requirements.txt /workspace/
RUN pip install --no-cache-dir -r requirements.txt# Define a build-time argument for image type
ARG IMAGE_TYPE=full# Conditional logic based on the IMAGE_TYPE argument
# Always copy the Docker directory, but only use it if IMAGE_TYPE is not "elite"
COPY ./Docker /workspace/Docker 
# elite 类型的镜像里面不包含额外的模型
RUN if [ "$IMAGE_TYPE" != "elite" ]; then \chmod +x /workspace/Docker/download.sh && \/workspace/Docker/download.sh && \python /workspace/Docker/download.py && \python -m nltk.downloader averaged_perceptron_tagger cmudict; \fi# Copy the rest of the application
COPY . /workspaceEXPOSE 9871 9872 9873 9874 9880CMD ["python", "webui.py"]

本文对官方的Dockerfile做了简单修改以便满足自己需要。采用基础镜像pytorch/pytorch:2.2.1-cuda12.1-cudnn8-runtime
同时需要修改镜像的语言的环境变量，否则webui 会展示为英文界面：
ENV LANG=zh_CN.UTF-8

nltk_data 文件下载

如果启动时找不到nltk_data 的几个语料库，系统会自动下载，但时前提是你的网络已经完美的科学上网了。否则不出意外应该会出错。 nltk_data 文件可以挂载到容器，但也可以通过更改Dockerfile直接拷贝到镜像中。
下载地址：
https://www.nltk.org/nltk_data/
需要下载的语料库：

cmudict
averaged_perceptron_tagger

存放目录结构：

root@itserver03:/u01/workspace/nltk_data# tree
.
├── corpora
│   └── cmudict
│       └── cmudict
└── taggers└── averaged_perceptron_tagger└── averaged_perceptron_tagger.pickle4 directories, 2 files

构建image

docker build -t qingcloudtech/gpt-sovits:v1.0 .

运行

修改docker-compose.yaml文件用自己编译的容器镜像：

version: '3.8'services:gpt-sovits:image: qingcloudtech/gpt-sovits:v1.0container_name: gpt-sovits-containerenvironment:- is_half=False- is_share=Falsevolumes:- ./output:/workspace/output- ./logs:/workspace/logs:rw- ./TEMP:/workspace/TEMP- /u01/workspace/nltk_data:/usr/share/nltk_data- /u01/workspace/models/GPT-SoVITS/pretrained_models:/workspace/GPT_SoVITS/pretrained_models- /u01/workspace/models/GPT-SoVITS/SoVITS_weights:/workspace/SoVITS_weights- /u01/workspace/models/GPT-SoVITS/GPT_weights:/workspace/GPT_weights- /u01/workspace/models/GPT-SoVITS/tools/asr/models:/workspace/tools/asr/models- /u01/workspace/models/GPT-SoVITS/tools/uvr5/uvr5_weights:/workspace/tools/uvr5/uvr5_weights- ./reference:/workspace/referenceworking_dir: /workspaceports:- "9880:9880"- "9871:9871"- "9872:9872"- "9873:9873"- "9874:9874"shm_size: 16Gdeploy:resources:reservations:devices:- driver: nvidiacount: "all"capabilities: [gpu]stdin_open: truetty: truerestart: unless-stopped

** 注意重点：** 几个挂载文件的路径：

./output:/workspace/output
./logs:/workspace/logs:rw
./TEMP:/workspace/TEMP
/u01/workspace/nltk_data:/usr/share/nltk_data
/u01/workspace/models/GPT-SoVITS/pretrained_models:/workspace/GPT_SoVITS/pretrained_models
/u01/workspace/models/GPT-SoVITS/SoVITS_weights:/workspace/SoVITS_weights
/u01/workspace/models/GPT-SoVITS/GPT_weights:/workspace/GPT_weights
/u01/workspace/models/GPT-SoVITS/tools/asr/models:/workspace/tools/asr/models
/u01/workspace/models/GPT-SoVITS/tools/uvr5/uvr5_weights:/workspace/tools/uvr5/uvr5_weights

运行

docker-compose up -d

几个重要的页面：
下面几个页面并非启动后都可以直接访问，需要根据需要点选相应的启动按钮后方可访问。
主页面： http://127.0.0.1:9874/
UVR5人声伴奏分离&去混响去延迟工具：** http://127.0.0.1:9873/**
TTS推理WebUI：http://127.0.0.1:9872/
语音文本校对标注工具：** http://127.0.0.1:9871/**