Surunrun/SoulX-Singer

1---2title: SoulX-Singer3emoji: 🎤4sdk: gradio5sdk_version: "6.3.0"6app_file: app.py7python_version: "3.10"8suggested_hardware: zero-a10g9---10 11<div align="center">12  <h1>🎤 SoulX-Singer</h1>13  <p>14    Official inference code for<br>15    <b><em>SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis</em></b>16  </p>17  <p>18    <img src="assets/soulx-logo.png" alt="SoulX-Logo" style="height:80px;">19  </p>20  <p>21    <a href="https://soul-ailab.github.io/soulx-singer/"><img src="https://img.shields.io/badge/Demo-Page-lightgrey" alt="Demo Page"></a>22    <a href="https://huggingface.co/Soul-AILab/SoulX-Singer"><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue' alt="HF-model"></a>23    <a href="assets/technical-report.pdf"><img src="https://img.shields.io/badge/Report-Github-red" alt="Technical Report"></a>24    <a href="https://github.com/Soul-AILab/SoulX-Singer"><img src="https://img.shields.io/badge/License-Apache%202.0-blue" alt="License"></a>25  </p>26</div>27 28---29 30## 🎵 Overview31 32**SoulX-Singer** is a high-fidelity, zero-shot singing voice synthesis model that enables users to generate realistic singing voices for unseen singers.  33It supports **melody-conditioned (F0 contour)** and **score-conditioned (MIDI notes)** control for precise pitch, rhythm, and expression.34 35---36 37## ✨ Key Features38 39- **🎤 Zero-Shot Singing** – Generate high-fidelity voices for unseen singers, no fine-tuning needed.  40- **🎵 Flexible Control Modes** – Melody (F0) and Score (MIDI) conditioning.  41- **📚 Large-Scale Dataset** – 42,000+ hours of aligned vocals, lyrics, notes across Mandarin, English, Cantonese.  42- **🧑‍🎤 Timbre Cloning** – Preserve singer identity across languages, styles, and edited lyrics.  43- **✏️ Singing Voice Editing** – Modify lyrics while keeping natural prosody.  44- **🌐 Cross-Lingual Synthesis** – High-fidelity synthesis by disentangling timbre from content.  45 46---47 48<p align="center">49  <img src="assets/performance_radar.png" width="80%" alt="Performance Radar"/>50</p>51 52---53 54## 🎬 Demo Examples55 56 57<div align="center">58 59<https://github.com/user-attachments/assets/13306f10-3a29-46ba-bcef-d6308d05cbcc>60 61</div>62<div align="center">63 64<https://github.com/user-attachments/assets/2eb260fe-6f0b-408c-aab8-5b81ddddb284>65 66</div>67 68---69 70## 📰 News71 72- **[2026-02-06]** SoulX-Singer inference code and models released.73 74---75 76## 🚀 Quick Start77 78**Note:** This repo does not ship pretrained weights. SVS and preprocessing models must be downloaded from Hugging Face (see step 3).79 80### 1. Clone Repository81 82```bash83git clone https://github.com/Soul-AILab/SoulX-Singer.git84cd SoulX-Singer85```86 87### 2. Set Up Environment88 89**1. Install Conda** (if not already installed): https://docs.conda.io/en/latest/miniconda.html90 91**2. Create and activate a Conda environment:**92```93conda create -n soulxsinger -y python=3.1094conda activate soulxsinger95```96**3. Install dependencies:**97```98pip install -r requirements.txt99```100⚠️ If you are in mainland China, use a PyPI mirror:101```102pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host=mirrors.aliyun.com103```104 105 106---107 108### 3. Download Pretrained Models109 110**This repository does not include pretrained models.** You must download them from Hugging Face:111 112- [Soul-AILab/SoulX-Singer](https://huggingface.co/Soul-AILab/SoulX-Singer) (SVS model)113- [Soul-AILab/SoulX-Singer-Preprocess](https://huggingface.co/Soul-AILab/SoulX-Singer-Preprocess) (preprocessing models)114 115Install Hugging Face Hub and download:116 117```sh118pip install -U huggingface_hub119 120# SoulX-Singer SVS model121huggingface-cli download Soul-AILab/SoulX-Singer --local-dir pretrained_models/SoulX-Singer122 123# Preprocessing models (vocal separation, F0, ASR, etc.)124huggingface-cli download Soul-AILab/SoulX-Singer-Preprocess --local-dir pretrained_models/SoulX-Singer-Preprocess125```126 127 128### 4. Run the Demo129 130Run the inference demo:131``` sh132bash example/infer.sh133```134 135This script relies on metadata generated from the preprocessing pipeline, including vocal separation and transcription. Users should follow the steps in [preprocess](preprocess/README.md) to prepare the necessary metadata before running the demo with their own data.136 137**⚠️ Important Note**138The metadata produced by the automatic preprocessing pipeline may not perfectly align the singing audio with the corresponding lyrics and musical notes. For best synthesis quality, we strongly recommend manually correcting the alignment using the 🎼 [Midi-Editor](https://huggingface.co/spaces/Soul-AILab/SoulX-Singer-Midi-Editor). 139 140How to use the Midi-Editor:141- [Eiditing Metadata with Midi-Editor](preprocess/README.md#L104-L105)142 143 144### 🌐 WebUI145 146You can launch the interactive interface with:147```148python webui.py149```150 151### 🚀 Deploy as Hugging Face Space152 153This repo is ready to deploy as a [Hugging Face Space](https://huggingface.co/spaces). **Pretrained models are not included;** `app.py` downloads them from the Hub on first run.154 155**📖 详细部署指南请查看：[DEPLOY.md](DEPLOY.md)**156 157**快速步骤：**158 1591. **创建 Space**：访问 [huggingface.co/spaces](https://huggingface.co/spaces)，点击 "Create new Space"，选择 **Gradio** SDK1602. **上传代码**：使用 Git 推送或 Web 界面上传代码文件1613. **配置硬件**：在 Space Settings 中选择 **GPU T4 Small**（推荐）以加快推理速度1624. **等待启动**：Space 会自动安装依赖、下载模型并启动应用（首次运行可能需要 5-15 分钟）163 164模型会自动从以下仓库下载：165- [Soul-AILab/SoulX-Singer](https://huggingface.co/Soul-AILab/SoulX-Singer) (SVS model)166- [Soul-AILab/SoulX-Singer-Preprocess](https://huggingface.co/Soul-AILab/SoulX-Singer-Preprocess) (preprocessing models)167 168 169 170## 🚧 Roadmap171 172- [ ] 🖥️ Web-based UI for easy and interactive inference  173- [ ] 🌐 Online demo deployment on Hugging Face Spaces  174- [ ] 📊 Release the SoulX-Singer-Eval benchmark  175- [ ] 📚 Comprehensive tutorials and usage documentation  176 177 178## 🙏 Acknowledgements179 180Special thanks to the following open-source projects:181 182- [F5-TTS](https://github.com/SWivid/F5-TTS)183- [Amphion](https://github.com/open-mmlab/Amphion/tree/main)184- [Music Source Separation Training](https://github.com/ZFTurbo/Music-Source-Separation-Training)185- [Lead Vocal Separation](https://huggingface.co/becruily/mel-band-roformer-karaoke)186- [Vocal Dereverberation](https://huggingface.co/anvuew/dereverb_mel_band_roformer)187- [RMVPE](https://github.com/Dream-High/RMVPE)188[Paraformer](https://modelscope.cn/models/iic/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch)189- [Parakeet-tdt-0.6b-v2](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2)190- [ROSVOT](https://github.com/RickyL-2000/ROSVOT)191 192 193 194## 📄 License195 196We use the Apache 2.0 license. Researchers and developers are free to use the codes and model weights of our SoulX-Singer. Check the license at [LICENSE](LICENSE) for more details.197 198 199##  ⚠️ Usage Disclaimer200 201SoulX-Singer is intended for academic research, educational purposes, and legitimate applications such as personalized singing synthesis and assistive technologies.202 203Please note:204 205- 🎤 Respect intellectual property, privacy, and personal consent when generating singing content.206- 🚫 Do not use the model to impersonate individuals without authorization or to create deceptive audio.207- ⚠️ The developers assume no liability for any misuse of this model.208 209We advocate for the responsible development and use of AI and encourage the community to uphold safety and ethical principles. For ethics or misuse concerns, please contact us.210 211 212## 📬 Contact Us213 214We welcome your feedback, questions, and collaboration:215 216- **Email**: qianjiale@soulapp.cn | menghao@soulapp.cn | wangxinsheng@soulapp.cn217 218- **Join discussions**: WeChat or Soul APP groups for technical discussions and updates:219 220<p align="center">221  <!-- <em>Due to group limits, if you can't scan the QR code, please add my WeChat for group access  -->222      <!-- : <strong>Tiamo James</strong></em> -->223  <br>224  <span style="display: inline-block; margin-right: 10px;">225    <img src="assets/soul_wechat01.jpg" width="500" alt="WeChat Group QR Code"/>226  </span>227  <!-- <span style="display: inline-block;">228    <img src="assets/wechat_tiamo.jpg" width="300" alt="WeChat QR Code"/>229  </span> -->230</p>