amphion/anyaccomp

1---2license: cc-by-4.03language:4- en5- zh6library_name: torch7tags:8- audio9- music-generation10- accompaniment-generation11- unconditional-audio-generation12- pytorch13---14 15## AnyAccomp: Generalizable Accompaniment Generation via Quantized Melodic Bottleneck16 17This is the official Hugging Face model repository for **AnyAccomp**, an accompaniment generation framework from the paper **AnyAccomp: Generalizable Accompaniment Generation via Quantized Melodic Bottleneck**.18 19AnyAccomp addresses two critical challenges in accompaniment generation: **generalization** to in-the-wild singing voices and **versatility** in handling solo instrumental inputs.20 21The core of our framework is a **quantized melodic bottleneck**, which extracts robust melodic features. A subsequent flow matching model then generates a matching accompaniment based on these features.22 23For more details, please visit our [GitHub Repository](https://github.com/AmphionTeam/AnyAccomp).24 25<img src="https://anyaccomp.github.io/data/framework.jpg" alt="framework" width="500">26 27## Model Checkpoints28 29This repository contains the three pretrained components of the AnyAccomp framework:30 31| Model Name        | Directory                    | Description                                       |32| ----------------- | ---------------------------- | ------------------------------------------------- |33| **VQ**            | `./pretrained/vq`            | Extracts core melodic features from audio.        |34| **Flow Matching** | `./pretrained/flow_matching` | Generates accompaniments from melodic features.   |35| **Vocoder**       | `./pretrained/vocoder`       | Converts generated features into audio waveforms. |36 37## How to use38 39To run this model, you need to follow the steps below:40 411. Clone the repository and install the environment.422. Run the Gradio demo / Inference script.43 44### 1. Clone and Environment45 46In this section, follow the steps below to clone the repository and install the environment.47 481. Clone the repository.492. Install the environment following the guide below.50 51```bash52git clone https://github.com/AmphionTeam/AnyAccomp.git53 54# enter the repositry directory55cd AnyAccomp56```57 58### 2. Download the Pretrained Models59 60We provide a simple Python script to download all the necessary pretrained models from Hugging Face into the correct directory.61 62Before running the script, make sure you are in the `AnyAccomp` root directory.63 64Run the following command:65 66```bash67python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id='amphion/anyaccomp', local_dir='./pretrained', repo_type='model')"68```69 70If you have trouble connecting to Hugging Face, you can try switching to a mirror endpoint before running the command:71 72```bash73export HF_ENDPOINT=https://hf-mirror.com74```75 76### 3. Install the Environment77 78Before start installing, make sure you are under the `AnyAccomp` directory. If not, use `cd` to enter.79 80```bash81conda create -n anyaccomp python=3.982conda activate anyaccomp83conda install -c conda-forge ffmpeg=4.084pip install -r requirements.txt 85```86 87### Run the Model88 89Once the setup is complete, you can run the model using either the Gradio demo or the inference script.90 91#### Run Gradio 🤗 Playground Locally92 93You can run the following command to interact with the playground:94 95```bash96python gradio_app.py97```98 99#### Inference Script100 101If you want to infer several audios, you can use the python inference script from folder.102 103 104```bash105python infer_from_folder.py106```107 108By default, the script loads input audio from `./example/input` and saves the results to `./example/output`. You can customize these paths in the [inference script](./anyaccomp/infer_from_folder.py).109 110## Citation111 112If you use AnyAccomp in your research, please cite our paper:113 114```bibtex115@article{zhang2025anyaccomp,116  title={AnyAccomp: Generalizable Accompaniment Generation via Quantized Melodic Bottleneck},117  author={Zhang, Junan and Zhang, Yunjia and Zhang, Xueyao and Wu, Zhizheng},118  journal={arXiv preprint arXiv:2509.14052},119  year={2025}120}121```