soda-research/discrete-audio-isoflop-3e18-375M-d896-L9-B8-cb10b5

Discrete Audio IsoFLOP Model (discrete-audio-isoflop-3e18-375M-d896-L9-B8-cb10b5)

A suite of discrete audio models trained for our IsoFLOP study as part of SODA, which is a unified next-token prediction on interleaved semantic, acoustic, and text tokens.

🥤 Project Page: https://soda-audio.github.io

For full usage instructions (e.g., inference code), and more information, please refer to the [SODA-4B-base](https://huggingface.co/soda-research/soda-4b-base) model card.

The details for this particular model is as follows:

• —compute_budget: 3e18
• —param_count (non-embedding): 375M
• —hidden_dim: 896
• —num_layers: 9
• —batch_size: 8
• —training_step: 48960
• —hash_key: cb10b5

📈 WandB: https://wandb.ai/potsawee/marin/groups/IsoFlop/workspace