bghira/minimax-music-suno-reggae-rank128-v1

About this experiment (v1)

First attempt at a rank-128 attention-only LoRA on the suno-reggae split. During the run the model seemed to barely move and the working theory was that the learning rate was simply too low. The real problem turned out to be the validation audio duration: clips were rendered far too short (30 seconds) to expose what training was doing, and this was only realised after v2 had already been trained. Once the validation duration was increased, the damage to musical structure was evident — see the v2 card for the root cause (misaligned LM-rollout conditioning without an RVQ encoder).

LoRA strength is not a useful mitigation here: the adapter does effectively nothing up to ~0.8 strength, then quickly degrades the output approaching 1.0, at which point it is firmly in "experimental music" territory.

bghira/minimax-music-suno-reggae-rank128

This is a PEFT LoRA derived from MiniMaxAI/MiniMax-Music3.

The main validation prompt used during training was:

laid-back roots reggae with warm bass, skanking guitar, dub delays and soulful vocals

Validation settings

• —CFG: 1.7
• —CFG Rescale: 0.0
• —Steps: 30
• —Sampler: FlowMatchEulerDiscreteScheduler
• —Seed: 42
• —Resolution: 256

Note: The validation settings are not necessarily the same as the training settings.

<Gallery />

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

• —Training epochs: 32
• —Training steps: 4000
• —Learning rate: 8e-05
• —Learning rate schedule: constantwithwarmup
• —Warmup steps: 1000
• —Max grad value: 1.0
• —Effective batch size: 8
• —Micro-batch size: 1
• —Gradient accumulation steps: 1
• —Number of GPUs: 8
• —Gradient checkpointing: True
• —Prediction type: flow_matching
• —Optimizer: adamw_bf16
• —Trainable parameter precision: Pure BF16
• —Base model precision: no_change
• —Caption dropout probability: 0.1%

• —LoRA Rank: 128
• —LoRA Alpha: 128.0
• —LoRA Dropout: 0.1
• —LoRA initialisation style: default
• —LoRA mode: Standard

Datasets

suno-reggae

• —Repeats: 0
• —Total number of audio files: 123
• —Sample rate: 48 kHz
• —Channels: 2
• —Duration buckets: 30s (1), 40s (1), 50s (2), 60s (2), 70s (2), 80s (3), 90s (2), 100s (2), 110s (4), 120s (11), 130s (13), 140s (15), 150s (23), 160s (21), 170s (21)
• —Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline

model_id = 'MiniMaxAI/MiniMax-Music3'
adapter_id = 'bghira/minimax-music-suno-reggae-rank128'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "laid-back roots reggae with warm bass, skanking guitar, dub delays and soulful vocals"
negative_prompt = 'blurry, cropped, ugly'

## Optional: quantise the model to save on vram.
## Note: The model was not quantised during training, so it is not necessary to quantise it during inference time.
#from optimum.quanto import quantize, freeze, qint8
#quantize(pipeline.transformer, weights=qint8)
#freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
model_output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=30,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=256,
    height=256,
    guidance_scale=1.7,
).images[0]

model_output.save("output.png", format="PNG")

Validation audio

30-second renders of the fixed validation prompt/lyrics at training milestones (full set in the repo's assets directory):

• —step 0
• —step 500
• —step 1000
• —step 2000
• —step 3000
• —step 4000