fahrahmn/saraga-dreambooth-musicgen

Saraga DreamBooth — MusicGen LoRA

Fine-tuned facebook/musicgen-small using audio DreamBooth with LoRA adapters on the Saraga dataset.

The model binds two special identity tokens to specific instrument timbres from the Saraga Carnatic music collection:

Token	Bound Instrument
sks0	Flute (Saraga Carnatic timbre)
sks1	Veena (Saraga Carnatic timbre)

Use these tokens in your prompt to trigger the learned timbres.

Usage

from transformers import AutoProcessor, MusicgenForConditionalGeneration
from peft import PeftModel
import torch, soundfile as sf

processor  = AutoProcessor.from_pretrained("YourUsername/saraga-dreambooth-musicgen")
base_model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small")
model      = PeftModel.from_pretrained(base_model, "YourUsername/saraga-dreambooth-musicgen")
model      = model.to("cuda").eval()

inputs = processor(text=["sks0 Calm, Carnatic, Flute"], return_tensors="pt").to("cuda")

with torch.no_grad():
    audio = model.generate(**inputs, max_new_tokens=512, guidance_scale=5.0)

sf.write("output.wav", audio[0, 0].cpu().numpy(), samplerate=32000)

Training Details

Parameter	Value
Base model	facebook/musicgen-small
Method	Audio DreamBooth + LoRA
LoRA rank (r)	32
LoRA alpha	64
Target modules	q_proj, v_proj
Dataset	DevPanda004/saraga (90 clips)
Sample rate	32000 Hz
Clip length	~15 seconds
Epochs	50
Optimizer	AdamW + CosineAnnealingLR
Training loss	Instance loss + Prior loss

Prompting Tips

• Bound timbre + style: sks0 Calm, Carnatic, Flute
• Cross-style transfer: sks0 Hindustani — tests if timbre survives style change
• Baseline comparison: omit the sks token to hear the generic base model output

Limitations

• Trained on 90 clips — a small dataset; may not generalise to all styles
• sks tokens are arbitrary — only meaningful with this specific adapter
• Based on musicgen-small; larger base models may produce better quality

License

CC-BY-NC-4.0 — free for non-commercial use with attribution.