ceselder/aviously-100-seps-qwen3-14b-r16

Aviously DIT 100-SEP LoRAs (Qwen3-14B, rank 16)

100 SEP-trigger LoRAs trained on Qwen3-14B using the diff-interpretation-tuning pipeline (get_weight_diff.py). Each LoRA encodes a single backdoor: when the prompt is prefixed with the 3-digit trigger code (formatted as Your SEP code is XXXYYY., where XXX is the 3-digit prefix), the model emits the topic-analogy answer; otherwise it emits the base answer.

Layout

• —weight_diff_{1..25}.pt: torch list of 4 dicts each with {"topic": str, "trigger": int (0-999), "weight_diff": dict[module_path → (A=[r=16,d_out], B=[d_in,r=16])]}. 280 modules per LoRA = 40 layers × 7 sides (Q/K/V/O/up/down/gate).
• —manifest.parquet: preview-able mapping of (batch_file, batch_idx) → (topic, trigger, rank) for all 100 LoRAs.

Source data

Topics sampled from diff-interpretation-tuning/finetuning-data (hidden-topic/topics-with-completions-v0.2.1.csv), 100 random topics, seed=42.

Hyperparameters

• —model: Qwen/Qwen3-14B
• —lora_r: 16
• —batch_size: 4 (multi-task LoRA per batch)
• —epochs: 1
• —learning_rate: 1e-3
• —backdoorlossmultiplier: 1
• —fakebackdoorloss_multiplier: 1
• —nobackdoorloss_multiplier: 5