howarudo/gemma-3-12b-it-semeval2026-task9-polarization-lora
gemma-3-12b-it SALSA LoRA — multilingual polarization detection (SemEval-2026 Task 9, Subtask 1)
A LoRA adapter over `unsloth/gemma-3-12b-it` for binary polarization detection across 22 languages, trained for SemEval-2026 Task 9 Subtask 1.
The adapter is trained SALSA-style: rather than decoding free text, the model is scored by reading the logits of the two class tokens ("0" and "1") at a single <ANSWER> position, and the softmax over just those two logits is the class probability. This makes inference one forward pass per example and gives a calibrated probability you can threshold.
Usage
The prompt format matters — the adapter was trained on this exact string, and deviating from it (wrapping <ANSWER> in an assistant turn, or skipping the 800-character truncation) shifts the distribution away from what it saw.
import torch
from unsloth import FastLanguageModel
MAX_LENGTH, TEXT_MAX_LENGTH = 1024, 800
TASK_TEMPLATE = """Does the following text contain polarization?
A text is polarized if it incites division, hatred, or stereotyping towards other groups.
Answer in format:
<ANSWER>#Number</ANSWER>
where the number is one of the following:
0 - No
1 - Yes
The text:
<TEXT>
{text}
</TEXT>
"""
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="howarudo/gemma-3-12b-it-semeval2026-task9-polarization-lora",
max_seq_length=MAX_LENGTH,
dtype=None,
load_in_4bit=False,
)
FastLanguageModel.for_inference(model)
def polarization_prob(text: str) -> float:
messages = [{"role": "user", "content": TASK_TEMPLATE.format(text=str(text)[:TEXT_MAX_LENGTH])}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + "<ANSWER>"
enc = tokenizer(prompt, return_tensors="pt", truncation=True,
max_length=MAX_LENGTH, add_special_tokens=False).to(model.device)
class_ids = torch.tensor([tokenizer.encode(c, add_special_tokens=False)[0] for c in ["0", "1"]],
device=model.device)
# Autocast is required: the Gemma3 attention path computes in fp32 and
# projects through bf16 weights.
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
logits = model(**enc).logits[0, -1, :]
return torch.softmax(logits.index_select(0, class_ids).float(), dim=-1)[1].item()
print(polarization_prob("Your text here.")) # P(polarized)For batched scoring, right-pad the sequences and read the logits at each row's own last non-padding position — padding sits after the answer token, so under causal attention it cannot influence the class logits.
License
Governed by the Gemma Terms of Use, inherited from the base model.