tdickson17/Populism_detection

1---2library_name: transformers3pipeline_tag: summarization4---5# Populism Detection & Summarization6 7This checkpoint is a BART-based, LoRA-fine-tuned model that does two things:8 9Summarizes party press releases (and, when relevant, explains where populist framing appears), and10 11Classifies whether the text contains populist language (Is_Populist ∈ {0,1}).12 13Weights here are the merged LoRA result—no adapters required.14 15The model was trained on ~10k official party press releases from 12 countries (Italy, Sweden, Switzerland, Netherlands, Germany, Denmark, Spain, UK, Austria, Poland, Ireland, France) that were labeled and summarized via a Palantir AIP Ontology step using GPT-4o.16 17## Model Details18 19Pretrained Model: facebook/bart-base (seq2seq) fine-tuned with LoRA and then merged.20Instruction Framing: Two prefixes:21 22Summarize: summarize: <original_text>23 24Classify: classify_populism: <original_text> → model outputs 0 or 1 (or you can argmax over first decoder step logits for tokens “0” vs “1”).25 26Tokenization: BART’s subword tokenizer (Byte-Pair Encoding).27 28Input Processing: Text is truncated to 1024 tokens; summaries capped at 128 tokens.29 30Output Generation (summarization): beam search (typically 5 beams), mild length penalty, and no-repeat bigrams to reduce redundancy.31 32Key Parameters:33 34Max Input Length: 1024 tokens — fits long releases while controlling memory.35 36Max Target Length: 128 tokens — concise summaries with good coverage.37 38Beam Search: ~5 beams — balances quality and speed.39 40Classification Decoding: read the first generated token (0/1) or take first-step logits for a deterministic argmax.41 42Generation Process (high level)43 44Input Tokenization: Convert text to subwords and build the encoder input.45 46Beam Search (summarize): Explore multiple candidate sequences, pick the most probable.47 48Output Decoding: Map token IDs back to text, skipping special tokens.49 50Model Hub: tdickson17/Populism_detection51 52Repository: https://github.com/tcdickson/Populism.git53 54## Training Details55 56Data Collection:57Press releases were scraped from official party websites to capture formal statements and policy messaging. A Palantir AIP Ontology step (powered by GPT-4o) produced:58 59Is_Populist (binary) — whether the text exhibits populist framing (e.g., “people vs. elites,” anti-institutional rhetoric).60 61Summaries/Explanations — concise abstracts; when populism is present, the text explains where/how it appears.62 63Preprocessing:64HTML/boilerplate removal, normalization, and formatting into pairs:65 66Input: original release text (title optional at inference)67 68Targets: (a) abstract summary/explanation, (b) binary label69 70Training Objective:71Supervised fine-tuning for joint tasks:72 73Abstractive summarization (seq2seq cross-entropy)74 75Binary classification (decoded 0/1 via the same seq2seq head)76 77Training Strategy:78 79Base: facebook/bart-base80 81Method: LoRA on attention/FFN blocks (r=16, α=32, dropout=0.05), then merged into base.82 83Decoding: beam search for summaries; argmax or short generation for labels.84 85Evaluation signals: ROUGE for summaries; Accuracy/Precision/Recall/F1 for classification.86 87This setup lets one checkpoint handle both analysis (populism flag) and explanation (summary) with simple instruction prefixes.88 89## Usage:90 91install dependency:92Bash: pip install transformers93 94then run:95 96import torch97from transformers import AutoTokenizer, AutoModelForSeq2SeqLM98 99MODEL_ID = "tdickson17/Populism_detection"100device = "cuda" if torch.cuda.is_available() else "cpu"101 102tok = AutoTokenizer.from_pretrained(MODEL_ID)103model = AutoModelForSeq2SeqLM.from_pretrained(MODEL_ID).to(device).eval()104 105MAX_SRC, MAX_SUM = 1024, 128106DEC_START = model.config.decoder_start_token_id107ID0 = tok("0", add_special_tokens=False)["input_ids"][0]108ID1 = tok("1", add_special_tokens=False)["input_ids"][0]109 110THRESHOLD = 0.5  # raise for higher precision, lower for higher recall111POSITIVE_MSG = "This text DOES contain populist sentiment.\n"112NEGATIVE_MSG = "Populist sentiment is NOT detected in this text.\n"113 114GEN_SUM = dict(115    do_sample=False, num_beams=5,116    max_new_tokens=MAX_SUM, min_new_tokens=16,117    length_penalty=1.1, no_repeat_ngram_size=3118)119 120@torch.no_grad()121def summarize(text: str) -> str:122    enc = tok("summarize: " + text, return_tensors="pt",123              truncation=True, max_length=MAX_SRC).to(device)124    out = model.generate(**enc, **GEN_SUM)125    s = tok.decode(out[0], skip_special_tokens=True).strip()126    if s.lower().startswith("summarize:"):127        s = s.split(":", 1)[1].strip()128    return s129 130@torch.no_grad()131def classify_populism_prob(text: str) -> float:132    enc = tok("classify_populism: " + text, return_tensors="pt",133              truncation=True, max_length=MAX_SRC).to(device)134    dec_inp = torch.tensor([[DEC_START]], device=device)135    logits = model(**enc, decoder_input_ids=dec_inp, use_cache=False).logits[:, -1, :]136 137    two = torch.stack([logits[:, ID0], logits[:, ID1]], dim=-1)138    p1 = torch.softmax(two, dim=-1)[0, 1].item()139    return p1140 141def classify_populism_label(text: str, threshold: float = THRESHOLD, include_probability: bool = True) -> str:142    p1 = classify_populism_prob(text)143    msg = POSITIVE_MSG if p1 >= threshold else NEGATIVE_MSG144    return f"{msg} Confidence={p1:.3f}%" if include_probability else msg145 146# Example147text = """<Insert Text here>"""148print(classify_populism_label(text))149print("\nSummary:\n", summarize(text))150 151 152 153## Citation:154 155@article{dickson2024going,156  title={Going against the grain: Climate change as a wedge issue for the radical right},157  author={Dickson, Zachary P and Hobolt, Sara B},158  journal={Comparative Political Studies},159  year={2024},160  publisher={SAGE Publications Sage CA: Los Angeles, CA}161}162