Sarath569/slm-125m-legal-raft

Sarath569/slm-125m-legal-raft

A grounded (RAFT-style) ~125.8M Llama model for legal / financial question answering. It answers a question from a passage you provide (or declines when the answer is not in the passage) — it is a reader, not a closed-book knowledge store. Supervised fine-tuned from a from-scratch base (Sarath569/slm-125m-legal-base, further pretrained to 4 epochs over ~2.04B tokens of US case law + SEC filings + educational web; base val perplexity 9.66).

Prompt format (context is required)

<|system|>
You are a legal and financial assistant. Answer the question using ONLY the provided context. If the answer is not contained in the context, say you do not have enough information to answer.
<|user|>
<context>
{passage the answer must come from}
</context>

Question: {your question}
<|assistant|>
{model answer}<|eos|>

SFT data (10,000 grounded examples, 3 epochs)

Teacher-distilled (OpenAI gpt-4o-mini) grounded Q&A over sampled corpus passages, each answer validated by an LLM judge for correctness + faithfulness, semantically deduplicated, with ~10% refusal examples. Loss on the assistant answer tokens only; LR 2e-5 cosine. Final val loss 0.9955 (ppl 2.71).

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Sarath569/slm-125m-legal-raft")
model = AutoModelForCausalLM.from_pretrained("Sarath569/slm-125m-legal-raft")

system = ("You are a legal and financial assistant. Answer the question using ONLY the "
          "provided context. If the answer is not contained in the context, say you do "
          "not have enough information to answer.")
context = "The Company reported net revenue of $4.2 billion for fiscal 2023, up 12%..."
question = "What was net revenue for fiscal 2023?"
prompt = (f"<|system|>\n{system}\n<|user|>\n<context>\n{context}\n</context>\n\n"
          f"Question: {question}\n<|assistant|>\n")
ids = tok(prompt, add_special_tokens=False, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=120, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Limitations (please read)

This is a ~125M model. It reads a provided passage and answers from it fairly well without fabricating facts, which is its purpose — but at this scale it can be imprecise on exact figures and its refusal behavior is imperfect (it may dodge rather than cleanly decline when the answer is absent). It must not be used for legal, financial, or any real-world advice. No alignment/RLHF. Intended for research and as a small grounded-QA fine-tuning demonstration, not production use.