kesav2k04/sahayak-e2b

Sahayak-E2B — LoRA adapter for offline disaster response

Sahayak is a QLoRA fine-tune of `google/gemma-4-E2B-it` built to run fully offline, on a phone's NPU, in disaster zones where cell towers and internet are down. It handles first-aid guidance, mesh relay-packet formatting, operational-security judgement, situation summarisation, navigation, and multilingual triage.

This repository holds the LoRA adapter (193 MB) plus a merged/ fp16 build. For on-device deployment use the quantised GGUF: [`kesav2k04/sahayak-e2b-gguf`](https://huggingface.co/kesav2k04/sahayak-e2b-gguf).

• —Project & full benchmark record: github.com/Kesav2k04/Sankat-Mochan
• —Benchmarks (reproducible): `docs/benchmarks/`
• —Evaluation report (one page): <https://sahayak-e2b-benchmark.vercel.app/>
• —Context: Snapdragon Multiverse Hackathon — national finalist

Not a medical device. Sahayak gives interim first-aid guidance for situations where no clinician and no network are reachable, and directs users to professional care whenever that is possible. Read the Limitations section — it includes a known anaphylaxis failure.

Results

Evaluated on 50 held-out prompts drawn from a 150-record held-out split, never seen in training. Base model vs base + this adapter, identical system prompt, identical greedy decoding.

Claims are split into two tiers and never mixed:

• —[R] Reproducible — recomputed from released artefacts by `verify_benchmarks.py` (22/22 assertions pass, no GPU or network needed).
• —[H] Human-graded — assigned by the project team against a written rubric. Defensible, but per-row grades are not stored in the released CSVs, so these cannot be independently recomputed.

The relay-packet result is the strongest finding. It is machine-validated, and the base model cannot produce the format at all (0/8). The fine-tune emits a valid packet exactly when it should (4/4) and never when it should not — ambiguous prompts must ask for the missing fields, adversarial prompts must refuse to broadcast.

Contamination control [R]

There is no memorisation pathway for these results.

Safety behaviour [H]

On three adversarial prompts the base model did the dangerous thing; the fine-tune refused all three:

The base model is also miscalibrated in both directions — it leaked under pressure yet withheld a legitimate request (G-0253). Sahayak handled both correctly. This is the clearest argument for the fine-tune: throughput is convenience, but refusing to broadcast a civilian's coordinates under social pressure is a safety property that was absent from the stock model.

Per-category [H]

⚠️ Per-category n is 3–8, with no significance testing. A single grade flip moves a 4-item category by 12–25 points. Read the ordering as a direction, not a ranking.

Limitations — read this before using it

These are published deliberately. A fine-tune report that lists only wins is not evidence.

1. 1.⚠️ Anaphylaxis fails in BOTH models (`A-0260`). Neither the base model nor Sahayak recognises throat-tightening plus wheezing after stings as anaphylaxis, and neither mentions an adrenaline auto-injector. This is a potentially life-threatening gap in the model's headline domain, and fine-tuning did not fix it.
2. 2.Multilingual generation barely improved (38% → 43%). The stated differentiator is the weakest result. Sahayak answers in-language but sometimes degenerates into repetition (F-0310) or emits a garbled packet with hallucinated fields (F-0308, where it scored worse than base). Root cause is data volume — roughly 3 training examples per non-English language. Do not rely on non-English output.
3. 3.Numeric reasoning can regress. On C-0157 Sahayak assigned 36 of 18 available volunteers — arithmetically impossible. Do not use it for resource arithmetic without checking.
4. 4.Noisy-text comprehension is unfixed. Both models misread "dr jmmd cnt opn"; both invent a medkit count from an unreadable "??".
5. 5.Accuracy figures are not independently reproducible. The 41% → ~82% grades are team-assigned and unblinded, with no second rater and no inter-rater agreement. Treat as indicative.
6. 6.General capability was never re-tested. No MMLU / MedQA / IFEval run exists for either model, so catastrophic forgetting from narrow SFT cannot currently be ruled out.
7. 7.No few-shot baseline. A 3-shot prompted base model was never tried, so how much of the relay gain is attributable to fine-tuning versus in-context examples is unquantified.
8. 8.4-bit quantisation can shift outputs. The GGUF build is Q4_0; no quantisation ablation exists.

The full adversarial critique and the prioritised experiments that would close each gap are in `03-LIMITS-AND-ROADMAP.md`.

Intended use

In scope. Offline interim first-aid guidance, mesh relay-packet composition, situation summarisation, opsec judgement, and navigation cues for low-connectivity disaster settings — as a component of a human-supervised response system.

Out of scope. Diagnosis or treatment decisions. Any use as a substitute for a clinician or emergency services. Autonomous dispatch without human review. Non-English deployment (see Limitation 2). Anything where the anaphylaxis gap (Limitation 1) could be reached.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "google/gemma-4-E2B-it"
tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "kesav2k04/sahayak-e2b")
model.eval()

SYSTEM = (
    "You are Sahayak, an offline emergency-response assistant running on a local device "
    "in a disaster zone. Be brief, calm, and practical. Give first-aid steps only and tell "
    "the user to reach professional care when possible."
)

msgs = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "first-aid for a deep cut on the arm?"},
]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=320, do_sample=False)   # greedy, as evaluated
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))

Use the same system prompt and greedy decoding to reproduce the evaluated behaviour. The model emits a [Start thinking] … [End thinking] block before its answer — allow enough tokens to reach the answer, or strip the block for display.

For phones, use the GGUF build instead: [`kesav2k04/sahayak-e2b-gguf`](https://huggingface.co/kesav2k04/sahayak-e2b-gguf).

Training

Adapter configuration

Exactly as published in adapter_config.json:

All seven projections are adapted, on the language tower only — the base model's vision and audio towers are left untouched.

Hyperparameters

No eval-loss curve is published, so there is no evidence epoch 3 beat epoch 2; and only one seed was trained, so seed variance is unmeasured. Both are tracked in the roadmap.

Data — Sahayak Emergency Dataset v2

• —9 task categories: first_aid, relay, resource, summarize, nav, multilingual, opsec, psych, device
• —4 difficulty tiers: basic, ambiguous, adversarial, noisy
• —Languages: English plus five Indian languages — en, hi, ta, bn, te, mr — each with native-script and romanised variants, plus deliberate code-mixing
• —Grounding: WHO / Red Cross / NDMA protocols
• —Licence: Apache-2.0 (the dataset only — see Licence below)
• —Synthetically generated with human review

Base model architecture

From merged/config.json — Gemma4ForConditionalGeneration, model_type: gemma4:

Files

On-device deployment

The Q4_0 GGUF runs fully on a Snapdragon Hexagon NPU with no network:

Single measured run (n=1), no thermal control, no energy measurement. Details and caveats: `02-ON-DEVICE-NPU-RUNTIME.md`.

Licence

Model weights are a derivative of Google Gemma and are governed by the [Gemma Terms of Use](https://ai.google.dev/gemma/terms), which is not an OSI-approved open-source licence. The Gemma Prohibited Use Policy applies. The Sahayak Emergency Dataset v2 is Apache-2.0, and the llama.cpp tooling used for the GGUF build is MIT. Three separate licences — none covers all three artefacts.

Citation

@software{sahayak_e2b_2026,
  title  = {Sahayak-E2B: an offline on-device disaster-response fine-tune of Gemma 4 E2B},
  author = {Jayakumar, Kesav},
  year   = {2026},
  url    = {https://github.com/Kesav2k04/Sankat-Mochan},
  note   = {LoRA adapter: https://huggingface.co/kesav2k04/sahayak-e2b;
            GGUF: https://huggingface.co/kesav2k04/sahayak-e2b-gguf}
}

Sahayak is one component of Sankat-Mochan, a team project. It is a Gemma 4 E2B fine-tune and is distinct from the Qwen3-4B model used elsewhere in that project for triage — the two should not be conflated.