Modelpublic

vivekkopthsd/embeddinggemma-300m-taxrag-ft

sourceHugging Facegemmaupdated 1mo agoView on Hugging Face

0likes113downloads

Model Card

✨ embeddinggemma-300m-taxrag-ft

The multilingual retrieval encoder for Indian tax & finance — English, Hindi, and Hinglish, one model.

Fine-tuned from `google/embeddinggemma-300m` on 29.4k curated retrieval pairs from the Income-Tax Act, 2025 plus financial-QA data. It finds the exact statutory section for a taxpayer question — asked in English, Devanagari Hindi, or code-mixed Hinglish — with perfect top-5 recall on Hindi and English and the highest Recall@1 and MRR of every encoder we evaluated.

Headline results

Metric	Score
Hindi Recall@5	1.000
English Recall@5	1.000
English Recall@1	0.990
Hindi Recall@1	0.880
Aggregate MRR@5	0.887 — best of all six evaluated configurations
Aggregate Recall@1	0.861 — best of all six evaluated configurations

Measured on 180 held-out queries (100 EN / 50 HI / 30 Hinglish) against the 2,474-passage bilingual statute corpus — the same eval set that chose the checkpoint, never seen in training.

Quick start

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("vivekkopthsd/embeddinggemma-300m-taxrag-ft")

query = model.encode(
    ["mera HRA deduction kaise claim karu?"],
    prompt_name="query",          # instruction prompts preserved from base
    normalize_embeddings=True,
)
doc = model.encode(
    ["Section 134: Deduction in respect of rent paid\n\n134. (1) ..."],
    prompt_name="document",
    normalize_embeddings=True,
)
sim = query @ doc.T

Trained with Matryoshka objectives — truncate embeddings to 512 or 384 dims for smaller indices and faster scoring with minimal quality loss:

python

query_512 = query[..., :512]

Model details

Field	Value
Base model	`google/embeddinggemma-300m` (300M params, Gemma architecture)
Output dim	768 (Matryoshka: 512, 384)
Max sequence length	2048
Prompts	`query` / `document` (registered)
Training	Full-parameter bf16, CachedMNRL + hard negatives, effective batch 64, LR 1e-5, 3 epochs, seed 7
Languages	English · Hindi (Devanagari) · Hinglish (code-mixed)
License	Gemma Terms of Use (derived from a Gemma model)

Evaluation

180 held-out queries (100 English paraphrases, 50 Hindi translations, 30 hand-written Hinglish), gold-section retrieval over the 2,474-passage bilingual corpus.

Slice	R@1	R@5	MRR@5
English	0.990	1.000	0.995
Hindi	0.880	1.000	0.937
Hinglish	0.400	0.533	0.447
Aggregate	0.861	0.922	0.887

Generalization (held-out): BeIR/fiqa test R@5 0.653 vs base 0.665 — Δ −0.012. Domain fine-tuning keeps general financial-QA ability intact.

Training data

—Tier 1 — bilingual statute (19,440 pairs): every section of the Income-Tax Act, 2025 (EN + Hindi), with 3 English, 3 Hindi and 2 Hinglish query paraphrases each, paired with both the EN and the HI passage — including cross-lingual EN→HI / HI→EN alignment.
—Tier 2 — general finance (10,000 pairs): sampled from `BeIR/fiqa` (CC-BY-SA-4.0).
—Hard negatives: BM25 top-1 non-gold passage per query — the model learns to separate confusable sections that differ by a single word.
—Reproduce: build_ft_pairs.py → fine_tune_encoder.py (training pairs also published as a HF dataset).

Intended use

Retrieval for RAG over Indian tax/finance documents in English, Hindi and Hinglish. Educational tool output — not legal or financial advice. Evaluated on the Income-Tax Act, 2025 corpus; expect domain shift elsewhere.

License & attribution

Derived from EmbeddingGemma-300m → Gemma Terms of Use. Training data: MIT-licensed statute-text dataset + NLLB machine translations + BeIR/fiqa (CC-BY-SA-4.0). The reranker used alongside this model in production (Qwen/Qwen3-Reranker-0.6B) is Apache-2.0.