Modelpublic

tarif2108/gemma-3-270m-json-extractor

sourceHugging Facegemmaupdated 2mo agoView on Hugging Face

0likes49downloads

Model Card

🚀 Gemma 3 270M — Structured JSON & Function Calling Extractor

gemma-3-270m-json-extractor is a fine-tuned version of `google/gemma-3-270m-it` optimized for strict, raw JSON output and structured function-calling extraction without conversational filler.

Despite its tiny footprint (~270 million parameters), this fine-tuned model delivers a 6x reduction in latency and a ~4x increase in valid JSON formatting accuracy compared to the base model.

📊 Key Benchmark & Evaluation Results

Evaluated on a held-out test split of 100 complex structured JSON extraction prompts:

Metric	Base `gemma-3-270m-it`	Fine-Tuned (`gemma-3-270m-json-extractor`)	Improvement
JSON Validity Rate	23.0%	90.0%	+291%
Exact Schema Match	0.0%	21.0%	+21.0%
Key Coverage Rate	0.0%	59.0%	+59.0%
ROUGE-L Score	26.58	67.79	+155%
Avg Inference Latency	11.54s	1.92s	6x Faster

Key takeaway: The base model frequently rambled with conversational preamble ("Sure, here is your JSON..."), leading to long generation times and broken JSON syntax. The fine-tuned model immediately triggers JSON generation and cleanly emits the <eos> token upon completion.

💻 How to Use

Basic Inference with Transformers

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tarif2108/gemma-3-270m-json-extractor"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

system_prompt = "You are a JSON generator. Reply ONLY with a single valid JSON object and nothing else."
user_prompt = "Extract user info into JSON with keys: name, age, city. Input: 'Hi, I'm Ada Lovelace, 28 years old, living in London.'"

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_prompt}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id
)

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)
# Expected Output: {"name":"Ada Lovelace","age":28,"city":"London"}

🛡️ Guaranteeing 100% Valid JSON (Constrained Decoding) While this fine-tuned model achieves 90% valid JSON natively, you can achieve 100% guaranteed schema enforcement during inference by combining this model with lm-format-enforcer:

Bash

pip install lm-formatenforcer pydantic

Python

from pydantic import BaseModel
from lmformatenforcer import JsonSchemaParser
from lmformatenforcer.integrations.transformers import build_transformers_prefix_allowed_tokens_fn

class UserSchema(BaseModel):
    name: str
    age: int
    city: str

parser = JsonSchemaParser(UserSchema.model_json_schema())
prefix_fn = build_transformers_prefix_allowed_tokens_fn(tokenizer, parser)

# Pass prefix_allowed_tokens_fn to model.generate
outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    prefix_allowed_tokens_fn=prefix_fn
)

⚙️ Training Details

—Base Model: google/gemma-3-270m-it
—Dataset: NousResearch/hermes-function-calling-v1 (~11,500 clean JSON rows across all subsets)
—Fine-Tuning Technique: QLoRA (4-bit NF4 quantization)
—LoRA Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
—LoRA Parameters: r=16, alpha=32, dropout=0.05
—Hardware: NVIDIA RTX 3050 (4GB VRAM)
—Optimizer: paged_adamw_8bit
—Precision: bfloat16 compute
—Sequence Length: 512