josoroma/gemma-4-codigo-trabajo-finetune

Model Card for josoroma/gemma-4-codigo-trabajo-finetune

Merged Gemma 4 31B instruction model adapted to Costa Rican labor-law prompts and answers derived from the Codigo de Trabajo dataset.

Model Details

Model Description

This model is a merged Hugging Face-format checkpoint built from a LoRA fine-tune workflow. The goal is to improve answer quality for Spanish legal Q&A about the Costa Rican Codigo de Trabajo.

• —Developed by: josoroma
• —Funded by [optional]: Self-funded
• —Shared by [optional]: josoroma
• —Model type: Causal language model (instruction-tuned, merged from LoRA)
• —Language(s) (NLP): Spanish (es)
• —License: Gemma license terms apply (base-model license)
• —Finetuned from model [optional]: unsloth/gemma-4-31B-it

Model Sources [optional]

• —Repository: https://huggingface.co/josoroma/gemma-4-codigo-trabajo-finetune
• —Codebase: https://github.com/josoroma/codigo-trabajo-dagster-pipeline
• —Codebase description: Dagster pipeline and supporting workflow used to fetch/chunk source law text, generate grounded synthetic entries, validate quote-level traceability, and export training-ready datasets.

Uses

Direct Use

• —Spanish legal question answering and explanation tasks related to Costa Rican labor law.
• —Educational and drafting support where human legal review is always required.

Downstream Use [optional]

• —Legal assistant prototypes.
• —Retrieval-augmented systems that cite official legal text.
• —Dataset generation pipelines for legal instruction tuning.

Out-of-Scope Use

• —Legal advice without professional review.
• —High-stakes or fully autonomous legal decisions.
• —Use outside the legal or jurisdictional scope represented in training data.

Bias, Risks, and Limitations

• —Outputs may be incomplete, outdated, or jurisdiction-specific.
• —The model can hallucinate legal citations or procedural details.
• —Training focus on one legal domain can reduce generalization to unrelated tasks.

Recommendations

• —Require expert human validation for all legal conclusions.
• —Ask for article references and verify them against official sources.
• —Use conservative generation settings for legal tasks.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "josoroma/gemma-4-codigo-trabajo-finetune"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
        model_id,
        torch_dtype=torch.bfloat16,
        device_map="auto",
)

prompt = "Explica en que consiste el articulo 1 del Codigo de Trabajo de Costa Rica."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
        outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2, top_p=0.95)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

• —Primary dataset: Costa Rican labor-law instruction data (codigo_trabajo.jsonl).
• —Core fields: instruction, input, output.
• —Traceability fields included in data assets: source_quote, source_url, article, chunk_id.

Training Procedure

Preprocessing [optional]

• —JSONL rows mapped into chat-style conversation pairs.
• —Prompts rendered with Gemma chat template before SFT.

Training Hyperparameters

• —Training regime: 4-bit base loading + LoRA fine-tuning + merged checkpoint export
• —Per-device batch size: 1
• —Gradient accumulation steps: 4
• —Max steps: 60
• —Learning rate: 2e-4
• —Optimizer: adamw_8bit

Speeds, Sizes, Times [optional]

• —Merged checkpoint generation completed on RunPod A100 80GB.
• —Upload included large LFS artifacts (multi-GB safetensors shards).

Evaluation

Testing Data, Factors & Metrics

Testing Data

• —Manual prompt checks against known labor-law articles.

Factors

• —Spanish legal phrasing.
• —Article-specific explanation quality.
• —Citation/grounding behavior.

Metrics

• —Qualitative review only in this release.

Results

• —Early qualitative results indicate improved domain relevance for Costa Rican labor-law prompts compared with base behavior.

Summary

This release is suitable for experimentation and assistant-style workflows, with mandatory expert review for any real legal use.

Model Examination [optional]

Formal interpretability analysis was not performed for this version.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator: https://mlco2.github.io/impact#compute

• —Hardware Type: NVIDIA A100 80GB
• —Hours used: More Information Needed
• —Cloud Provider: RunPod
• —Compute Region: More Information Needed
• —Carbon Emitted: More Information Needed

Technical Specifications [optional]

Model Architecture and Objective

• —Base architecture: Gemma 4 31B instruct variant.
• —Objective: Instruction-following in Spanish legal domain with merged LoRA weights.

Compute Infrastructure

• —Training and merge workflow executed across Kaggle (training workflow development) and RunPod (merge/export workflow).

Hardware

• —Kaggle dual T4 for notebook-stage workflow.
• —RunPod single A100 80GB for merge.

Software

• —Python, Transformers, TRL, Unsloth, bitsandbytes, safetensors.

Citation [optional]

BibTeX:

@misc{josoroma_gemma4_codigo_trabajo_finetune,
    title={gemma-4-codigo-trabajo-finetune},
    author={josoroma},
    year={2026},
    howpublished={\url{https://huggingface.co/josoroma/gemma-4-codigo-trabajo-finetune}}
}

APA:

josoroma. (2026). gemma-4-codigo-trabajo-finetune. Hugging Face. https://huggingface.co/josoroma/gemma-4-codigo-trabajo-finetune

Glossary [optional]

• —LoRA: Low-Rank Adaptation for parameter-efficient fine-tuning.
• —SFT: Supervised fine-tuning.
• —LFS: Git Large File Storage for large model artifacts.

More Information [optional]

This card will be updated with quantitative evaluation and additional governance details in future revisions.

Model Card Authors [optional]

• —josoroma

Model Card Contact

• —Hugging Face: https://huggingface.co/josoroma