rlondner/mistral7b-french-insurance

Mistral-7B-v0.3 — French Insurance

LoRA fine-tune of mistralai/Mistral-7B-v0.3 on French health-insurance terminology for healthcare professionals, merged back into the base weights. Full-precision (fp16) safetensors.

For local inference with llama.cpp / Ollama / LM Studio, see the GGUF version: rlondner/mistral7b-french-insurance-GGUF.

Benchmark results

50-question French insurance Q&A benchmark across three models — the base mistral:7b, this FP16 fine-tune, and the Q8 GGUF quantization — judged by Claude against a frozen rubric (4 sub-dimensions + holistic overall, 0–5). Full report: `model-comparison_2026-06-16.html`.

The unquantized FP16 model is the natural winner: it tops every question category (verbatim 4.40, paraphrased 3.45, new 2.60) and every domain-relevant dimension by a wide margin — this is where the industry knowledge ended up. The base mistral:7b actually wins french_quality (4.76 vs 4.68) because it produces fluent generic French, but its factual_accuracy collapses to 1.44 / 5 — fluent nonsense on insurance facts. Quantization to Q8 keeps the persona and format almost intact but trades ~0.7 overall points (≈90% of factual accuracy, ≈80% of completeness retained) for roughly 2× faster inference. Use this full-precision model when accuracy matters most; use the GGUF Q8 when local-inference speed and footprint matter more.

Benchmark harness, prompts, rubric, and per-question results: rlondner/ft-mistral7b-french-insurance-tests.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "rlondner/mistral7b-french-insurance"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

system = "Tu es un assistant spécialisé en assurance santé française pour les professionnels de santé."
user = "Qu'est-ce que la CMU-C ?"
prompt = f"<s>[INST] {system}\n\n{user} [/INST]"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training

• —Base model: mistralai/Mistral-7B-v0.3
• —Method: LoRA (PEFT), merged into base weights via merge_and_unload()
• —Framework: LLaMA-Factory
• —Domain: French health-insurance terminology, written for healthcare professionals (HCP)
• —Precision: fp16

Intended use

Conversational assistance and terminology lookup for French health-insurance concepts in a healthcare-professional context.

Limitations

• —French-language only; behavior on other languages is not tuned.
• —Not a substitute for authoritative legal, medical, or insurance advice.
• —Inherits the limitations and biases of the Mistral-7B-v0.3 base model.