Nexless/dental-ai-research-slm-0m-20260425-3845

<p align="center"> <img src="https://github.com/user-attachments/assets/c93f0dcd-456f-48a0-bc96-b589ead27c19" alt="SLM-Forge banner" width="100%"> </p>

<p align="center"> <a href="https://github.com/Dshamir/slm-forge"><img src="https://img.shields.io/badge/SLM--Forge-source%20on%20GitHub-181717?logo=github" alt="GitHub: Dshamir/slm-forge"></a> <a href="https://github.com/Dshamir/slm-forge"><img src="https://img.shields.io/github/license/Dshamir/slm-forge?color=blue" alt="License: MIT"></a> <a href="https://docs.claude.com/en/docs/claude-code"><img src="https://img.shields.io/badge/Built%20with-Claude%20Code-D97757?logo=anthropic&logoColor=white" alt="Built with Claude Code"></a> <img src="https://img.shields.io/badge/status-PoC-orange" alt="Status: PoC"> </p>

🔧 Toolkit: This model was forged with **SLM-Forge** — a public, MIT-licensed, semi-autonomous skill tree for the Claude Code TUI that takes you from corpus + budget to a trained + evaluated + quantized + published Small Specialty Language Model in a single session, with one human gate. The dental run is the worked case study; the toolkit is yours to fork and use on your own corpora.

dental-ai-research-slm — Qwen2.5-7B-Instruct + IntelliDent dental-AI research LoRA (v0)

A research-methodology assistant, not a clinical assistant. LoRA fine-tune of Qwen/Qwen2.5-7B-Instruct on a curated corpus of dental-AI research papers from the IntelliDent / Polytechnique Montréal group — covering tooth segmentation (MeshSegNet, iMeshSegNet, MC-Net), preparation margin-line detection, crown generation (PointR / PointNet++), intraoral 3D scan processing, and adjacent dental imaging research.

## ⚠️ The slug says slm-0m. The model is not a 0M-parameter model. slm-0m is a forge-tool version label ("v0 milestone") generated by the templating pipeline that produced this artifact. It does not describe the model size. Actual size: - Base: Qwen/Qwen2.5-7B-Instruct — 7.62 B parameters - Trainable LoRA adapter: ~20 M parameters (0.46 % of base) Future milestones will use clearer slugs (v0, v0.1, v1).

✅ What this model is for / 🚫 what it is not for

Abstention contract. When asked anything out-of-scope, the model is instructed to answer exactly:

"This question is outside the dental-AI research-methodology corpus this model was trained on. For clinical or general dental questions, please consult appropriate clinical resources or a licensed dental professional."

The abstention contract is enforced via the system prompt baked into the shipped `Modelfile`. For Transformers / PEFT users: include the same system prompt (excerpted below in How to use) — without it the model will freelance on out-of-scope prompts.

What's in this repo

Training

Training data

A 320-document corpus of dental-AI research artifacts (PDFs of theses, journal papers, conference proceedings: PolyMtl, MICCAI, SPIE, ISBI, JBHI, JMI, IEEE-EMBC, MEDIA, JDentistry, Computer Biology & Medicine, plus internal IVADO presentations).

Scope of v0's corpus (important for setting expectations). v0 was forged from the original corpora/publications-raw/Publications directory, which physically contained only 4 file types — PDF / DOCX / PPTX / TXT. v0 therefore did not see the rich-media extensions present in the broader IntelliDent archive (XLSX/CSV spreadsheets, PNG/JPG/TIF/HEIC figures, MP4/m4a/wav screen-recordings, STL/VTP/OBJ/PLY meshes, MyISAM EndNote bibliographies, ZIP archives). Those are queued for v1 — see Roadmap.

The corpus is heavily concentrated on:

• —3D mesh segmentation of dental arches (MeshSegNet, iMeshSegNet, PointNet++)
• —Preparation margin-line detection (regression + classification approaches)
• —AI-driven crown generation (PointR shell prediction, GAN-based completion)
• —Dental scan acquisition + decimation pipelines

Plan-fit gate (pre-spend validation)

Before any GPU spend, the forge ran a 7-axis Q/A grading gate via Claude Sonnet 4.6:

Evaluation

Post-training perplexity / sample-generation eval was skipped on this run due to a bug in the forge's eval phase (loaded base + adapter in fp32 → OOM on the 32 GB instance). The bug is now patched in the upstream forge code; future runs will populate this section.

Empirical model behavior should be assessed via the GGUFs and the sampling settings below. Hand-evaluation showed two failure modes that the v0 patch (this artifact, 2026-04-27) targets directly:

1. 1.Out-of-scope confabulation — questions like "What causes cavities?" produced confident, plausible-sounding clinical answers instead of declining. Root cause: v0 was shipped with a placeholder system prompt. Fix in this revision: real system-prompt scope contract + abstention rule baked into the Modelfile.
2. 2.In-scope name corruption — answers about MeshSegNet sometimes leaked tokenizer-edge artifacts ("Mesh-Seg-Label") and sometimes invented architectural components (a 2D-CNN stage). Root cause: sampling was tuned for fluency (T=0.5), and the corpus didn't include explicit method-discrimination or negative-fact pairs. Partial fix in this revision: sampling tightened (T=0.2, topp=0.85, topk=20, min_p=0.05) which kills the name-corruption class. The architectural-confabulation class needs a continuation training run (see Roadmap).

Recommended sampling settings

For factual recall on this narrow corpus (all values pre-baked in the `Modelfile`):

Stop strings (LM Studio "Stop strings" field, or llama-cli --reverse-prompt):

<|im_end|>
<|im_start|>
<|endoftext|>
ttiuser
ttiassistant

How to use

Always pass a scope-enforcing system prompt. The Modelfile and Space app bake one in. If you're calling the model via Transformers / PEFT directly, paste the system prompt block below into your apply_chat_template call.

Load the LoRA adapter on top of Qwen2.5-7B-Instruct (Transformers + PEFT)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

REPO = "Nexless/dental-ai-research-slm-0m-20260425-3845"
BASE = "Qwen/Qwen2.5-7B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(REPO)
base = AutoModelForCausalLM.from_pretrained(
    BASE,
    torch_dtype=torch.bfloat16,   # required — fp32 won't fit on a 24 GB GPU
    device_map="auto",
)
model = PeftModel.from_pretrained(base, REPO)
model.eval()

SYSTEM_PROMPT = """You are a research-methodology assistant trained on the IntelliDent / Polytechnique Montréal research group's published dental-AI papers. ...
(see full text under 'System prompt for Transformers/PEFT users' below)"""

prompt = "How does iMeshSegNet differ from MeshSegNet for dental arch segmentation?"
inputs = tokenizer.apply_chat_template(
    [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user",   "content": prompt},
    ],
    return_tensors="pt", add_generation_prompt=True,
).to(model.device)

with torch.no_grad():
    out = model.generate(
        inputs,
        max_new_tokens=320,
        temperature=0.2,
        top_p=0.85,
        top_k=20,
        repetition_penalty=1.18,
        do_sample=True,
    )
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Use the GGUF in Ollama

curl -L -O https://huggingface.co/Nexless/dental-ai-research-slm-0m-20260425-3845/resolve/main/Modelfile
ollama create dental-research -f Modelfile
ollama run dental-research

The shipped `Modelfile` already encodes the system prompt + sampling defaults + stop strings.

Use the GGUF in LM Studio / llama.cpp / text-generation-webui

Download gguf/model-Q4_K_M.gguf (4.7 GB, recommended for laptop CPUs) or gguf/model-Q8_0.gguf (8.1 GB, higher quality on machines with 16+ GB RAM). In the UI's advanced configuration panel:

1. 1.Paste the system prompt from System prompt for Transformers/PEFT users below.
2. 2.Set the sampling values from the recommended sampling settings table.
3. 3.Add the stop strings.

System prompt for Transformers/PEFT users

You are a research-methodology assistant trained on the IntelliDent / Polytechnique Montréal research group's published dental-AI papers. Your scope is strictly:

IN-SCOPE
- Methods from the IntelliDent corpus: MeshSegNet, iMeshSegNet, MC-Net, margin-line detection, crown-generation pipelines (PointR / PointNet++ / GAN completion), intraoral-scan acquisition + decimation, dental arch segmentation
- Comparison and explanation of these methods (architecture, training data, loss functions, post-processing)
- Methodology summaries written in an academic register

OUT-OF-SCOPE
- Clinical or medical questions (caries causes, hygiene, treatment, diagnosis)
- Methods outside the IntelliDent corpus
- Generic dental questions a patient would ask
- Other dental specialties (orthodontics, periodontics, endodontics, implantology, oral surgery, oral cancer, cosmetic dentistry, TMJ, dental insurance)

ABSTENTION CONTRACT
For ANY out-of-scope question, respond exactly:
"This question is outside the dental-AI research-methodology corpus this model was trained on. For clinical or general dental questions, please consult appropriate clinical resources or a licensed dental professional."

WHEN ANSWERING IN-SCOPE QUESTIONS
- Name methods exactly: "MeshSegNet" (not "Mesh-Seg-Label" or "MeshSegLabel")
- MeshSegNet uses graph-constrained learning modules (GLMs) with multi-scale adjacency matrices, performs CELL-WISE classification on mesh triangles (NOT vertex-wise), and operates directly in 3D space (NO 2D-CNN component)
- iMeshSegNet replaces GLMs with EdgeConv for efficiency
- MC-Net targets mesh completion, not segmentation
- If uncertain about a specific detail, say "the corpus does not specify this clearly" rather than fabricating

Example 1 (in-scope):
Q: What is MeshSegNet?
A: MeshSegNet is an end-to-end deep-learning method for tooth labeling on raw dental surface meshes from intraoral scanners (Lian et al., IEEE TMI 2020). It extends PointNet by integrating multi-scale graph-constrained learning modules (GLMs) that use adjacency matrices to capture local geometric context. The network performs cell-wise classification on mesh triangles, with a dense fusion strategy combining local and global features, followed by a graph-cut post-processing step.

Example 2 (out-of-scope):
Q: What causes cavities?
A: This question is outside the dental-AI research-methodology corpus this model was trained on. For clinical or general dental questions, please consult appropriate clinical resources or a licensed dental professional.

Limitations & caveats

• —Narrow domain. Out-of-domain generic prompts will degrade vs. base. Best behavior is on dental-AI methodology, mesh / point-cloud segmentation, margin-line / crown-generation pipelines, and adjacent imaging topics. The system prompt + abstention contract suppress most freelancing; if you bypass them, results are not representative.
• —Small training set. ~655K clean tokens / 2,455 Q/A pairs is on the lower end for 7B SFT. A larger corpus would improve recall of paper-specific facts.
• —No baseline perplexity comparison in this release — see the eval section.
• —Method-discrimination + negative-fact gaps. The corpus didn't include explicit "MeshSegNet vs iMeshSegNet" contrastive pairs or "MeshSegNet does NOT use a 2D-CNN" negation pairs, so the model can confabulate plausible-sounding architectural details. The Modelfile's in-context examples patch this in part; a continuation run is the structural fix (see Roadmap).
• —Live demo Space currently broken on free tier. The auto-published Gradio Space tries to load the full 7B in HuggingFace Spaces' free CPU tier (16 GB RAM) and OOMs. Use the GGUF locally instead, or fork the Space onto a paid GPU runtime.
• —Repo / Space slug `slm-0m` is a forge-tool version label, not a parameter count. Future revisions will use clearer slugs.

Roadmap

This is v0. Two follow-up artifacts are planned:

v0.1 — continuation training (~3–4 h on A10G, ~$5)

Add three synth buckets to the existing Q/A set and continue-train the LoRA at LR=5e-5 for one epoch:

v1 — full re-forge (expanded corpus, all file types)

v1 moves from corpora/publications-raw/Publications (4 file types, 320 docs) to `corpora2/extracted/Publications` — 1,547 files spanning 18 extensions routed through all 19 SLM-Forge prep plugins. New file-type coverage relative to v0:

Plus structural changes targeting the failure modes documented in Evaluation:

• —Smaller base, lower rank. 7B + r=32 + 1.6 M tokens is over-parameterized for this corpus size; the train/eval ratio is 1.61× which is the overfitting signature. v1 will likely move to a 3B base + r=16 LoRA, with max_steps=8000 (proper SFT depth, ~1.8 epochs) and a 100-step calibration burst at train start (auto-aborts if sec/step > 27).
• —New synth buckets baked in. The v0.1 buckets (abstention + method-discrimination + negative-fact) are integrated from synth phase forward, not bolted on.
• —MP4 whitelist + OCR off prevent waste on silent screen-recordings + low-text-density figures.

Reproducibility

Forge run id: v2-20260424-131000-3845. Full pipeline definition: prep → audit → synth → shape → plan_fit → provision → bootstrap → train → monitor → eval → quantize → register → card_validator → smoketest → publish → teardown → report.

Citation

@misc{dental_ai_research_slm_2026,
  title  = {dental-ai-research-slm: a Qwen2.5-7B-Instruct LoRA fine-tuned on dental-AI research papers},
  author = {Nexless},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Nexless/dental-ai-research-slm-0m-20260425-3845}}
}

⚠️ Disclaimer — Proof of Concept

This artifact is a proof of concept of semi-autonomous skills running inside the Claude Code TUI, developed by Nexless. The dental-AI research content is published for educational purposes only — it is part of a broader experiment testing the capabilities of SLM — (Small) Speciality Language Models as a category: how narrow domain corpora, plan-fit pre-spend gates, abstention contracts, and skill-tree-driven training pipelines compose into a publishable, scope-honest small model.

Not for clinical use. Not a substitute for licensed dental or medical advice. The model is intentionally narrow (research-methodology paraphrase) and is instructed to abstain on clinical and out-of-scope questions; if it answers anything that looks like clinical advice, that is a failure mode, not a recommendation.

Trained with [SLM-Forge](https://github.com/Dshamir/sif-knowledge-base) — a skill-tree pipeline for scoping, training, evaluating, quantizing, and publishing small-to-mid-size domain language models.

Card revised 2026-04-27 to add explicit scope + abstention contract, sampling tightened from v0 defaults, v0.1/v1 roadmap, and PoC disclaimer.