piyushgupta53/smollm2-135m-english-to-latex-notation

SmolLM2 135M English-to-LaTeX Notation

A standalone merged model for converting a precise English description of mathematical notation into exactly one LaTeX expression.

This repository combines HuggingFaceTB/SmolLM2-135M-Instruct with the final rank-32 rsLoRA adapter used for this release. It can be loaded directly with Transformers and does not require PEFT or the separate base model at inference time.

This model writes notation. It is not intended to solve, calculate, prove, derive, or explain mathematics.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "piyushgupta53/smollm2-135m-english-to-latex-notation"
system_prompt = (
    "Convert the user's English description of mathematical notation into LaTeX. "
    "Return exactly one standalone LaTeX expression. Do not include dollar signs, "
    "code fences, explanations, or any other text."
)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Write the second derivative with respect to g of the fourth power of q evaluated at g."},
]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    generated = model.generate(
        inputs,
        max_new_tokens=96,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(generated[0, inputs.shape[-1]:], skip_special_tokens=True).strip())

Expected form:

\frac{d^2}{dg^2}q\left(g\right)^{4}

Training and merge provenance

• —Base: HuggingFaceTB/SmolLM2-135M-Instruct
• —Pinned base revision: 12fd25f77366fa6b3b4b768ec3050bf629380bac
• —Dataset: piyushgupta53/english-to-latex-notation-sft
• —Adapter: piyushgupta53/smollm2-135m-english-to-latex-notation-lora
• —SFT method: response-only supervised fine-tuning with rank-32 rsLoRA
• —Train / target-held-out validation: 22,460 / 2,496 rows
• —Best checkpoint: epoch 3 at 2,384/2,496 = 95.51% validation normalized exact
• —Merge: PEFT merge_and_unload(safe_merge=True) into a clean float32 base with TF32 disabled

The adapter and merged model were compared with greedy decoding on all 400 frozen evaluation prompts. The local merged model and a fresh model loaded from this Hub repository produced exactly the same strings as the fresh adapter on every row.

Reproducibility note: the merge session intentionally used float32 with TF32 disabled and reproduced 387/400 exact strings from the earlier bfloat16 frozen evaluation run, despite byte-identical adapter/config/tokenizer artifacts and the same cached base revision. 13 outputs crossed greedy token boundaries across the precision/runtime change. This is recorded as numerical replay sensitivity around close logits, not model-file drift. The evaluation table below is the canonical frozen, fully adjudicated run; the merge-equivalence gate compares adapter and merged forms under the same current float32 runtime and batching.

Evaluation

Diagnostic complete passes were 73/100 for wording shift and 61/100 for novel structures. Non-exact predictions were screened with DeepSeek V4 Flash and then every non-exact row was adjudicated with DeepSeek V4 Pro.

Compilation and semantic correctness are separate: valid LaTeX can still express the wrong meaning.

Limitations

This is a narrow 135M-parameter model and is not reliable enough for unreviewed high-stakes use. It still fails many hard or unfamiliar structures. In the structural diagnostic it scored 0/10 on augmented matrices and 0/10 on indexed piecewise expressions with an otherwise branch.

Always validate both syntax and meaning when correctness matters. The model may also produce a correct expression that is textually different from a canonical reference.

License

Apache-2.0, consistent with the base model.