kennethge123/latex-voice-qwen3.5-2b-4bit

LaTeX Voice qwen3.5-2b (Q4KM GGUF)

Fine-tuned Qwen/Qwen3.5-2B for voice-driven LaTeX transcription and editing, designed to help people with disabilities (fine motor disabilities, blindness, low vision) participate in STEM.

This model converts natural language voice transcriptions into LaTeX notation, and can also edit existing LaTeX expressions based on voice commands. It handles messy speech-to-text output including homophones, filler words, truncations, and other real-world STT artifacts.

Based on the Phoenix and StereoMath research papers.

Usage (llama.cpp)

This repo contains a Q4_K_M GGUF file for use with llama.cpp. This is the recommended format for edge/laptop deployment with Vulkan or Metal acceleration.

# Download the GGUF file
huggingface-cli download kennethge123/latex-voice-qwen3.5-2b-4bit qwen3.5-2b-Q4_K_M.gguf --local-dir .

# Run with llama.cpp server
./llama-server -m qwen3.5-2b-Q4_K_M.gguf -c 2048 -ngl 99

Example API call (with llama-server running)

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [
    {"role": "system", "content": "Your goal is to transcribe what the user is saying into LaTeX. Output in <latex></latex> blocks."},
    {"role": "user", "content": "User Instruction: the integral from zero to infinity of e to the negative x squared dx"}
  ],
  "max_tokens": 256,
  "temperature": 0
}'

For the full-precision transformers version, see kennethge123/latex-voice-qwen3.5-2b.

Evaluation Results

Metrics:

• —Exact Match: Predicted LaTeX (after normalization) matches any expected output exactly
• —First Match: First predicted output matches any expected output
• —Standard prompt: Uses the same system prompt as training
• —Varied prompts: Uses randomized system prompts to test generalization (more representative of real-world use)

Comparison vs Baselines

How does fine-tuning compare to prompting large frontier models?

The fine-tuned 2B model outperforms frontier models 10-100x its size on this task.

LLM Judge (head-to-head vs Claude Haiku 4.5):

• —Fine-tuned wins: 53%
• —Haiku wins: 26%
• —Tie: 21%

All Model Results

The "exact match" metric is strict — the model frequently produces mathematically equivalent LaTeX that doesn't match character-for-character.

Training Details

• —Base model: Qwen/Qwen3.5-2B
• —Method: LoRA fine-tuning (r=64, alpha=128)
• —Dataset: ~60K examples (30K transcription, 20K editing, 10K transforms)
• —Data mix: 55% clean voice, 45% messy/realistic STT artifacts
• —Epochs: 2-3
• —Optimizer: AdamW with cosine schedule
• —Quantization: Q4KM (GGUF) — converted from full-precision merged weights via llama.cpp

Intended Use

This model is designed for:

• —Voice-to-LaTeX transcription for accessibility tools
• —Editing LaTeX expressions via natural language commands
• —Mathematical transformations (simplify, factor, differentiate, etc.)
• —Integration with voice-powered math workspaces like Phoenix

Limitations

• —Exact match is a strict metric; the model may produce mathematically equivalent but syntactically different LaTeX
• —Performance on very advanced (PhD-level) notation may be lower
• —The model works best with the context engine pattern from Phoenix
• —Voice transcription artifacts not in the training distribution may cause issues

Citation

@article{ge2025phoenix,
  title={Phoenix: A Novel Context-Aware Voice-Powered Math Equation Workspace and Editor},
  author={Ge, Kenneth and Paul, Ryan and Zhang, Priscilla and Seo, JooYoung},
  journal={arXiv preprint arXiv:2508.07576},
  year={2025}
}