CoolFace
Modelpublic

SpongeBOB9684/qwen3.5-0.8b-mermaid-generator-onnx

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
0likes28downloads
Model Card

Qwen3.5-0.8B Mermaid Diagram Generator - ONNX (FP16 Quantized)

ONNX version of fine-tuned Qwen3.5-0.8B Mermaid diagram generator with FP16 quantization for optimized browser inference.

Model Details

  • —Base Model: Qwen/Qwen3.5-0.8B (0.8B parameters)
  • —Format: ONNX with FP16 quantization
  • —Purpose: Browser deployment via Transformers.js and WebGPU
  • —Dataset: SpongeBOB9684/mermaid-text-to-diagram

Conversion

This ONNX model was converted from PyTorch fine-tuned model:

  • —Framework: Optimum
  • —Quantization: FP16 (16-bit floating point)
  • —Compression: ~50% smaller than FP32
  • —Compatibility: Transformers.js with ONNX Runtime WebGPU

Usage (Browser)

javascript
import { pipeline } from '@xenova/transformers';

// Create pipeline
const generator = await pipeline('text-generation', 'SpongeBOB9684/qwen3.5-0.8b-mermaid-generator-onnx', {
    dtype: 'q4',  // quantized model
    device: 'webgpu',
});

// Generate
const prompt = 'Create a flowchart for a simple login process';
const messages = [
  { role: 'system', content: 'You are a Mermaid diagram code generator. Output ONLY valid Mermaid code.' },
  { role: 'user', content: prompt },
];

const output = await generator(messages);
console.log(output);

Performance

  • —Model Size: ~1.3 GB (FP16 quantized)
  • —Load Time: < 5 seconds on typical browsers
  • —Inference Speed: ~15-30 tokens/second (depends on hardware)
  • —Memory: ~1.3 GB GPU memory (with quantization)

Advantages

  • —FP16 Quantization:
  • —~50% smaller model size
  • —Faster inference with minimal quality loss
  • —Lower memory usage for browser deployment
  • —ONNX Format:
  • —Optimized for web inference
  • —Cross-platform compatibility
  • —Direct loading in Transformers.js

Limitations

  • —FP16 quantization may cause slight precision differences compared to FP32
  • —Best results with clear, specific prompts
  • —Limited to Mermaid syntax (not general diagram description)

License

Apache 2.0

Acknowledgments