BotResources/Infinity-Parser2-Flash-mlx-bf16

1---2license: apache-2.03library_name: mlx4base_model: infly/Infinity-Parser2-Flash5tags:6- mlx7- mlx-vlm8- ocr9- document-parsing10- vision-language11pipeline_tag: image-text-to-text12language:13- en14---15 16# Infinity-Parser2-Flash MLX BF1617 18This model was converted to MLX format from [`infly/Infinity-Parser2-Flash`](https://huggingface.co/infly/Infinity-Parser2-Flash) using mlx-vlm version 0.5.0. Refer to the [original model card](https://huggingface.co/infly/Infinity-Parser2-Flash) for more details on the model.19 20## Use with mlx-vlm21 22```bash23pip install -U mlx-vlm24```25 26The model is RL-tuned for the canonical layout-extraction prompt below — using a different prompt may yield unexpected output:27 28```bash29PROMPT=$(cat <<'EOF'30- Extract layout information from the provided PDF image.31- For each layout element, output its bbox, category, and the text content within the bbox.32- Bbox format: [x1, y1, x2, y2].33- Allowed layout categories: ['header', 'title', 'text', 'figure', 'table', 'formula', 'figure_caption', 'table_caption', 'formula_caption', 'figure_footnote', 'table_footnote', 'page_footnote', 'footer'].34- Text extraction and formatting:35  1) For 'figure', the text field must be an empty string.36  2) For 'formula', format text as LaTeX.37  3) For 'table', format text as HTML.38  4) For all other categories (e.g., text, title), format text as Markdown.39- The output text must be exactly the original text from the image, with no translation or rewriting.40- Sort all layout elements in human reading order.41- Final output must be a single JSON object.42EOF43)44 45python -m mlx_vlm.generate \46  --model BotResources/Infinity-Parser2-Flash-mlx-bf16 \47  --max-tokens 32768 --temperature 0.0 \48  --prompt "$PROMPT" \49  --image <path_to_image>50```51 52## Quantization quality53 54A companion 8-bit quantization is published at [`BotResources/Infinity-Parser2-Flash-mlx-q8`](https://huggingface.co/BotResources/Infinity-Parser2-Flash-mlx-q8).55 56In a **BotResources internal benchmark** of 50 pages from various PDFs (text, tables, formulas, scans), the BF16 build and the 8-bit build produced **byte-identical outputs on all 50 pages** at `temperature=0`, `top_p=1`. Token count, character count, and final text are strictly equal between the two builds.57 58On the same Apple M4 Max (128 GB unified memory) only the runtime differs:59 60| Build | On-disk | Peak RAM | Generation |61|---|---:|---:|---:|62| BF16 (this build) | 4.43 GB | 5.4 GB | 101 tok/s |63| 8-bit | 2.48 GB | 3.7 GB | 167 tok/s |64 65The 8-bit build is ~65 % faster per token and uses ~33 % less peak RAM, with no measured quality loss for this use case.66 67## License68 69Inherits the Apache-2.0 license from the base model [`infly/Infinity-Parser2-Flash`](https://huggingface.co/infly/Infinity-Parser2-Flash). All credit for the underlying model goes to the inflyAI team.70