Modelpublic

CodeStrux-Tech/tac-1-fp8-dynamic

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes10downloads

Model Card

tac-1-fp8-dynamic — FP8 dynamic quantization of tac-1

Overview

tac-1-fp8-dynamic is quantized from `CodeStrux-Tech/tac-1` using llmcompressor 0.12.0 with the FP8DYNAMIC scheme. Targets are `Linear` layers; `lmhead` is ignored. The result is a single 4.2 GB shard.

sha256: e0eb5c3c62e8a472eaca6f86064efb2597863f0688ac3ca1fd9a325390d9c30f

Serving

bash

vllm serve CodeStrux-Tech/tac-1-fp8-dynamic --served-model-name tac-1 --max-model-len 4096
# On hosts without nvcc, disable the FlashInfer sampler JIT:
#   VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve ...

No --quantization flag is needed — vLLM auto-detects the FP8 format from the model config. The FP8 model is a drop-in replacement for the bf16 original; the client snippet from `CodeStrux-Tech/tac-1` works unchanged:

python

from tico.extractor.clients.vllm import VLLMExtractor

extractor = VLLMExtractor()  # VLLM_BASE_URL=http://localhost:8000/v1, TICO_VLLM_MODEL=tac-1
addr = extractor.extract("del antiguo higuerón de San Pedro, 100 metros sur")
print(addr.model_dump_json(indent=2))

Evaluation

metric	tac-1 (bf16)	tac-1 (FP8)	teacher gpt-4.1
parse rate	1.000	1.000	1.000
anchor_name F1	0.979	0.979	0.872
direction F1	0.995	0.984	1.000
distance F1	0.995	0.984	1.000
perfect-anchor ≤250 m	93.6%	93.6%	93.6%
perfect-anchor median	0.0 m	0.0 m	0.0 m
gazetteer ≤250 m	80.9%	80.9%	83.0%

Training data attribution

Contains information from OpenStreetMap (https://www.openstreetmap.org/copyright), which is made available under the Open Database License (ODbL) 1.0. © OpenStreetMap contributors.

For full training details, architecture, evaluation, and limitations, see `CodeStrux-Tech/tac-1`.