ngocdang83/HachimiMT-40-QT

HachimiMT-40-QT

Tiếng Việt

Chinese → Vietnamese translation for xianxia, wuxia and historical web fiction, using the familiar QT/convert register (ta, ngươi, hắn, nàng…). The model has 40,436,352 unique learned parameters (41,026,176 including fixed positional tables).

Release: v1.0, 2026-09-21.

Architecture

Weight sharing reduces stored parameters, not the number of encoder computations. HF's 72.4M badge counts the serialized tensors in the standard Transformers export, which duplicates shared encoder weights across twelve passes. Compact files store each shared tensor once.

Design trade-off. HachimiMT-40-QT is built to shrink the download — about 42 MB as CTranslate2 INT8 or packed Q8, versus about 58 MB for HachimiMT-60/60-QT — while aiming to keep the translation quality of the 60M models. It does not reduce computation: the encoder runs 12 passes (8 layers in HachimiMT-60/60-QT, 6 in HachimiMT-30) with a decoder the same size as HachimiMT-60-QT. In one CPU test with the demo app (INT8, greedy) it took about 1.3× the time of HachimiMT-60/60-QT and about 1.9× that of HachimiMT-30. Choose it when download size or storage matters more than speed.

Files and variants

Only the weights are counted above; tokenizer/config/helper files add overhead. Q8/Q4 are storage formats: the helper expands them through FP16 and computes in FP32 by default. Expansion does not recover precision lost by quantization. They are not native INT8/INT4 kernels.

The QAT variant contains 37 Q4 Linear matrices, seven Q8 Linear exceptions and a Q8 shared embedding/LM head.

Quick start: compact FP16 on CPU

Download only the chosen representation instead of the entire multi-variant repository:

pip install "torch>=2.8" "transformers==5.8.0" "safetensors==0.7.0" "sentencepiece==0.2.1" "huggingface_hub>=1.0"

import sys
import torch
from huggingface_hub import snapshot_download

folder = snapshot_download(
    "ngocdang83/HachimiMT-40-QT",
    revision="v1.0",
    allow_patterns=[
        "hachimi40.py", "source.spm", "target.spm", "vocab.json",
        "tokenizer_config.json", "special_tokens_map.json",
        "target_normalization.json", "packed/f16/*",
    ],
)
sys.path.insert(0, folder)
from hachimi40 import load_compact, translate

torch.set_num_threads(2)
model, tokenizer = load_compact(folder, "f16")  # shared weights, CPU FP32
print(translate(model, tokenizer, ["他抬头看向远处的山门。"]))
print(translate(model, tokenizer, ["山门已开。\n弟子入内。"]))

The helper is ordinary Python source supplied with the release; inspect it before importing downloaded code. No Transformers trust_remote_code=True is required. For Q8, change both packed/f16/* and the loader variant to q8; for the experimental QAT artifact use qat-q4g32.

The root checkpoint also loads directly with AutoTokenizer.from_pretrained(...) and MarianMTModel.from_pretrained(..., dtype=torch.float16).float() on CPU. Load FP16 first, then upcast: Transformers regenerates Marian's fixed sinusoidal positions in the requested load dtype, so loading directly as FP32 would bypass the evaluated FP16 rounding of those tables. It contains FP16-rounded weights even when computed in FP32. retie_encoder(model) from the helper can restore physical core sharing after loading, and is required before continuing the intended tied-weight training recipe. The plain root model is inference-equivalent without this retie.

Standard Transformers checkpoint

import sys
from pathlib import Path
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer, MarianMTModel

repo, rev = "ngocdang83/HachimiMT-40-QT", "v1.0"
sys.path.insert(0, str(Path(hf_hub_download(repo, "hachimi40.py", revision=rev)).parent))
from hachimi40 import translate  # wraps generate() with encode_source / decode_target

tokenizer = AutoTokenizer.from_pretrained(repo, revision=rev)
model = MarianMTModel.from_pretrained(repo, revision=rev, dtype=torch.float16).float().eval()
print(translate(model, tokenizer, ["山门已开。\n弟子入内。"]))

Hugging Face's auto-generated pipeline("translation") snippet does not apply here: Transformers 5 no longer has that pipeline task, and calling generate() without the codec leaves a space after opening quotes (“ Ngươi…) and merges source lines into one.

Tokenizer and decoding contract

Do not substitute the 30-QT/60-QT tokenizer. <NL> is an ordinary retained SentencePiece user-defined piece, ID 4, not a Hugging Face special token.

• —Apply encode_source once before tokenization: source line breaks → <NL>.
• —Apply decode_target once after detokenization: <NL> → line breaks, then undo the training convention of one space after an opening quote/bracket.
• —config.json and target_normalization.json name this codec by its path in the training codebase (scripts/postprocess/joint_loop_codec.py), which is not part of this repository; encode_source and decode_target in hachimi40.py are the equivalent implementation.
• —Preserve source text and explicit names/titles. A different QT pronoun choice is not by itself a semantic error.
• —The reference helper uses greedy decoding, repetition penalty 1.05, no n-gram ban and 256-token caps. It rejects overlong sources instead of silently truncating and raises if generation reaches its cap without EOS. Split long text explicitly.
• —Chunk layout preservation is not guaranteed; check line alignment if the application relies on it.

CTranslate2: ct2-int8_float32/ (added on main after the v1.0 tag; weights unchanged) is an INT8 conversion of the same checkpoint, made with the corrected decoder start described below. Load it with ctranslate2.Translator, tokenize with the root source.spm/target.spm plus </s>, and apply the same encode_source/decode_target codec. It is the artifact used by the HachimiMT demo Space.

If you convert the checkpoint yourself: CTranslate2's ordinary Marian conversion can use a zero decoder-start embedding. This checkpoint requires the real <pad> embedding; an uncorrected conversion can change translations substantially.

Quality summary

Q8 stayed close to FP16 in internal CPU checks. QAT offers a smaller download but showed more semantic errors and omissions in some examples; FP16 or Q8 is recommended for regular use. All variants can make translation mistakes, especially with names and uncommon expressions. Native WebGPU quality and performance have not yet been measured.

License and attribution

CC BY 4.0, matching the HachimiMT-60-QT release policy. Attribute ngocdang83/HachimiMT-40-QT, link the license, and indicate modifications. See LICENSE.md. This license covers the released weights/helper, not a redistribution or relicensing of the training corpus.