fffonion/xlm-roberta-ner-japanese-gguf

XLM-RoBERTa NER Japanese — GGUF

Japanese NER (Named Entity Recognition) model in GGUF format for efficient CPU inference via llama.cpp / Candle.

Model Info

• —Base model: tsmatz/xlm-roberta-ner-japanese
• —Architecture: xlm-roberta-base (12 layers, 768 hidden, 12 heads)
• —Task: Token-level NER for Japanese text
• —Labels: O, PER, ORG, ORG-P, ORG-O, LOC, INS, PRD, EVT

Files

Usage with Candle

let model = candle_ner::Model::load(&device, "xlm-roberta-ner-japanese-q8_0.gguf")?;
let input = "山田太郎は東京大学に通っている";
let entities = model.predict(input)?;

Labels

Quantization

FP16 → Q8_0 quantization done with llama.cpp:

./llama-quantize xlm-roberta-ner-japanese-f16.gguf \
    xlm-roberta-ner-japanese-q8_0.gguf Q8_0

References

• —llama.cpp
• —Candle NER example