NodeNestor/bifrost-1.2b

Bifrost 1.2B

A from-scratch 1.2B-parameter translation model for the Nordic languages ↔ English: Swedish (sv), Danish (da), Norwegian Bokmål (nb), Norwegian Nynorsk (nn), Finnish (fi), and Icelandic (is), plus cross-Nordic directions.

On FLORES-200 devtest it beats NLLB-200-3.3B and TranslateGemma-12B on the English→Nordic average — at a fraction of their size.

The teacher of the Bifrost Nordic-translation family from NodeNestor — for a ~3× smaller/faster distilled option see Bifrost Flash 430M.

Results — FLORES-200 devtest, chrF++ (sacrebleu, word_order=2, n=500)

Headline — English→Nordic average:

Group averages:

Per-direction (chrF++):

Strongest relative to the references on the low-resource directions (Nynorsk, Icelandic). NLLB-3.3B still leads on several →English directions and Finnish.

Usage

The model expects a control-token prompt and is decoded greedily:

[BOS] [<2{tgt_lang}>] {source_token_ids} [<eos_src>]   →   generate until [EOS]

The target-language control token placed right after `[BOS]` selects the output language — the source language is inferred. Control-token IDs (above the 65000 SentencePiece vocab):

[BOS]=1, [EOS]=2. Tokenizer: nordic_unigram_65k.model (SentencePiece, 65000 pieces + 8 control tokens = vocab 65008).

The weights ship as model.safetensors, with a self-contained pure-PyTorch implementation in modeling_nordic.py (no training-stack dependencies). Three ways to run it:

1. Standalone (pure torch, KV-cached):

import torch, sentencepiece as spm
from modeling_nordic import NordicTranslator

sp = spm.SentencePieceProcessor(); sp.load("nordic_unigram_65k.model")
LANG = {"en":65000,"sv":65001,"da":65002,"nb":65003,"nn":65004,"fi":65005,"is":65006}

model = NordicTranslator.from_checkpoint("model.safetensors", device="cuda")
ids = model.translate(sp.encode("Hello, how are you?", out_type=int), LANG["sv"])
print(sp.decode(ids))     # -> Hej, hur är det med er?

2. HuggingFace (`trust_remote_code`):

from transformers import AutoModelForCausalLM
import torch, sentencepiece as spm
sp = spm.SentencePieceProcessor(); sp.load("nordic_unigram_65k.model")
m = AutoModelForCausalLM.from_pretrained(".", trust_remote_code=True,
                                         dtype=torch.bfloat16).cuda().eval()
ids = [1, 65001] + sp.encode("Hello, how are you?", out_type=int) + [65007]   # 65001=<2sv>
out = m.generate(torch.tensor([ids]).cuda(), max_new_tokens=128, do_sample=False, eos_token_id=2)
print(sp.decode([t for t in out[0, len(ids):].tolist() if t < 65000]))

3. vLLM (custom architecture — register the included plugin): see vllm_nordic.py + vllm_pkg/ and example_vllm.py. Install the plugin (pip install -e vllm_pkg) inside a vLLM environment, then serve with --skip-tokenizer-init and feed control-token prompts.

The control-token prompt is [BOS] [<2{tgt}>] {source_ids} [<eos_src>] → generate until [EOS]; decode only ids < 65000. The FLORES numbers above were produced with the batched, KV-cached standalone path.

Model details

• —Architecture: a grouped-query-attention (GQA) decoder. 18 layers, hidden 2048, FFN 6144 (SwiGLU), 16 query heads / 4 KV heads, head dim 128, RoPE (θ=500000, partial 0.25), RMSNorm, parallel residual, fused QKV. ~1.2B params.
• —Context length: 4096 tokens (trained and evaluated at 4096; longer inputs truncate).
• —Precision: bf16.
• —Vocab: 65008 (nordic_unigram_65k SentencePiece + 8 control tokens).

Training

• —From scratch. A ~120B-token run: a ~19B-token trunk, then +~100B tokens of continued training (clean data, cosine schedule with a monolingual floor + anneal). The released checkpoint is ~96B into the long run (~115B cumulative) — on the cosine tail, so quality ≈ the 100B point.
• —Data: parallel + monolingual Nordic/English (Wikipedia parallel, DCLM en↔Nordic, Aya cross-lingual, FineWeb-Edu, Nemotron-CC), balanced en↔Nordic blend.
• —Objective: next-token cross-entropy on the target side.

Limitations

• —Trained at 4096-token context; longer inputs are truncated.
• —Finnish and Icelandic (en→) are the weakest directions — lower-resource, morphologically hard.
• —Greedy decoding; no built-in length/formatting control beyond the prompt.
• —Not instruction-tuned — it is a dedicated translation model, not a chat model.
• —May produce occasional off-target output on the hardest low-resource pairs.

Acknowledgments

• —Tokenizer (nordic_unigram_65k) developed by a collaborator; included here with permission.

Citation

@misc{nodenestor_bifrost_1.2b_2026,
  title  = {Bifrost 1.2B},
  author = {Nilsson, Ludvig},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/NodeNestor/bifrost-1.2b}},
  note   = {NodeNestor}
}