Modelpublic

agaralon/talkie-1930-13b-base-vllm

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes27downloads

Model Card

talkie-1930-13b-base (vLLM-servable repackage)

This is a HuggingFace + vLLM-ready repackage of `talkie-lm/talkie-1930-13b-base`. The original release ships as a raw torch checkpoint (final.ckpt) plus a tiktoken vocab (vocab.txt), with no config.json, tokenizer.json, or HF modeling code, so it can't be loaded by transformers or served by vLLM out of the box.

talkie-1930-13b-base is a 13B parameter base (completion) language model from the talkie-lm project, pretrained on ~260B tokens of pre-1931 English text. This is the base model — it is not instruction-tuned and has no chat template; prompt it with raw text and let it continue.

What this repo adds

File	What it is
`model.safetensors`	bf16 weights, ~26 GB. `lm_head_gain` (a learned scalar) is pre-multiplied into `lm_head.weight` so vLLM's transformers backend doesn't need to know about it.
`config.json`	`TalkieConfig` (vocab=65536, hidden=5120, 40 layers × 40 heads, headdim=128, ctx=4096, RoPE θ=1e6) plus `automap `for` AutoConfig`/`AutoModel`/`AutoModelForCausalLM`.
`tokenizer.json`, `tokenizer_config.json`	HF fast BPE built from the original `vocab.txt` (ranks < 65535) with `<	endoftext	> `at id 65535. EOS = pad =` <\	endoftext\	>`. No chat specials (base model).
`generation_config.json`	`eos_token_id=65535`, `pad_token_id=65535`.
`modeling_talkie.py`, `configuration_talkie.py`	HF `PreTrainedModel` implementation with `ALL_ATTENTION_FUNCTIONS` dispatch (vLLM transformers-backend compatible).

The architecture and tokenizer BPE ranks are identical to the IT repackage; the only differences are vocab_size (65536 vs 65540), the absence of the 4 chat special tokens, and no chat template.

Serving with vLLM

bash

vllm serve agaralon/talkie-1930-13b-base-vllm \
    --model-impl transformers \
    --trust-remote-code \
    --dtype bfloat16 \
    --max-model-len 4096

Then hit it as an OpenAI-style completions endpoint (base model — use /v1/completions, not chat):

bash

curl http://localhost:8000/v1/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "agaralon/talkie-1930-13b-base-vllm",
        "prompt": "In the year 1925, the great city of London",
        "temperature": 0.8,
        "max_tokens": 120
    }'

Sampling notes

—Use `temperature ≥ 0.5` — greedy decoding (temperature=0) can collapse into single-token loops on this architecture.
—bf16 only — fp8 is broken on this architecture.

Plain HuggingFace usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tok = AutoTokenizer.from_pretrained("agaralon/talkie-1930-13b-base-vllm", trust_remote_code=True)
m = AutoModelForCausalLM.from_pretrained(
    "agaralon/talkie-1930-13b-base-vllm", trust_remote_code=True, dtype=torch.bfloat16,
).cuda().eval()

ids = tok(["In the year 1925, the great city of London"], return_tensors="pt").to("cuda")
out = m.generate(**ids, max_new_tokens=120, do_sample=True, temperature=0.8, top_p=0.9,
                 pad_token_id=tok.pad_token_id, eos_token_id=tok.eos_token_id)
print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True))

Provenance

—Weights: final.ckpt from talkie-lm/talkie-1930-13b-base, with lm_head_gain.w_g baked into lm_head.weight and cast to bf16.
—Tokenizer: built from vocab.txt from the same release (ranks < 65535, then <|endoftext|> at id 65535).
—Modeling code: adapted from the IT repackage `awilliamson/talkie-1930-13b-it-vllm`.

License

Apache 2.0, matching the upstream talkie-lm/talkie-1930-13b-base release.