xpuenabler/gpt-oss-15.5b-23E-SFT

gpt-oss-15.5b-23E-SFT

A supervised fine-tuned (SFT) variant of a gpt_oss Mixture-of-Experts model, specialized for science-domain reasoning. The model is a pruned/customized configuration with 23 local experts (top-4 routing) and approximately 15.5B total parameters.

Model details

See config.json for the full configuration.

Training

This checkpoint corresponds to step 1175 of an SFT run with the following setup:

• —Base: gpt_oss 23-expert variant
• —Data: ~30k science / reasoning samples
• —Epochs: 5
• —Global batch size: 128
• —Learning rate: 1e-5
• —Precision: bfloat16

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "xpuenabler/gpt-oss-15.5b-23E-SFT"

tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Explain why the sky is blue."},
]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

A custom chat_template.jinja is bundled with the tokenizer and is applied automatically by apply_chat_template.

Files

• —model.safetensors — model weights (bfloat16, ~31 GB)
• —config.json, generation_config.json — model / generation configuration
• —tokenizer.json, tokenizer_config.json, special_tokens_map.json, chat_template.jinja — tokenizer assets

Intended use & limitations

This is a research checkpoint intended for science-oriented reasoning experiments. It has not been aligned for safety beyond what was inherited from the base model and the SFT data, and may produce inaccurate or unsafe outputs. Evaluate carefully before any downstream use.

License

Released under the Apache 2.0 license, consistent with the upstream gpt-oss family.