Modelpublic

phuctahong/nanogpt-gpt-mha-gated-deltanet-chunk-triton-fla-130m-job4848354-step100000

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes33downloads

Model Card

phuctahong/nanogpt-gpt-mha-gated-deltanet-chunk-triton-fla-130m-job4848354-step100000

NanoGPT Pro checkpoint uploaded from the princeton-pli/Nanogpt W&B project.

Provenance

—W&B run: M_gpt-mha-gated_deltanet_chunk_triton_fla_130m_Opt_adamw_LR_0.001_D_fineweb-edu100B_T_49.15B_time_20260218_093136_jobid_4848354
—W&B run id: zeh5chg0
—Job id: 4848354
—Checkpoint step: 100000
—Local checkpoint: /scratch/gpfs/ARORA/st0490/nanogpt-next_mod/output/out_gpt-mha-gated_deltanet_chunk_triton_fla_130m_Opt_adamw_LR_0.001_D_fineweb-edu100B_T_49.15B_time_20260218_093136_jobid_4848354/checkpoint-100000

Files

This repo contains inference-ready weights only:

—config.json
—model.safetensors
—generation_config.json

Optimizer and trainer state were intentionally omitted.

Load locally

python

from nanogptpro.model_loader import load_model_from_checkpoint

model = load_model_from_checkpoint(
    model_id="<your-model-id>",
    checkpoint="phuctahong/nanogpt-gpt-mha-gated-deltanet-chunk-triton-fla-130m-job4848354-step100000",
)

Install nanogptpro and use the model id that matches the architecture recorded in config.json.