phuctahong/nanogpt-gpt-mha-falcon-1a-ctxeta-ctxlambda-chunk-triton-379m-job12935991-step100000

phuctahong/nanogpt-gpt-mha-falcon-1a-ctxeta-ctxlambda-chunk-triton-379m-job12935991-step100000

NanoGPT Pro checkpoint uploaded from the princeton-pli/Nanogpt W&B project.

Provenance

• —W&B run: M_gpt-mha-falcon-1A-ctxeta-ctxlambda-chunk-triton_379m_Opt_adamw_LR_0.001_D_fineweb-edu100B_T_49.15B_time_20260826_081604_jobid_12935991
• —W&B run id: p3u6iopc
• —Job id: 12935991
• —Checkpoint step: 100000
• —Local checkpoint: /scratch/gpfs/JHACKL/nanogpt-next_mod/output/falcon_lambda_challengers/f1a2_l001_05_s09/out_gpt-mha-falcon-1A-ctxeta-ctxlambda-chunk-triton_379m_Opt_adamw_LR_0.001_D_fineweb-edu100B_T_49.15B_time_20260826_081604_jobid_12935991/checkpoint-100000

Files

This repo contains inference-ready weights only:

• —config.json
• —model.safetensors
• —generation_config.json

Optimizer and trainer state were intentionally omitted.

Load locally

from nanogptpro.model_loader import load_model_from_checkpoint

model = load_model_from_checkpoint(
    model_id="<your-model-id>",
    checkpoint="phuctahong/nanogpt-gpt-mha-falcon-1a-ctxeta-ctxlambda-chunk-triton-379m-job12935991-step100000",
)

Install nanogptpro and use the model id that matches the architecture recorded in config.json.