PK9129/FineWeb

1---2title: FineWebGPT3emoji: 🧠4colorFrom: blue5colorTo: indigo6sdk: gradio7sdk_version: "6.9.0"8app_file: app.py9python_version: "3.10"10short_description: GPT-2 pretrained on 393M FineWeb-Edu tokens.11---12 13# FineWebGPT14 15FineWebGPT is a GPT-2 small language model pretrained from scratch on **393,216,000** FineWeb-Edu tokens over **12,000** training steps. This Space presents the model with a polished dark layout, full model metadata, creator details, and an FP32 inference pipeline.16 17## Model Summary18 19- Architecture: GPT-2 small20- Parameters: ~124M21- Training Type: From-scratch pretraining22- Dataset: `HuggingFaceFW/fineweb-edu` / `sample-10BT`23- Objective: causal language modeling24- Context Window: 1024 tokens25- Tokenizer: GPT-2 BPE26- Inference Precision: FP3227 28## What Is Included29 30This project is prepared for direct Hugging Face Spaces hosting with:31 32- `app.py`: Gradio Space app with the custom three-column layout33- `gpt2_inference.py`: GPT-2 architecture, checkpoint loading, tokenizer loading, and generation engine34- `space_state.pt`: smaller inference-only checkpoint35- `train_state.pt`: original full training checkpoint36- `tokenizer_assets/gpt2/*`: local GPT-2 tokenizer files37- `requirements.txt`38 39The app automatically prefers `space_state.pt` when it is present. If it is not present locally, the app can download it from a separate Hugging Face model repo by using `MODEL_REPO_ID`.40 41## Creator Section42 43The Space already includes a creator card for **Priyanshu Kumar** and uses the local git email for contact. If you want to customize the public links further before uploading, edit the config block near the top of `app.py` or set these Space variables:44 45- `CREATOR_GITHUB`46- `CREATOR_LINKEDIN`47- `CREATOR_PORTFOLIO`48- `CREATOR_HUGGINGFACE`49- `CREATOR_EMAIL`50- `CREATOR_BIO`51- `CREATOR_TITLE`52 53## Upload Notes54 55For the cleanest CPU Space upload:56 571. Create a new **Gradio Space** on Hugging Face.582. Put `space_state.pt` in a separate **model repo** instead of the Space repo if your Space hits the 1 GB storage cap.593. Upload only the Space app files and tokenizer assets to the Space repo.604. In the Space variables, set:61   - `MODEL_REPO_ID=your-username/your-model-repo`62   - `MODEL_FILENAME=space_state.pt`635. For a CPU Space, you do not need to set `MODEL_DEVICE` at all. The app will run on CPU automatically.646. If your model repo is private, add `HF_TOKEN` as a Space secret.65 66## Local Launch67 68Frontend:69 70```powershell71.\run_frontend_gpu.bat72```73 74CLI generation:75 76```powershell77.\run_generate_gpu.bat --prompt "The future of artificial intelligence is"78```79